This page summarizes how data is transmitted between client and server, on a low level / network point of view. It is there to enable easy extension of code, which requires a good understanding of how all this works, and also to describe the low-level protocol, so that a client can be easily implemented using another language or another compiler.
For applicative / protocol information, see Communication protocol.
The principles of data exchange are similar for both client and server, so in the following text we will use the usual symbols Tx to designate the "Transceiver" of data (the "sender") and Rx to designate the "Receiver" of data.
Both client and server hold a reception buffer, allocated with a fixed number of bytes, through RX_BUFF_SIZE. This value is the same for client and server, has some functions using that value for checking are used in both. It is impossible to send/receive a data frame longer than this value. This value can be changed at build time. To make sure everything works seamlessly:
The client will always be the one that initiates a data exchange, and the server is expected to always respond with data (even as simple as an "acknowledge" message). If the server does not respond, this means that either there is a loss of network capabilities, either it crashed. At present, the two demo clients do not handle this, and they might just sit and wait forever. The server handles this: in case the client doesn't communicate in a given amount of time (once a navigation session is initiated), the server will send a "blind" error message and return to initial state.
When sending data, a buffer is build, and then passed to the boost::asio sending code, using a synchronous method. For reading data, all the user code (server or client) uses readSocketData(). This function will fill a buffer (passed as argument) with the data read from distant machine. This function also gathers some stats on how the Rx element handles the reception (how many call to the socket read operation, etc.). This data sits in NetworkRxStats and the client can fetch its value on the server with a message prot::MT_ServerStatsRequ (see state fsmn::SS_ServerDataRequ).
Every frame is build from an data type inherited from prot::NetwData. Each frame holds the same header, with additional data added after these, depending on the nature of the message. They follow the same format, whatever the direction (server to client or client to server). To avoid compiler padding, the objects are converted to and from raw bytes with prot::NetwData::fillFromBuffer() and prot::NetwData::copyToBuffer() and the inherited functions.
Endianness: Big Endian (low order first). For example the integer value "1" stored on a uint32_t
will have the layout 01 00 00 00
. This can be checked by running bin_test/test_data_types_1
(after make test
).
Each of these types holds a (virtual) printing method, prot::NetwData::printData() that can be used to show relevant data value.
Format:
Each frame has a header of 27 bytes. It is build from the abstract type prot::NetwData.
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+----- ~~~ --+----+ | | | | | | | | A | B | C | D | E | F : (16 bytes) | | | | | | | | +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+----- ~~~ --+--- +------ 0 1 2 3 4 5 6 7 8 9 10 11 12 ~~~ 26
Description:
Size | Description | Data type | |
---|---|---|---|
A | 1 | Message type | prot::En_MessageType |
B | 1 | Data type | prot::En_DataType |
C | 4 | total length of the frame | uint32_t |
D | 4 | frame index, as set by the sender | uint32_t |
E | 1 | current state of server FSM (or -1 / 0xFF if frame send by client) | int8_t |
F | 16 | 16 bytes long null-terminated string of state (defined by state_strings) | char * 16 |
The full frame has additional bytes, depending on the data type sent.
Each of the data types is identified by a corresponding value of type prot::En_DataType in the above header:
Enum value | Data type | Additional bytes | Description | |
---|---|---|---|---|
1 | DT_SimpleData | prot::ND_SimpleData | 0 | used for Ack. |
2 | DT_InitialContact | prot::ND_InitialContact | 130 | - uint16_t: agent Id- 2 strings of 64 bytes, to store whatever meta information is needed |
3 | DT_ICR | prot::ND_IC_Response | 72 | - 2 x uint32_t (nb of edges and nodes)- sizeof( GraphProperties ) (64) |
4 | DT_DiffInfo | prot::ND_DiffInfo | 4 | - uint16_t: Nb of images following- uint_8: Nb of planes per image- uint_8: Nb of additional text files transmitted (see Current Work In Progress) |
5 | DT_ErrorData | prot::ND_ErrorData | 2 | - En_ErrorType - En_ErrorSeverity |
6 | DT_ImageData | prot::ND_ImageData | 160 + variable | (see 3.2 - Image Data) |
7 | DT_NavRequest | prot::ND_NavRequest | 33 | - 2 x Location (2 double )- uint8_t: Nb images |
8 | DT_PathInfo | prot::ND_PathInfo | 6 + variable | - 2 (uint16_t): nb of spheres - 4 ( float ): total distance- + 16 per sphere location (Location) See 3.3 - Path information |
9 | DT_SphRequest | prot::ND_SphRequest | 3 | - uint8_t: image plane (RGB/depth/labels)- uint16_t: sphere index |
10 | DT_ServerStats | prot::ND_ServerStats | 76 | See NetworkRxStats |
The table below shows the size of datatypes generated at runtime, with test_data_types_1.cpp with command-line option -auto
Any discrepancy between the above and below table should be carefully considered !
Datatype | Total length | Additional bytes | Comment | |
---|---|---|---|---|
1 | ND_SimpleData | 27 | 0 | |
2 | ND_InitialContact | 157 | 130 | |
3 | ND_IC_Response | 99 | 72 | |
4 | ND_DiffInfo | 31 | 4 | |
5 | ND_ErrorData | 29 | 2 | |
6 | ND_ImageData | 274 | 247 | |
7 | ND_NavRequest | 60 | 33 | |
8 | ND_PathInfo | 81 | 54 | path of 3 locations |
9 | ND_SphRequest | 30 | 3 | |
10 | ND_ServerStats | 99 | 72 |
Every transmission of an image is done using the prot::ND_ImageData type. The image itself is transmitted as a PNG format data chunk. This ensures that we benefit from the lossless compression inherent to the png format. Besides the general header, this data type holds some additional members, followed by the image bytes. These members are:
uint32_t
(4 bytes)int32_t
(4 bytes). This is useful when sending some data back, from client to server. The client will send some difference data for each sphere, thus the datatype needs to hold the sphere index. In case the image is not a sphere (for instance, if it is related to 2.2 - Image-based navigation request), then this value will be -1.Thus the size of the header is 27 (common header) + 133 = 160 bytes The png image itself follows these 3 members.
This data structure is used in three situations:
Related functions:
The transmission of the computed path from server to client is done using a prot::ND_PathInfo data type. If the path has \( n \) steps (including starting sphere and destination sphere), it is followed by \( n\) objects of type Location, that are just a pair of double
, thus it has a length of 2x8=16 bytes. The coordinates of each of the spheres are - added to / read from - the data frame, using the functions below.
Related functions: