Nominate our 2022 Qt Champions!

[QtNetwork] Reading and writing data

  • Is it neccessary to write size of the message in the front of it (as it's written in fortune client example)? Does it mean that I can't be sure if the whole message arrives at once or not? In other words, after readyRead signal I have to check if I get the whole message or just a part of it, right?

    I wonder because I didn't do that and everything worked fine (the reason may be the small size of data I sent).

    The second question is about the security. Could somebody give me some advice what I should pay attention to? Does QtSocket have any build-in protection?

    Sorry for these stupid questions but I'm quite newbie when it comes to networking.

    Thanks for all help :)

  • The size-in-front is really convenient since you know beforehand how much buffers you need to reserve. This is especially important with C code which is very annoying to write when you have to do variable length data. In C++ it is mostly a performance optimization since e.g. QByteArray makes it really easy to just continuously append data to a buffer. That size is part of your client side protocol and is completely ignored by the networking code.

    A socket is just a stream of bytes. There is no protection of any kind with regards to what those bytes mean. That is completely up to the applications that use the stream (which makes it such a success;-). The size of the following string is such a application specific detail: You need to verify that since anybody might send you some short number and then lots of data... a socket will not do anything to protect you from that: In fact it can't since it does not know that you happen to interpret some bytes as the number of characters in a string.

    You will need to take care of byte order issues if you want to have arbitrary computers talk to each other. Little endian machines are the norm nowadays, but there still are some big endian ones out there:-)

  • That depends what you are trying to transport from one end to the other.

    If the data content is self-containing you do not need to add the number of bytes you are sending. The length of a data message or a block is typically an integral part. Otherwise you need other means to detect when your data is completely received. A special character sequence is a possibility.

    The answer for the size of information sent can be found in the fortune server. The server sends a block of data contained in a byte array. The write sends only the content of the QByteArray. In order to know how much to expect the size has been sent before the data.

    "The Fortune Server ": is using TCP/IP for communication. TCP/IP does some monitoring of data transfer.

    If you send once in a while some data, let us assume an array of 10 integers, you can just assume at the receiving side that the data may sequentially, but not necessarily at once. Slow communication may cause that not all data has been received already when the readyRead() signal has been triggered. You need to loop and collect the data until the transmission is finished.

    Even so, this is possible it is typically not advisable to send just plain data such as double or int and so on. You need to form messages which probably will have some implicit or explicit information of their size.

  • Reading/Writing over and SSL Socket is a good start for security, it may also be a good idea to encrypted the data before you send it and decrypt it on the other side so you can use it.

  • Thanks for the answer, especially koahnig's one :) That's what I wanted to know. I didn't realise that fact probably because of fast connection and small message size - packages always arrived at once. Now I know that it's really necessary to add information about their size.

    When I wrote about security I meant protection from potential attack. I think it's possible that somebody could send self-made package with invalid data in order to crash the server. Where can I found any information how to prevent that (or just try).

  • [quote author="Baca48" date="1309974375"]Is it neccessary to write size of the message in the front of it (as it's written in fortune client example)? Does it mean that I can't be sure if the whole message arrives at once or not? In other words, after readyRead signal I have to check if I get the whole message or just a part of it, right?[/quote]

    There actually two reasons about why that example was coded that way, and they're somehow related:

    QDataStream has no means to recover from short reads: all the data it needs to successfully decode an object must be available when you use the operator>> to deserialize the object. Therefore, you cannot use it before being sure that all data was received. Which brings us to:

    TCP has no built in mechanism for separating data in "records". You can't just send some bytes followed by a "record marker" which tells the receiver that he has received all the data pertinent to a record. What TCP provides is a raw stream of bytes. Eventually, you can (half-)close the connection to signal the other peer that the transmission is over.

    1+2 imply that you must use some other mechanism to know (on the receiver side) if you already have all the data you need or you must wait for some more. For instance, you can introduce in-band markers like "\r\n" (like IRC or - up to a certain degree - HTTP do).

    The solution in the fortune example is prepending to the "actual" data (the serialized QString with the fortune message) the length, in bytes, of that data; then it sends the length (as a 16 bit integer) followed by the data itself. The receiver first reads the length; then it reads up that many bytes, then it knows it can decode the fortune. If there's not enough data available (both for the length - i.e. you received less than 2 bytes - and the payload itself) the client simply does nothing and waits for more.

    Notice that:

    • the design ain't new: it's what all most protocols do. In the "standard" TCP/IP stack, TCP, IP, Ethernet and so on all have a field in their "headers" which specify the lenght of the payload (or of the whole "record");
    • the transmission of the "length" uses a 16bit unsigned integer sent in a specific byte order: it's not memcpy()d into the buffer, but QDataStream is used on it to both store it and read it back. Although it may seem trivial, this actually completes the definition of the protocol you're using.
    • if QDataStream had been able to recover from short reads (f.i. by throwing an exception and leaving the data in the device), you would not have needed to send the length of the payload, since QDataStream already sends the length of the string (as a 32 bit unsigned bigendian integer) followed by the UTF-16 chars.

  • Thanks for a very long and nice explanation. I have to admit that this forum is probably the best forum I know :) My friend's given me some links about security and I hope they'd answer my question how I can protect from potential attacks.

    Topic can be closed. Thx again!

Log in to reply