[SOLVED] Qt Network (Qt 5.0.1) : Sending a huge data set with QTcpSocket



  • Hello,

    I'm programming a custom dynamic library which will allow me to use QTcpSocket with my custom parameters without retyping them every time I need it. The joys of "anticipated laziness"... However, a question came up to me; let me explain it.

    A user has to send a few mebibytes of data to a remote computer. Both the sender and the receiver use a program which connects them to the network through a QTcpSocket. Because of TCP/IP protocol set and because of our current technology, data won't be sent and received instantaneously; actually, data will progressively be transmitted and this operation may take several minutes.

    My question is: how will each of both QTcpSockets behave in this case? Will sending socket's signal "bytesWritten" be emitted only when the whole data set has been sent or every time a packet has been transmitted? Will receiving socket's signal "readyRead" be emitted only when the whole data set is available for reading or every time a packet has been received? Am I forgetting something important for this case?

    Thank you in advance for your help.

    Adishatz!



  • Although the TcpSocket can be used to "stream" data it is still chopped up into blocks and you will get a readyRead() once there is a TCP packet available. The maximum TCP packet size is something like 64kb so anything larger will be chopped up (well, I don't think you can send bigger then that - I have not tried).

    Each time a readyRead is emitted you need to read that data out and handle it and perhaps reconstruct your data at the far side (if it is not all contained in the single block).

    So, certainly your "few mebibytes" will need to be sent over multiple blocks / packets and you will receive it in parts.

    If you want to see what you have to deal with then you can use something like Wireshark (a network traffic monitoring tool) which is free to download and easy to use. If you have not already got it (or similar) I would highly recommend it as an aid to any network development.



  • Thank you for your answer, code_fodder.

    If I have well understood, my huge data set will be chopped into packets and each time a packet is transmitted:

    • "sender-side", the signal "bytesWritten" will be emitted with, as argument, the size of this packet;
    • "receiver-side", the signal "readyRead" will be emitted and only the received packet(s) is/are available for reading.

    In this case, I should have the data set chopped (and even encapsulated) into packets upstream of the data transfer and include alongside each socket a "receiving buffer" which will permit an easier data reconstruction.

    Have I been right, so far?



  • Yes I think that about covers it :)

    To be honest, I never actually tried to write a "huge" block in one go! I am now thinking maybe, your QTcpSocket will be able to buffer quite a lot, but as I say its a bit untested area from my point of view.

    However so long as you do the following:

    Sender Side:
    I usually go with QByteArray's as they are convenient for me so I don't have to worry about specifying the size. However either way you should get a bytesWritten emitted, you can check that all is well using that.
    You could setup a timer (QTimer is easy to use) tor even waitForBytesWritten() if you are worried about the data not getting there in time...

    Receiver side:
    Service the readyRead() signal and check for the amount of data using bytesAvailable() and then you can read up to that amount of data out (or less if its convenient). The buffer only empties as you read out from it. But each time a new chunk of data arrives you will get another readyRead().

    Note: Keeping your chucks of data small-ish is a good idea sometimes to that you can constantly monitor the "health" of your data transmission. I.e.



  • Ok! I think I get it. So, thank you for your help. ;)

    Adishatz!



  • no problem, good luck :)


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.