Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

In which Thread is a blocking socket running? what happen if we share them in QRunnables of a QThreadPool?



  • Hi
    I've my main Thread (other than the GUI one) that is delegating some computation Tasks to a QThreadPool in a lazy evaluation framework: the main thread can continue doing stuff but will wait on the result of a Task if we need that result right now and that the Task is not already finished (using a QWaitCondition).

    I would like to be able to delegate the computation work on a server and so that my Tasks (running in a QThreadPool) would use some existing Sockets. As there are many computations, I'm thinking to instantiate as many socket connections as the size of my ThreadPool. They would stay alive for the whole time of the program. So basically each Runnable would have the exclusivity of a Socket and thus don't need to use a Mutex to share a unique connection with all the other Runnables. (This is for performance issues and also not opening a new connection for each QRunnable)

    So a QTcpSocket would be used by several QRunnables but one after each other. I'm thinking to have a QStack of QTcpSockets of the size of the QThreadPool and each QRunnable would pop one when it is running.

    So how would it work?
    My QRunnables would first write in a QDataStream on the Socket. Would that be done in the thread of the Runnable?
    It would then waitForReadyRead for a result, would this happen in the thread of QRunnable?
    I've a big doubt about that. As the sockets (blocking mode) are created and owned by the main Thread and then just lent to some QRunnable for write/read operation. I've a feeling that everything would happen in my main thread. Is that right?



  • anybody has any ideas on this? I'm going to have to implement this solution soon, I'd be happy to have some other point of views before starting :)
    Especially if I need or not to instantiate a special Thread where all the Sockets would be moved to make sure that they don't run in the event loop of my "mainThread". Or if it is ok, all the interaction would be synchronous and happen in the Runnable using it, so in a Thread of my ThreadPool.
    Let me know if I need to explain more the question or use case.
    Cheers!


  • Lifetime Qt Champion

    Hi,

    You have to move your socket to the thread that will be using it, it should then be used in that context.

    For something to run in the new thread context, it has to either be created in a reimplementation of the run method or be moved to the new QThread object (worker object approach).



  • ok that is what I thought.
    so there is no way to make a shared socket work in a QRunnable... that's a shame :'(
    I suppose I will dedicate a QThread for all my shared socket then and use blocking calls in the QRunnable that will use them.



  • Hello,
    last question before I start my dev, I've another doubt on how to proceed.
    I've understood that it is not possible to have the QTcpSockets running in blocking way in the Thread of a QRunnable.

    If I create a special QThread for my all my TCP connections (I will have around 30). I will move all my keep alived QTcpSockets in that Thread.
    They will be used by the QRunnables of a QThreadPool in a blocking way preferably using a QDataStream to send an Object and then read one in return.
    So if I do like in this post (first comment )and put sending method with the "waitForBytesWritten()" in my QRunnable, will this work and only block the QThread of the QRunnable or will it block the special QThread where all my sockets lives?

    I see in the doc that waitForBytesWritten can fail on windows (https://doc.qt.io/qt-5/qabstractsocket.html#waitForBytesWritten)
    What would be a portable way to make it? if possible in a non blocking way.
    I've my QRunnables that need to send an Object request and then wait for the answer, another Object result. The QRunnable would have an handle on a QTcpSocket that will live in its own QThread (the special one with several other sockets)
    Should I maybe first emit the request Object with a signal to the Socket and then wait on QWaitCondition that the Socket will notify once it has sent the request and fully received the result?
    Would that be the way to go?
    how can I know that the request is fully sent without a waitForBytesWritten? (I can't block the QThread holding all my Sockets)
    Idem, how can I know that the answer is fully received?

    PS: the object I need to send on the Socket (via a DataStream) is mainly composed by few int, QStrings and more importantly a QMap<int, double>.
    Do I need to send some kind of headers first with the size or will QDataStream handle correctly the QMap with a variable size?


  • Lifetime Qt Champion

    @mbruel said in In which Thread is a blocking socket running? what happen if we share them in QRunnables of a QThreadPool?:

    how can I know that the request is fully sent without a waitForBytesWritten?

    My experience says, you can never know if a request is fully sent out on computers with operating systems, as the OS buffers the data. This applies to TCP, UDP, serial ports and also CAN bus. waitForBytesWritten only tells you, that Qt has handed the data over to the OS.

    The only relyable way is to use a protocol that answers each command, so you know the data has arrived on the other side (and probably is already checked for consistency).

    Regards



  • @aha_1980
    Well I'm really sending just one object and expecting another one in response.
    This is kind of a simple protocol... I don't really need to know that the request is fully sent, fully receiving the answer is enough.

    I just need to make sure QDataStream will know when my Object (containing a QMap with a variable size) fully received. Is it the case? The documentation says QDataStream handles QMap.

    I also need to make sure I don't block the QThread where several Sockets lives.

    PS: I'll be on a secured local network so I rely totally on TCP for the data consistency. Will the QDataStream manages errors? Well I suppose TCP do the work with the checksum of the packets. If a packet is corrupted it will ask to be sent again no?


  • Lifetime Qt Champion

    @mbruel said in In which Thread is a blocking socket running? what happen if we share them in QRunnables of a QThreadPool?:

    so I rely totally on TCP for the data consistency.

    The whole internet does, so why not :)

    Will the QDataStream manages errors?

    Why don't you try it yourself? My guess (because I really don't know) is, it will take care that the data sizes are correct, but it surely cannot know the contents of your data.

    Well I suppose TCP do the work with the checksum of the packets. If a packet is corrupted it will ask to be sent again no?

    It will, but remember: it's just a checksum, no CRC. I'd recommend you to add a CRC to your data to be on the save side.

    Edit: just for completeness: the underlying layers like Ethernet have checksum mechanism too, so data corruption should be very rare. However, there is a reason that download services provide md5sum files; and sometimes you have a corrupted download which could not be detected otherwise. So there is still a reason for CRC on user side.

    Regards


Log in to reply