Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Make async readyRead() calls with QTcpServer/QTcpSocket in combination with QEventLoops



  • Hi

    I have a Qt based module that can connect to a HTTP server like a special proxy.
    The proxied data could be accessed by a local HTTP server interface using QTcpServer / QTcpSocket or a virtual QIODevice.

    For each request that comes in to the local HTTP server or the QIODevice's read()-method it will retrieve data from a remote HTTP server. It will download data from the server, intercept the data and provide it to the local TCP socket listening on localhost. The second access interface via the QIODevice does the same - allows accessing remote HTTP data like a local file (read, seek, ...).

    The QIODevice approach needs to retrieve the data and therefore the read()-method must blocking until the requested data amount was fully retrieved.
    Especially for the virtual QIODevice approach inside the readData() method a QEventLoop.exec() will be called. This event loop will be exited after all the requested new data is available.

    Here are some simplified implementation specific details:

    There's a shared QNetworkAccessManager

    QSharedPointer<QNetworkAccessManager> getNetworkAccessManagerStream();
    

    For each incoming request the QTcpServer provides a QTcpSocket that will be connected to the readRequest() slot

    connect(socket, SIGNAL(readyRead()), this, SLOT(readRequest()), Qt::QueuedConnection);
    

    The readRequest() slot will retrieve the remote data, and act as special proxy that intercepts the data

    readRequest() {
      Remotenet::readData();
    }
    

    It try's to get data remotley via HTTP

    Remotenet::readData() {
      QNetworkReply *networkReply = getNetworkAccessManagerStream()->get(networkRequest);
      connect(networkReply, SIGNAL(readyRead()), this, SLOT(readStreamData()));
    }
    
    Remotenet::readStreamData() {
      QByteArray datagram = networkReplyStream->read(readLen);
    }
    

    Everything works fine as long there are no concurrent requests. If there are concurrent requests, the active request will paused until the newer request was finished. That happens while one of the QEventLoops are executed.

    I have researched and thought about it, and found one special part in the QT doc:
    readyRead() is not emitted recursively; if you reenter the event loop or call waitForReadyRead() inside a slot connected to the readyRead() signal, the signal will not be reemitted (although waitForReadyRead() may still return true).
    See: https://doc.qt.io/qt-5/qiodevice.html#readyRead

    In my case a second readRequest() is coming in, and both the running and new executing their QEventLoops. The second request gives the control back to the QEventLoops from the first request frequently, but the first request get's not be resumed before the second request was finished. So there are working concurrent readyRead() calls at the local HTTP server, but not at the "Remotenet" proxy module. There are only sequential readyRead() calls, and the readyRead() method for the first request will called not before after the second request was finished. And that although this are different "Remotenet" objects.

    Could this issue be related to the mentioned "recursively" emitting, described by the docs above?

    What's good approach to solve this? And make the readyRead() callable parallel?
    Note:

    • QIODevice requests a hard data size, so blocking is needed until all requested data is retrieved
    • When leaving the readyRead() and try to send the data later, the socket may be closed
    • Using Qt threads cause different warnings and do not solve the issue

    Thank you

    Stefan


Log in to reply