QTcpServer won't receive client data on connected QTcpSocket

[jsulm]

@dmitttri said in QTcpServer won't receive client data on connected QTcpSocket:

while (workerThreadShouldRun)
{

You're blocking Qt event loop again.
You should use signals/slots instead. There is readyRead signal which you should use.

[dmitttri]

@jsulm said in QTcpServer won't receive client data on connected QTcpSocket:

You're blocking Qt event loop again.
You should use signals/slots instead. There is readyRead signal which you should use.

Hi, thanks for your kind answer!

I don't get this point with blocking the Qt event loop, by doing my own processing in a dedicated thread. I mean, threads are meant to be used with some kind of while loop until the job is done. doWork is implemented according to the Qt documentation, which states:

public slots:
    void doWork(const QString &parameter) {
        QString result;
        /* ... here is the expensive or blocking operation ... */
        emit resultReady(result);
    }

doWork in my case should communicate with the connected client, perform some dedicated job and then exit. Are you suggesting to use readyRead signal handler to process incoming data, and call doWork from it, taking care that doWork do not loop, but finish soon as possible? In such case my yet to come processing state machine would end up in readyRead handler, and there will be mutiple doWork methods doWork1, doWork2, which implements job steps. I can try such approach

I am sorry for asking such questions, but this is somehow fundamental for me to understand why and how my thread handler manage to stops main Qt event loop, which executes in the main program process.

I have already used thread with a Qt GUI application, and always had this pattern of while (threadShoudRun) and never had issues with event loop and GUI processing. Why this time it happens to be so problematic to run a thread processing loop?

Are there any other Qt examples of a multi threaded TCP server, other than just fortune examples?

[Christian Ehrlicher]

I already told you why an event loop is needed - to get the tcp data from the os into the QTcpSocket buffer so you can read it.

[jsulm]

@dmitttri said in QTcpServer won't receive client data on connected QTcpSocket:

threads are meant to be used with some kind of while loop until the job is done

That's what Qt event loop is for...

[dmitttri]

@Christian-Ehrlicher said in QTcpServer won't receive client data on connected QTcpSocket:

I already told you why an event loop is needed - to get the tcp data from the os into the QTcpSocket buffer so you can read it.

Sure thanks, but it don't really provide an explanation how to tell a new thread, run in an event loop so your tcp socket can get chance to read data.

I thought it was about worker thread article.

As explained there, in case of the Worker object approach, run method just runs exec() and this is should enable thread to run an event loop, while doWork is meant to do the thread processing job.

Other valid approach (also explained there) is to override run method and do the threaded work there. This was my original approach also posted above. Which does not work.

So I tried both options, posted the code, and with the short hints you and other members provided, I still could not understand the culprit and get it work. But this is related to my very low Qt coverage.

Thanks for your help, I will figure out what I am doing wrong, and I will take care to post the proper solution.

[dmitttri]

@jsulm said in QTcpServer won't receive client data on connected QTcpSocket:

@dmitttri said in QTcpServer won't receive client data on connected QTcpSocket:

threads are meant to be used with some kind of while loop until the job is done

That's what Qt event loop is for...

Hm, this is what I may be missing, as someone not really familiar with Qt.

Threads Events QObjects

In particular, under Per-thread event loop there is an explanation of what I couldn't understand up to now. Apart of the Main Qt event loop, there are Per-thread event loops and specially networking classes are required to explicitly run such loop.

[jsulm]

@dmitttri Please read https://wiki.qt.io/QThreads_general_usage

[dmitttri]

@jsulm said in QTcpServer won't receive client data on connected QTcpSocket:

@dmitttri Please read https://wiki.qt.io/QThreads_general_usage

Yep, I did it already. In the provided example, process is very simple:

void Worker::process() { // Process. Start processing data.
    // allocate resources using new here
    qDebug("Hello World!");
    emit finished();
}

This method simply return immediately, while my processing (doWork in the code above) requires networking (socket read/write) and several discrete steps to complete.

Worker approach creates a thread object, which run() method will just perform exec(), allowing thread event loop to run. This is perfectly clear.

Then it creates Worker object which has process() method, and moves Worker object to the thread context. Now process() will execute in the thread context too.

Key questions are:

1. How run() and process() now coexists and executes in the same thread?"
2. When process() is running, what's happening with the run method? It hangs until process finishes?

It seems that in my case, where process() method uses Networking, I should not use process() method at all.

More precise, instead of the process() method it seems that I need a socket readyRead slot handler which executes in the worker thread, process incoming data, keep processing state variables, and emit finished() signal. once whole client related work is done.

[dmitttri]

@dmitttri

There was a serious mistake with the client code, in the pokeServer which just writes to the socket and checks for number of bytes written. This is something you would normally do with Linux socket API.

The issue was, this write operation has to be completed by the Qt event loop, what I was missing was at least to call not so popular synchronous waitForBytesWritten, to ensure these bytes really hits the network stack.

What made it even worse, is that right after pokeServer, my test code enters sleep on the mainClient.cpp, which directly stops Qt event loop of processing written bytes.

[dmitttri]

@dmitttri

For the sake of completeness of this post, I am providing very simple TCP client/server example with client sending data, and server receiving it.

Server side is not implemented with threads, and will serve me as the starting point to properly implement threaded server, based on all recommendations I got up to now.

CLIENT SIDE

Simple Client.h

#ifndef SIMPLECLIENT_H
#define SIMPLECLIENT_H

#include <QTcpSocket>
#include <QAbstractSocket>

#define SOCKET_OPERATION_TIMEOUT_MS 3000

class SimpleClient
{
public:
    SimpleClient() {};

    bool connectToHost(QString serverIP, qint16 serverPort)
    {

        this->socket = new QTcpSocket;
        socket->connectToHost(QHostAddress(serverIP), serverPort);

        qDebug() << "Connecting to" << serverIP << ":" << serverPort;
        socket->connectToHost(QHostAddress(serverIP), serverPort);

        if (!socket->waitForConnected(SOCKET_OPERATION_TIMEOUT_MS))
        {
            qDebug() << "Error, connection timeout, server not accessible.";
            return false;
        }
        qDebug() << "Connected";
        return true;
    }

    bool write(const char *data, int len)
    {
        if (socket->state() != QAbstractSocket::ConnectedState)
        {
            qDebug() << "Write error, client not connected";
            return false;
        }

        int bytesWritten = this->socket->write(data, len);
        qDebug() << "Written" << bytesWritten << "bytes";

        if(!this->socket->waitForBytesWritten())
        {
            qDebug() << "Write error";
            return false;
        }
        qDebug() << "Write success";
        return true;
    }

    void disconnectFromHost()
    {
        if (socket->state() == QAbstractSocket::ConnectedState)
        {
            this->socket->disconnectFromHost();
            qDebug() << "Disconnected";
        }
    }

private:
    QTcpSocket *socket = nullptr;
};

#endif // SIMPLECLIENT_H

MainClient.cpp

#include <QThread>
#include <QDebug>
#include "SimpleClient.h"

int main(int argc, char *argv[])
{
    SimpleClient client;

    client.connectToHost("127.0.0.1", 9000);
    client.write("Hello Server", sizeof("Hello Server"));

    // NOTE: It's fine to sleep for some time now, we don't have GUI
    // or any networking async operations, required to be handled by the event loop
    QThread::sleep(2);
    client.disconnectFromHost();

    return 0;
}

SERVER SIDE (not threaded)

SimpleServer.h

#ifndef SIMPLESERVER_H
#define SIMPLESERVER_H

#include <QDebug>
#include <QTcpServer>
#include <QTcpSocket>

class SimpleServer : public QObject
{
    Q_OBJECT;

public:
    SimpleServer() {};

    bool start(QString serverIP, qint16 serverPort)
    {
        tcpServer = new QTcpServer;

        connect (tcpServer, &QTcpServer::newConnection, this, &SimpleServer::onNewConnection);

        qDebug() << "Awaiting for new connections "
                 << "IP" << serverIP
                 << "PORT" << serverPort;

        tcpServer->listen(QHostAddress(serverIP), serverPort);
        return true;
    }

private slots:
    void onNewConnection()
    {
        qDebug() << "New connection";

        QTcpSocket *socket = tcpServer->nextPendingConnection();
        connect(socket, &QAbstractSocket::disconnected, socket, &QObject::deleteLater);

        qDebug() << "Socket info:" << socket->state() << socket->openMode();
        socket->waitForReadyRead();

        int bytesAvailable = socket->bytesAvailable();
        qDebug() << "Received" << bytesAvailable << "bytes";
    }
    
private:
    QTcpServer *tcpServer = nullptr;
};

#endif // SIMPLESERVER_H

MainServer.cpp

#include <QCoreApplication>
#include <QtCore>

#include "SimpleServer.h"


int main(int argc, char *argv[])
{
    QCoreApplication app(argc, argv);

    SimpleServer server;
    server.start("127.0.0.1", 9000);

    return app.exec();
}

[dmitttri]

@dmitttri

[SOLVED]
I am posting the solution for the original post:

1. Client had an issue, after socket write call it's required to call sync waitForBytesWritten, which will process actual socket writing
2. Server had an issue, in the run method, we can't just call socket read, it's required to call waitForReadyRead which takes care to process socket and fill it with incoming data.

Thanks everyone for their suggestions and help!

I am also posting working sample code, which is only meant for demo purposes, to show a simple synchronous approach to TCP client and server

DEMO TCP CLIENT AND SERVER WITH THREADS

MainClient.cpp

#include "SimpleClient.h"

int main(int argc, char *argv[])
{
    SimpleClient client;

    if (!client.connectToServer("127.0.0.1", 9000))
    {
        return -1;
    }

    if (!client.pokeServer("Hello Server"))
    {
        return -1;
    }

    return 0;
}

SImpleClient.h

#ifndef SIMPLECLIENT_H
#define SIMPLECLIENT_H

#include <QTcpSocket>
#include <QAbstractSocket>

#define SOCKET_OPERATION_TIMEOUT_MS 3000

class SimpleClient
{
public:
    SimpleClient();

    bool connectToServer(QString serverIP, qint16 serverPort);
    int  pokeServer(const char *pokeMsg);

private:
    QTcpSocket *socket = nullptr;
};

#endif // SIMPLECLIENT_H

SImpleClient.cpp

#include <QByteArray>
#include "SimpleClient.h"


SimpleClient::SimpleClient()
{
    this->socket = new QTcpSocket;
}

bool SimpleClient::connectToServer(QString serverIP, qint16 serverPort)
{    
    qDebug() << "Connecting to" << serverIP << ":" << serverPort;

    socket->connectToHost(QHostAddress(serverIP), serverPort);

    if (!socket->waitForConnected(SOCKET_OPERATION_TIMEOUT_MS))
    {
        qDebug() << socket->errorString();
        return false;
    }

    qDebug() << "Connected";
    return true;
}


int SimpleClient::pokeServer (const char *pokeMsg)
{
    qDebug() << "Sending :" << QString(pokeMsg);

    this->socket->write(pokeMsg, strlen(pokeMsg));

    if (!this->socket->waitForBytesWritten(SOCKET_OPERATION_TIMEOUT_MS))
    {
        qDebug() << socket->errorString();
        return false;
    }

    if (!this->socket->waitForReadyRead(SOCKET_OPERATION_TIMEOUT_MS))
    {
        qDebug() << this->socket->errorString();
        return false;
    }

    QByteArray serverReply = this->socket->readAll();
    QString replyString(serverReply.data());

    qDebug() << "Received:" << replyString;
    qDebug() << "Success";
    return true;
}

MainServer.cpp

#include <QCoreApplication>
#include <QtCore>

#include "SimpleServer.h"


int main(int argc, char *argv[])
{
    QCoreApplication app(argc, argv);

    SimpleServer server;
    server.start("127.0.0.1", 9000);

    return app.exec();
}

SimpleServer.h

#ifndef SIMPLESERVER_H
#define SIMPLESERVER_H

#include <QDebug>
#include <QTcpServer>

class SimpleServer : public QTcpServer
{
    Q_OBJECT;

public:
    SimpleServer(QObject *parent = nullptr);

    bool start(QString serverIP, qint16 serverPort);

protected:
    void incomingConnection(qintptr socketDescriptor) override; \
};

#endif // SIMPLESERVER_H

SimpleServer.cpp

#include <QHostAddress>

#include "SimpleServer.h"
#include "ServerJobThread.h"


SimpleServer::SimpleServer(QObject *parent)
    : QTcpServer(parent) {}


bool SimpleServer::start(QString serverIP, qint16 serverPort)
{
    qDebug() << "Awaiting for new connections "
             << "IP" << serverIP
             << "PORT" << serverPort;

    this->listen(QHostAddress(serverIP), serverPort);
    return true;
}


void SimpleServer::incomingConnection(qintptr socketDescriptor)
{
    qDebug() << "New incoming connection";

    ServerJobThread *thread = new ServerJobThread(socketDescriptor, this);
    connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
    thread->start();
}

ServerJobThread.h

#ifndef SERVERJOBTHREAD_H
#define SERVERJOBTHREAD_H

#include <QThread>
#include <QTcpSocket>

class ServerJobThread : public QThread
{
    Q_OBJECT

public:
    ServerJobThread(qintptr socketDescriptor, QObject *parent);

    void run() override;

private:
    qintptr socketDescriptor;
};

#endif // SERVERJOBTHREAD_H

ServerJobThread.cpp

#include <QByteArray>
#include <QTime>

#include "ServerJobThread.h"

#define SOCKET_OPERATION_TIMEOUT_MS 3000
#define THREAD_MAX_RUNNING_TIME_S      5

#define THREAD_ID this->currentThreadId()

ServerJobThread::ServerJobThread(qintptr socketDescriptor, QObject *parent)
    : QThread(parent)
{
    this->socketDescriptor = socketDescriptor;
}


void ServerJobThread::run()
{
    bool workerThreadShouldRun = true;
    QTcpSocket tcpSocket;

    if (!tcpSocket.setSocketDescriptor(this->socketDescriptor))
    {
        qDebug() << THREAD_ID << "setSocketDescriptor fail";
        return;
    }

    QTime workerThreadTimeoutTime = QTime::currentTime().addSecs(THREAD_MAX_RUNNING_TIME_S);
    while (workerThreadShouldRun)
    {
        if (tcpSocket.bytesAvailable() == 0)
        {
            tcpSocket.waitForReadyRead(SOCKET_OPERATION_TIMEOUT_MS);
        }
        else
        {
            QByteArray buffer = tcpSocket.readAll();
            if (buffer.size() > 0)
            {
                qDebug() << THREAD_ID
                         << "Received:" << QString(buffer.data());

                qDebug() << THREAD_ID
                         << "Sending :" << QString("Hello Client");

                tcpSocket.write("Hello Client");
                tcpSocket.waitForBytesWritten(SOCKET_OPERATION_TIMEOUT_MS);

                workerThreadShouldRun = false;
            }

            if (QTime::currentTime() > workerThreadTimeoutTime)
            {
                qDebug() << THREAD_ID
                         << "Thread max running time expired";
                workerThreadShouldRun = false;
            }
        }
    }
    qDebug() << THREAD_ID << "Thread finished";
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.14)

project(Simple-TCP-Client-Server LANGUAGES CXX)

set(CMAKE_AUTOUIC ON)
set(CMAKE_AUTOMOC ON)
set(CMAKE_AUTORCC ON)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(QT NAMES Qt6 Qt5 REQUIRED COMPONENTS Core Network)
find_package(Qt${QT_VERSION_MAJOR} REQUIRED COMPONENTS core Network)

add_executable(Server
    ServerJobThread.h
    ServerJobThread.cpp
    SimpleServer.h
    SimpleServer.cpp
    MainServer.cpp
)
target_link_libraries(Server Qt6::Core Qt6::Network)

add_executable(Client
    SimpleClient.h
    SimpleClient.cpp
    MainClient.cpp
)
target_link_libraries(Client Qt6::Core Qt6::Network)

CLIENT STDOUT

$ ./Client
Connecting to "127.0.0.1" : 9000
Connected
Sending : "Hello Server"
Received: "Hello Client"
Success

SERVER STDOUT

$ ./Server
Awaiting for new connections  IP "127.0.0.1" PORT 9000
New incoming connection
0x9a60 Received: "Hello Server"
0x9a60 Sending : "Hello Client"
0x9a60 Thread finished