QTcpSocket client, write problem...

SPlatten

I think I spoke to soon I still have the problem:

QSocketNotifier: Socket notifiers cannot be enabled or disabled from another thread

I've modified all my QObject::connect's, ensuring that I am not using Qt::DirectConnection. I'm still getting this message. I've been careful that I am not calling write from any thread, I always emit a write signal which connects to a slot called onWrite which actually does the::

qint64 int64Written = mpsckClient->write(arybytMsg);

KroMignon

@SPlatten said in QTcpSocket client, write problem...:

I think I spoke to soon I still have the problem:
QSocketNotifier: Socket notifiers cannot be enabled or disabled from another thread

As an experimented developer, you should know that multithreading can solve problems but always introduce new problems ;)

When using QObject based classes, you should always ensure that you are using them in the thread in which they have be created / moved.
So, are you sure qint64 int64Written = mpsckClient->write(arybytMsg); is done in the right thread (mpsckClient->thread() == QThread::currentThread()).

SPlatten

@KroMignon I thought that one of the benefits of using signals and slots is that you can be assured that they are thread safe?

KroMignon

@SPlatten said in QTcpSocket client, write problem...:

I thought that one of the benefits of using signals and slots is that you can be assured that they are thread safe?

Yes it is, but it depends how you are doing it.

QObject::connect(srcObject, SIGNAL(), destObject, SLOT()) will ensure SLOT() will run in destObject work thread, when signal is emitted.

EDIT:
but
QObject::connect(srcObject, SIGNAL(), destObject, SLOT(), Qt::DirectConnection) will run SLOT() in thread from which signal has been emitted.

SPlatten

@KroMignon Thank you, rapidly learning this.

KroMignon

@SPlatten said in QTcpSocket client, write problem...:

Thank you, rapidly learning this.

Other things you should take care off when you are doing multithreading, is to set parent for class member to ensure that when you are moving the class instance to another thread, all member will follow.

To better explain what I mean, suppose following example:

class TestClass : public QObject
{
    Q_OBJECT
public:
    explicit TestClass(const QHostAddress &address, quint16 portNum, QHostAddress QObject * parent=nullptr)
         : QObject(parent), mSocket(new QTcpSocket(this))
   {
   }
private:
    QTcpSocket* mSocket;
};

class TestClass2 : public QObject
{
    Q_OBJECT
public:
    explicit TestClass2(const QHostAddress &address, quint16 portNum, QHostAddress QObject * parent=nullptr)
         : QObject(parent), mSocket(new QTcpSocket())
   {
   }
private:
    QTcpSocket* mSocket;
};

And somewhere in code:

auto sock = new TestClass(QHostAddess::LocalHost, 888);
auto sock2 = new TestClass2(QHostAddess::LocalHost, 889);
auto t = new QThread();
sock->moveToThread(t);
sock2->moveToThread(t);
t->start();

now:

sock and sock->mSocket are moved to thread t.
sock2 is move the thead t but sock2->mSocket still lives in thread in which it have been created

SPlatten

I feel like I am going around and round in circles and just not progressing. I have one process which listens for data on port 8123. It also starts another process which connects to the first process on the host IP and port 8123.

The processes connect without problem, I am trying to write data from the second process to the first. I emit a signal from the socket that is used to connectToHost on the first process, I have a slot that connects to the emitted write signal:

void clsMsgSender::onWrite(QJsonObject objJSON) {    
    if ( mpsckClient == nullptr ) {
        return;
    }
    QAbstractSocket::SocketState sckState = eGetSockState();

    if ( sckState != QAbstractSocket::ConnectedState ) {
#if defined(DEBUG_SOCKETS)
        qdbg() << "Not connected!";
#endif
        return;
    }
    QMutexLocker lock(&mMutex);
#if defined(DEBUG_SOCKETS)
    QString strMsg(QJsonDocument(objJSON).toJson(QJsonDocument::Compact));
    qdbg() << QString("clsMsgSender::onWrite, to: %1:%2, data: %3")
                .arg(mpsckClient->peerAddress().toString(), QString::number(mpsckClient->peerPort()), strMsg);
#endif
    clsJSON::addCommonJSONflds(objJSON, nullptr, mpModule);
    //Insert a unique message ID into the message
    objJSON.insert(clsJSON::mscszMsgID, QString::number(clsMsgSender::ulnglngGetUniqueMsgID()));
    //Associate this TCP socket with the output data stream
    QByteArray arybytMsg;
    arybytMsg = QJsonDocument(objJSON).toJson(QJsonDocument::Compact);
    //Write message
    qint64 int64Written = mpsckClient->write(arybytMsg);
#if defined(DEBUG_SOCKETS)
    qdbg() << QString("clsMsgSender::onWrite, written:%1")
                .arg(int64Written);
#endif
    if ( int64Written > 0 ) {
    //Remove the item from the queue
        mqueMsgsOut.dequeue();
    }
    //No longer busy
    mblnBusy = false;
}

When I emit the write signal I take a JSON packet from a queue, the item is only removed from the queue when the write completes. I can see using the Qt Creator debugger that in the onWrite slot the socket write is successful and returns the number of bytes written. However I see no evidence that the first application is receiving the data. What could be the cause?

KroMignon

@SPlatten said in QTcpSocket client, write problem...:

However I see no evidence that the first application is receiving the data. What could be the cause?

TCP sockets are stream interfaces, and they use buffers (reception/transmission) to avoid sending too much small packets.
To force data transfer you could do:

// write to transmission buffer
qint64 int64Written = mpsckClient->write(arybytMsg);
// force transmission
mpsckClient->flush();

SPlatten

@KroMignon , I'm wondering if sockets is the best mechanism for the purpose I am trying to use them for.

The main process will launch X number of processes, each process will provide a specific purpose.

Messages sent from the main process will issue requests to the other processes and the other processes will send the results of these requests back to the main. I am using JSON for packaging data messages between all applications.

I'm now looking at QSharedMemory, but I don't know if I can use QSharedMemory for this kind of communication?

JonB

@SPlatten
You can use shared memory if you prefer. But I think you'll end up with the same sort of thread issues as you're having with TCP.

KroMignon

@SPlatten said in QTcpSocket client, write problem...:

I'm now looking at QSharedMemory, but I don't know if I can use QSharedMemory for this kind of communication?

If you don't want to lose all done work, I would suggest you to use QLocalServer and QLocalSocket which are kind of TCP socket but which is a more aggressive transmission rule.
Data are sent almost directly, on entering event loop.
And you don't have to handle shared lock for shared memory access.

SPlatten

@KroMignon , thank you, I will look into QLocalServer and QLocalSocket, it sounds like a better fit as my intention is that all processes will run on the same system.

SGaist

Hi,

As already written in one of your other threads: QTcpSocket shall be created in the thread they are going to be used in. They cannot be moved around. You can pass around the descriptor but not the QTcpSocket itself. See the threaded QTcpServer/QTcpClient examples.

SPlatten

@SGaist , I create the sockets in the main thread and then use signals to communicate between threads, where the connections are also created in the main thread.

kshegunov

@SPlatten said in QTcpSocket client, write problem...:

@SGaist , I create the sockets in the main thread and then use signals to communicate between threads, where the connections are also created in the main thread.

The place of making the connection(s) is irrelevant. There are 2 things that matter:

For sockets - the thread where the descriptor was initialized into, or in other words the thread where the socket's been opened in. @SGaist is half correct in saying they can't be moved around, they can, but iff they've not been opened yet. The thread that socket's been opened in then is and must be the thread the QObject lives in and is fixed forever and ever until the end of time.
For any signal slot emission - the thread that the signal was emitted from and the receiving object context's thread. (direct connections are uninteresting here). The slot's going to be executed in the receiver's thread, while depending on the signal emission's thread either the call will be direct, or queued. For the same connection this may vary within the program if the (same) signal emissions happen from different threads (which is possible), so sometimes it may default to direct calls, sometimes to queued calls. The point is, however, that you shouldn't care. The design should be made such that it doesn't matter if the slot's executed immediately or later, which is also why you shouldn't (99% of cases) tamper with the connection type when doing the QObject::connect call.

I imagine your object threads are wrong and you hold instances of objects that live in a different thread, so that's why you get the errors. To be certain this doesn't happen, always provide a valid parent for QObjects with the major exception of when the object is a root of a QObject tree that's going to be moved to another thread. Thereafter everything is easy peasy.

SPlatten

@kshegunov , my listening class implements the incomingConnection method:

void clsListener::incomingConnection(qintptr sckDescriptor) {    
    //Every new connection will be run in a newly created thread
    clsServer* pThread = new clsServer(sckDescriptor, this);        
    //We have a new connection
    qDebug() << sckDescriptor << " Connecting...";
    //Connect signal/slot
    //Once a thread is not needed, it will be beleted later
    QObject::connect(pThread, &QThread::finished
                    ,pThread, &QThread::deleteLater);
    pThread->start();
}

From what you are saying the thread created here is wrong in that it passes the socket descriptor?

The constructor for clsServer:

clsServer::clsServer(qintptr sckDescriptor, QObject* pParent)
                                            : QThread(pParent)
                                            , msckDescriptor(sckDescriptor)
                                            , mpsckIncoming(nullptr) {
}

The prototype for clsServer:

 class clsServer : public QThread {
    Q_OBJECT

    private:
        qintptr msckDescriptor;
        QTcpSocket* mpsckIncoming;
        mqueJSON mqueMsgsIn;

    public:
        explicit clsServer(qintptr sckDescriptor, QObject* pParent = nullptr);
        ~clsServer();

        void cleanup();
        void run();

    signals:
        void error(QTcpSocket::SocketError socketerror);

    public slots:
        void onDataIn();
        void onDisconnected();
        void onErrorOccurred(QAbstractSocket::SocketError error);
    };

The thread body:

void clsServer::run() {
    QTcpSocket* pSocket = new QTcpSocket();
    //Set the ID
    qdbg() << "clsServer::run";
    if( !pSocket->setSocketDescriptor(msckDescriptor) ) {
    //Something's wrong, we just emit a signal
        emit error(pSocket->error());
        return;
    }
    //Connect socket and signal
    mpsckIncoming = pSocket;
    QObject::connect(mpsckIncoming, &QAbstractSocket::readyRead
                    ,this, &clsServer::onDataIn);
    QObject::connect(mpsckIncoming, &QAbstractSocket::errorOccurred
                    ,this, &clsServer::onErrorOccurred);
    QObject::connect(mpsckIncoming, &QAbstractSocket::disconnected
                    ,this, &clsServer::onDisconnected);
    //We'll have multiple clients, we want to know which is which
    qdbg() << msckDescriptor << " Client connected";
    //Make this thread a loop,
    //thread will stay alive so that signal/slot to function properly
    //not dropped out in the middle when thread dies
    exec();
}

Actually, the descriptor passed into the constructor is not actually used until the thread body run loop is entered. Again, it looks very wrong now.

However, the above is the class that listens for incoming data, the class that sends messages from one process to another:

class clsMsgSender : public QThread {
    Q_OBJECT

    private:        
        static const quint16 mscuint16ConnectionTO;
        static mpMsgSenders msmpMsgSenders;
        static clsMsgSender* mspService;
        static qulonglong msulnglngUniqueMsgID;

        mutable QMutex mMutex;
        bool mblnBusy, mblnConnected, mblnRun, mblnStopped;
        clsModule* mpModule;
        QTcpSocket* mpsckClient;
        mqueJSON mqueMsgsOut;
        QString mstrHost;
        quint16 muint16Port;

        QAbstractSocket::SocketState eGetSockState();
        QJsonObject objAnythingToDo();
        void push(const QJsonObject& crobjJSON);
        void setModule(clsModule* pModule);
        quint16 uint16GetPort();

    public:
        explicit clsMsgSender(clsModule* pModule, QObject* pParent = nullptr);
        ~clsMsgSender();

        bool blnBusy();
        bool blnRunning() { return mblnRun; }
        mpMsgSenders::iterator itGetMsgSndr(quint16 uint16Port, QString& rstrKey);
        QTcpSocket* pGetClient() { return mpsckClient; }
        static clsMsgSender* pGetMsgSndr(quint16 uint16Port);
        static clsMsgSender* pGetService() { return clsMsgSender::mspService; }
        void setHost(QString strHost);
        static QString strGetLocalIP();
        void terminate() { mblnRun = false; }
        static qulonglong ulnglngGetUniqueMsgID();

    signals:
        void connectToHost(const QString& crstrHost, quint16 uint16Port);
        void removeTrkrs(clsMsgSender* pMsgSndr);
        void sendJSON(const QJsonObject& robjJSON);
        void write(QJsonObject robjJSON);

    public slots:
        void onConnectToHost(const QString& crstrHost, quint16 uint16Port);
        void onConnected();
        void onDisconnected();
#if defined(DEBUG_SOCKETS)
        void onErrorOccurred(QAbstractSocket::SocketError sckError);
        void onHostFound();
#endif
        void onRemoveTrkrs(clsMsgSender* pMsgSndr);
        void onSendJSON(const QJsonObject& crobjJSON);
        void onWrite(QJsonObject robjJSON);
        void run();
    };

Implementation:

clsMsgSender::clsMsgSender(clsModule* pModule, QObject* pParent)
                    : QThread(pParent)
                    , mblnBusy(false), mblnConnected(false)
                    , mblnRun(true), mblnStopped(false)
                    , mpModule(nullptr)
                    , mpsckClient(new QTcpSocket()) {
    setModule(pModule);
    clsMsgSender::mspService = this;
    QObject::connect(this, &clsMsgSender::connectToHost
                    ,this, &clsMsgSender::onConnectToHost);
    QObject::connect(this, &clsMsgSender::removeTrkrs
                    ,this, &clsMsgSender::onRemoveTrkrs);
    QObject::connect(this, &clsMsgSender::sendJSON
                    ,this, &clsMsgSender::onSendJSON);
    QObject::connect(this, &clsMsgSender::write
                    ,this, &clsMsgSender::onWrite);
    //Thread body
    QObject::connect(this, &QThread::started
                    ,this, &clsMsgSender::run);
    QObject::connect(this, &QThread::finished
                    ,this, &QThread::deleteLater);
    QObject::connect(mpsckClient, &QAbstractSocket::connected
                    ,this, &clsMsgSender::onConnected);
    QObject::connect(mpsckClient, &QAbstractSocket::disconnected
                    ,this, &clsMsgSender::onDisconnected);
#if defined(DEBUG_SOCKETS)
    QObject::connect(mpsckClient, &QAbstractSocket::errorOccurred
                    ,this, &clsMsgSender::onErrorOccurred);
    QObject::connect(mpsckClient, &QAbstractSocket::hostFound
                    ,this, &clsMsgSender::onHostFound);
#endif
    mpsckClient->setSocketOption(QAbstractSocket::LowDelayOption, 1);
}

KroMignon

@SPlatten As I told you in previous posts, QThread class is a little bit tricky and sub-classing it is in most case not the best way to do.

Because:

the clsServer instance leaves in the thread in which it have been created, supposing main thread
mpsckIncoming will leave in the hosted thread

this means, with:

QObject::connect(mpsckIncoming, &QAbstractSocket::readyRead, this, &clsServer::onDataIn);

slot onDatin() will run in this thread which is the main thread and using mpsckIncoming will not work properly, because you are in the wrong thread.

SPlatten

@KroMignon, this morning whilst looking closer at the code and trying to resolve it I found the problem, there is nothing wrong with my threads of sockets, the actual problem is that my debug isn't reporting it properly, which I am looking at now.