PyQt5 - TCP readyRead only firing once if server is on its own QThread

Igor86

Hi all!

I am having a difficult time figuring out what I am doing wrong.

I had a TCP server running on the main thread, but given I did not like that because when the Interface was doing something it delayed the packets. But beside that it was working great. So i moved it to its own thread, and now i am still able to connect to the server, and send a packet to it, but it only gets and replies to the first one. then its over....

any help is greatly appreciated...

best regards

Igor

main.py:

# server
        self.serverThread = QThread()
        self.serverWorker = server.Server()
        self.serverWorker.moveToThread(self.serverThread)
        self.signalSendCommand.connect(self.serverWorker.client.send_command)
        self.serverWorker.client.statusSignal.connect(self.update_button_status)
        self.signalSettingsChanged.connect(self.serverWorker.settings_changed)
        self.signalSettingsChanged.connect(self.serverWorker.client.settings_changed)
        self.serverThread.start()

server.py:

import sys

from PyQt5.QtCore import QObject
from PyQt5.QtNetwork import QHostAddress, QTcpServer
from cvLogger import logger as log

import client
import definitions
import settingsClass

sys.path.append(definitions.DIST_PATH)


class Server(QObject):
    def __init__(self):
        QObject.__init__(self)
        self.TCP_LISTEN_TO_PORT = 2001
        self.server = QTcpServer()
        self.server.newConnection.connect(self.on_new_connection)
        self.client = client.Client(self)
        self.settings = settingsClass.Settings()
        self.start_server()

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("server exited..")

    def settings_changed(self, settings):
        self.settings = settings

    def on_new_connection(self):
        while self.server.hasPendingConnections():
            self.client.set_socket(self.server.nextPendingConnection())

    def start_server(self):
        if self.server.listen(
                QHostAddress.Any, self.TCP_LISTEN_TO_PORT
        ):
            log.info("Server is listening on port: {}".format(self.TCP_LISTEN_TO_PORT))
        else:
            log.critical("Server couldn't wake up")

client.py:

import sqlite3
import sys
import numpy

from PyQt5.QtCore import *
from cvLogger import logger as log

import cvUtils
import database as db
import definitions
import settingsClass
import signals
import stopwatch

sys.path.append(definitions.DIST_PATH)

def on_connected():
    log.info("Client connected")

def on_disconnected():
    log.info("Client disconnected")

class Client(QObject):
    statusSignal = pyqtSignal(int)
    start_camera = pyqtSignal(str)
    tgSig = signals.TgSig()
    resultSentSignal = pyqtSignal(str, numpy.ndarray, numpy.ndarray, str, str, float, float, tuple,
                                  bool, str, float, int, int)

    def __init__(self, parent=None):
        super().__init__(parent)
        self.socket = None
        self.settings = settingsClass.Settings()

    def settings_changed(self, settings):
        self.settings = settings

    def set_socket(self, socket):
        self.socket = socket
        self.socket.connected.connect(on_connected)
        self.socket.disconnected.connect(on_connected)
        self.socket.readyRead.connect(self.on_ready_read)
        log.info("Client connected from Ip %s" % self.socket.peerAddress().toString())

    def on_ready_read(self):
        msg = self.socket.readAll()
        msg_txt = str(msg, 'utf-8').strip()
        messages = msg_txt.split("\r")
        segments = messages[0].split(",")
         if segments[0] == "STA":
            status = int(segments[1])
            self.statusSignal.emit(status)
            # send back OK message
            out = 'OK'
            if self.settings.chkSendCr:
                out = out + '\r'
            self.socket.write(bytearray(out, 'ascii'))

JonB

@Igor86 said in PyQt5 - TCP readyRead only firing once if server is on its own QThread:

now i am still able to connect to the server, and send a packet to it, but it only gets and replies to the first one. then its over....

I don't see the client sending anything to the server unless/until on_ready_read() is called, and I don't see the server sending anything to the client anywhere.

Igor86

@JonB The client is Hercules, and am sending data there.. here is a screenshot of what happened when i tried to send:

The first "STA,1" is received and "OK" is answered by the server. all successive messages are without answer.

Igor86

Just to elaborate, my python program is the TCP SERVER. my client is another program.
all 3 files posted above are fom my server. main.py (main thread), server.py (tcp server on its own thread), client.py (client handling of the server). there is no more code involved other than this.

so when my external application connects, it is able to send mesages and stay connected. the server does only detect the first packet tough.. then it ignores the rest. The connection is not dropped, the client is still able to send data, but the server is not evaluating it / firing readyRead.

JonB

@Igor86
You say nothing about when your client sends the 3 STA messages. You do not say if it waits for the OK before sending the next one.

If it sends multiple messages without waiting for the server's response then your code in on_ready_read() will/can only pick up the first one and send the OK acknowledgment. Is this your case? You would need to look through all the data received from readAll() (and presumably acknowledge each one, if that is what you want).

The above has nothing to do with whether you have secondary threads or just one thread --- it is always the case.

Your code also has a deeper assumption which is not valid, even if the client does wait for server's OK before sending another message. It assumes that on_ready_read()/readAll() will be called/collect a "complete" message, such as STA,1. But it may not: it might only receive the first character, S, on one call, and then it might receive TA,1 in another message. Your code does not accommodate that, and because you simply have an if segments[0] == "STA" it will neither respond not print anything out in that case, so you won't even know.....

Igor86

@JonB thank you for your answer. The sta,1 are sent by me manually.. there were a few seconds between them.

It is repeatable that it replies "ok" only to the first message. closing and reopening the connection fixes the problem.

i also added a print to beginning of the readReady and it only printed once. so indeed it is not firing anymore after the first time.

This did not happen before i movedi it to its own thread. it worked...

So I agree on the points you made about the assumptions I make in the code, but i strongly believe that this is not the cause of the problem...

In fact, we can remove the whole "if" condition and simply write back "ok" whenever ANYTHING arrives, and it behaves the same

JonB

@Igor86
Let's assume you are correct. Then, I don't know, does the server-separate-thread continue to service a message loop for messages after the first?

You do not try to access (read or write) the socket from different threads, do you?

I think you should set up a tiny, standalone example with your separate-thread server to test.

Igor86

@JonB thank you, I just made a minimal reproducible test, and this is what got printed out:

QObject: Cannot create children for a parent that is in a different thread.
(Parent is QNativeSocketEngine(0x25e41138b70), parent's thread is QThread(0x25e3e9ce3f0), current thread is QThread(0x25e40d34970)

what is the meaning of this?

here is the sample code: https://igormasin.it/fileuploads/tcptest.zip

JonB

@Igor86
That's the whole point! (But now not sure if that is the case with your original code you did not get the same error there?)

When using threads exactly as it says Qt does not allow QObjects to be created parented to a QObject in another thread.

here is the sample code: https://igormasin.it/fileuploads/tcptest.zip

Others may download/look at an external file, but not me. If the code isn't small enough to paste it's (probably) not a minimal, reproducible example.

Igor86

@JonB yes the message is shown there too, but it was logged as a info and not as a error by mistake so it was not displayed..

But how can i solve this? do i have to move the code from the client.py to server.py?

sure i can post the code here..

here we go:

main.py:

import sys
import server
from PyQt5 import QtWidgets, QtCore
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
from PyQt5.uic import loadUi


class UiMainWindow(QMainWindow):

    def __init__(self):
        super(UiMainWindow, self).__init__()
    
        # server
        self.serverThread = QThread()
        self.serverWorker = server.Server()
        self.serverWorker.moveToThread(self.serverThread)
        self.serverThread.start()
        #self.serverThread.start_server()
        self.show()

if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)
    MainWindow = QtWidgets.QMainWindow()
    ui = UiMainWindow()
    sys.exit(app.exec_())

server.py

import sys
from PyQt5.QtCore import QObject
from PyQt5.QtNetwork import QHostAddress, QTcpServer
import client


class Server(QObject):
    def __init__(self):
        QObject.__init__(self)
        self.TCP_LISTEN_TO_PORT = 2001
        self.server = QTcpServer()
        self.server.newConnection.connect(self.on_new_connection)
        self.client = client.Client(self)
        self.start_server()

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("server exited..")

    def on_new_connection(self):
        while self.server.hasPendingConnections():
            self.client.set_socket(self.server.nextPendingConnection())

    def start_server(self):
        if self.server.listen(
                QHostAddress.Any, self.TCP_LISTEN_TO_PORT
        ):
            print("Server is listening on port: {}".format(self.TCP_LISTEN_TO_PORT))
        else:
            print("Server couldn't wake up")

client.py:

import sys

from PyQt5.QtCore import *


def on_connected():
    print("Client connected")


def on_disconnected():
    print("Client disconnected")


class Client(QObject):

    def __init__(self, parent=None):
        super().__init__(parent)
        self.socket = None


    def set_socket(self, socket):
        self.socket = socket
        self.socket.connected.connect(on_connected)
        self.socket.disconnected.connect(on_connected)
        self.socket.readyRead.connect(self.on_ready_read)
        print("Client connected from Ip %s" % self.socket.peerAddress().toString())

    def on_ready_read(self):
        msg = self.socket.readAll()
        msg_txt = str(msg, 'utf-8').strip()
        messages = msg_txt.split("\r")
        segments = messages[0].split(",")
        

        if segments[0] == "STA":
            status = int(segments[1])
            # send back OK message
            out = 'OK\r'
            self.socket.write(bytearray(out, 'ascii'))

Igor86

Fixed it! thank you for your precious help!

JonB

@Igor86 said in PyQt5 - TCP readyRead only firing once if server is on its own QThread:

but it was logged as a info and not as a error by mistake so it was not displayed..

Grrrr :)

self.client.set_socket(self.server.nextPendingConnection())

server is running in its own QThread, but it passes a socket to client which is running in the main thread. Not allowed!

Fixed it! thank you for your precious help!

You have just replied with this while I was typing. Good :)

Igor86

@JonB sorry for bothering again, but i cant figure this out. I tried to put both files (server.py and client.py) toghether into the same file. its my understanding that this runs all on the same thread, in the server class.. but i am obviously wrong, since i get the same error as above again.. what am i missing? main.py hasn't changed. just brought toghether server and client into server so it makes more sense..

import sys
from PyQt5.QtCore import *
from PyQt5.QtNetwork import QHostAddress, QTcpServer


    
class Server(QObject):
    def __init__(self):
        QObject.__init__(self)
        self.TCP_LISTEN_TO_PORT = 2001
        self.server = QTcpServer()
        self.server.newConnection.connect(self.on_new_connection)
        self.start_server()
        self.socket = None

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("server exited..")

    def on_new_connection(self):
        while self.server.hasPendingConnections():
            self.set_socket(self.server.nextPendingConnection())

    def start_server(self):
        if self.server.listen(
                QHostAddress.Any, self.TCP_LISTEN_TO_PORT
        ):
            print("Server is listening on port: {}".format(self.TCP_LISTEN_TO_PORT))
        else:
            print("Server couldn't wake up")
            
    def set_socket(self, socket):
        self.socket = socket
        self.socket.connected.connect(self.on_connected)
        self.socket.disconnected.connect(self.on_disconnected)
        self.socket.readyRead.connect(self.on_ready_read)
        print("Client connected from Ip %s" % self.socket.peerAddress().toString())

    def on_ready_read(self):
        msg = self.socket.readAll()
        msg_txt = str(msg, 'utf-8').strip()
        messages = msg_txt.split("\r")
        segments = messages[0].split(",")
        

        if segments[0] == "STA":
            status = int(segments[1])
            # send back OK message
            out = 'OK\r'
            self.socket.write(bytearray(out, 'ascii'))
            
    def on_connected(self):
        print("Client connected")


    def on_disconnected(self):
        print("Client disconnected")

JonB

@Igor86
I'm not the expert on this thread stuff --- precisely because it's nasty, and most Qt stuff can be done without secondary threads despite what newcomers seem to think --- but isn't the server object in the worker thread but the client object is back in the main thread? You need to read https://doc.qt.io/qt-6/threads-qobject.html. QThread *QObject::thread() const returns the thread in which the object lives, use that dotted around your use of server/client/set_socket() etc. to see which thread you are in compared against QThread::currentThread() (https://stackoverflow.com/questions/33280672/qthreadcurrentthread-vs-qobjectthread).