How to handle a threading socket server

DieterV · 10 Sept 2023, 19:20

Hi,

I'm trying to achieve something which I think would be fairly basic:

Have a main window with a QPlainTextEdit widget for feedback during inital prototyping
Start a thread that waits for a raw socket connection
If a new connection comes in, create a new thread that handles input from that connection, likely emitting a signal that is connected in the main window to update the gui and be processed.

My code (I've tried to limit to what matters):

class MainWindow(qtw.QMainWindow, Ui_wdw_main_window):

    def __init__(self, *args, obj=None, **kwargs):
        super().__init__(*args, **kwargs)
        self.setupUi(self)

        self.txt_edit = qtw.QPlainTextEdit(self)
        self.server = Server()
        self.threads = []
        self._connected_socks = []
        self.server_thread = qtc.QThread()
        self.server.moveToThread(self.server_thread)
        self.server.sgn_new_connection.connect(self._new_connection)
        log.debug('Now starting server...')
        self.server_thread.started.connect(self.server.run)
        self.server_thread.start()
        self.threads.append(self.server_thread)

    @qtc.pyqtSlot(str, int, socket.socket)
    def _new_connection(self, ip: str, port: int, sock: socket.socket) -> None:
        if sock in self._connected_socks:
            log.warning(f'{ip} already connected.')
            return

        self._connected_socks.append(sock)

        self.txt_edit.appendPlainText(f'Connection from {ip}.')
        self._conn = ClientHandler(ip=ip, port=port, sock=sock)
        self._thread = qtc.QThread()  # CRASH HAPPENS HERE ON 2nd CONNECTION
        self._conn.moveToThread(self._thread)
        self._conn.sgn_new_message.connect(self._new_message)
        self._thread.started.connect(self._conn.run)
        self._thread.start()

    def _new_message(self, ip: str, msg: str) -> None:
        log.debug(f'message: {msg}')
        self.txt_edit.appendPlainText(f'[{ip}] -> {msg}')

In _new_connection I check if the socket is already connected. I don't know why yet, but this slot is called twice each time a new client connects. I'm not proud about this hack but until I figure out why it's called twiced, this seems to work.

The two QObject subclasses handling the sockets:

class Server(qtc.QObject):
    sgn_new_connection = qtc.pyqtSignal(str, int, socket.socket)
    sgn_new_message = qtc.pyqtSignal(str, str)
    sgn_connection_closed = qtc.pyqtSignal(str)

    def __init__(self, ip: str = '0.0.0.0', port: int = 8080):  # nosec B104
        # Thread.__init__(self)
        super().__init__()
        self.ip = ip
        self.port = port

    def run(self):
        log.debug(f'Starting server on {self.ip}:{self.port}')
        ip = self.ip
        port = self.port
        tcpServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        tcpServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        tcpServer.bind((ip, port))
        threads = []

        tcpServer.listen(10)
        log.debug(f'Server listening on {self.ip}:{self.port}.')

        while True:
            (sock, (ip, port)) = tcpServer.accept()
            self.sgn_new_connection.emit(ip, port, sock)
            log.debug(f'New connection from {ip}:{port}')
            self.sgn_new_connection.emit(ip, port, sock)

    def _new_message(self, ip: str, msg: str) -> None:
        log.debug('triggered in server')
        self.sgn_new_message.emit(ip, msg)


class ClientHandler(qtc.QObject):
    sgn_new_message = qtc.pyqtSignal(str, str)
    sgn_connection_closed = qtc.pyqtSignal(str)

    def __init__(self, ip, port, sock):
        super().__init__()
        self.ip = ip
        self.port = port
        self.sock = sock
        log.debug(f'Socket thread started for {ip}:{port}')

    def run(self):
        while True:
            data = self.sock.recv(2048)
            if len(data) == 0:
                log.debug('Socket closed from remote side.')
                self.sgn_connection_closed.emit(self.ip)
                self.terminate()
            data_str = data.decode('utf-8').strip()
            self.sgn_new_message.emit(self.ip, data_str)
            log.debug(f'Data Rx from {self.ip}: {data_str}')

This works for one connection. However, when i open a second connection, the application crashes in when the thread get created in _new_connection in the main window.

I've tried a number of things to get this working, but since I don't have a good understanding of how to work with threading yet (and it seems to be a minefield) I'm afraid of doing things the wrong way and having them work but not reliable.
Originally, I started a new thread for each connection from within the Server class. This seemed to work, but starting a thread for each socket from within the server thread started in the main window, I couldn't get signals to work correctly.and

I looked into a QRunnable and QThreadPool but from what I read, a QRunnable does not support signals and slots, which would make it useless.
I also looked into subclassing QThread, but most sources I found adviced against this and I see no benefit in my case.

So, a couple of questions:

Is my approach correct? (clearly not, it crashes :) )
If not, why?
I don't get any feedback when the app crashes, just "QThread: Destroyed while thread is still running". I still need to figure out how to close running threads when the app crashes and all the other scenarios. Is there a way to have PyQT5 use a child logger of my root logging object?

Thanks in advance...

SGaist · 10 Sept 2023, 18:53

Hi,

First thing: since you are using Qt, why not make use of QTcpServer and QTcpSocket ?
Second thing: since you are using Qt, if you make use of the classes mentioned above, you can avoid the use of threads as Qt being asynchronous, you likely won't need any thread at all which will simplify your code.

DieterV · 10 Sept 2023, 19:07

@SGaist

I'm afraid the answer to both questions would be "because I didn't know any better" :). I looked for a way to create a socket server. Since qt has it's own implementation of threads, I used those.

I'll look for example code and work from there. This seemed like a good time to bite the bullet and learn how to work with threads, but if I can avoid it, I'll gladly take that chance :).

Thanks!