Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function

[JesusM97]

Hello, I have a QTcpServer which is initialized with my window and waits for connections.
In this window I also have a QComboBox with some names. When I select a name I store it in a class variable ("senialSelected").
Then I have a button that launch a external process (QProcess). This process has a tcp client that connects to my QTcpServer.
When the tcp client is connected, I send the text selected in the QComboBox. If I want to send another name, I have to relaunch the QProcess (is made in this way because it is the way that interests me most).

My problem is that the first selected name is sended well but the next ones aren't. I have debugged the signal emited when the QComboBox changed and it is beeing emited well and before the QTcpServer receives the connection and sends the data but, in the send function, the class variable that stores the name has the old value.

class additionalWindow (QWidget):
    def __init__(self):
            super.__init__()

            self.senialComboBox = QComboBox()
            #Here i add some items (addItem(string)). I exclude this part because I think is not interesting.
            self.senialComboBox.currentIndexChanged.connect(lambda: self.senialComboBoxChanged())

            self.lanzarVentanaButton = QPushButton("Lanzar visualizador")
            self.lanzarVentanaButton.clicked.connect(lambda: self.lanzarVisualizadorColocalSenial())

            self.senialSelected = "Antorchas"
            self.unityProcess = None
            self.tcpServer = QTcpServer()
            self.tcpServer.listen(QHostAddress.Any, 5555)
            self.tcpServer.newConnection.connect(lambda: self.socketConnet())
            self.clientTcpSocket = None
            #I have excluded layout creation and management to simplify the code. The QWidget is showed well

    def socketConnet(self):
        self.clientTcpSocket = self.tcpServer.nextPendingConnection()
        print("new client")
        self.clientTcpSocket.readyRead.connect(lambda: self.socketReceive())
        self.clientTcpSocket.disconnected.connect(lambda: self.clientTcpSocket.deleteLater())
        self.socketSend()

    def socketReceive(self):
        print("receiving data...")
        recvStr = self.clientTcpSocket.readAll()
        print(recvStr)

    def socketSend(self):
        #Here I am sending the old value (always the first value I have selected)
        msgEncoded = QByteArray(self.senialSelected.encode("UTF-8"))
        self.clientTcpSocket.write(msgEncoded)
        print("Sending: " + self.senialSelected)


    def senialComboBoxChanged(self):
        senial = self.senialComboBox.currentText()
        print("Selected: "+ senial)
        #Here the variable is being modified and this function executes before the send function
        self.senialSelected = senial
        self.senialImage.setPixmap("./Resources/img/"+senial+"_image.png")


    def lanzarVisualizadorColocalSenial(self):
        if self.unityProcess:
            self.unityProcess.finished.disconnect()
            self.unityProcess.terminate()
        self.unityProcess = QProcess()
        self.unityProcess.start("CueSelectionExe/CuesPosSelection.exe")
        self.unityProcess.waitForStarted()
        self.unityProcess.finished.connect(lambda: self. procesoTerminado())


    def procesoTerminado(self):
        print("Closing window")
        self.unityProcess = None
        self.close()

Console print example:

What can be happening? Thanks!

[JonB]

@JesusM97 said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

Yes! Now the variable is beeing sending with the right value. Thank!

Yes, I kind of had a feeling this might be the root cause of the problem.... What this is telling you is that when you closed the first additionalWindow it did not destroy that window/widget, it left it in existence but not visible. That meant the self.tcpServer = QTcpServer() did not get destroyed, and it was still "listening" for a new connection from self.tcpServer.listen(QHostAddress.Any, 5555). When you then created a fresh instance you executed that line a second time. You then had two instances of QTcpServer both listening for any incoming connections on port 5555. That is bad! It turned out that the first one created accepted the new client connection, and that instance still had the old self.senialSelected = senial value from the first self.senialComboBox = QComboBox(), which you could no longer see/interact with. Ugh!

You have a real problem with Python lambdas here, especially with Qt. Your problem comes from the fact that you are using lambdas as your slots on signals. This is an issue with Python/Qt. Let me explain.

If you have a class with:

class MyClass(QObject):
    def __init__(self):
        self.something.signal.connect(self.slot)

    def slot(self):
        pass

the signal is connected to a slot which is a method in self. If you remove all live references to a created MyClass instance (e.g. you close the window) Python will do what it normally does: once all references gone/closed, its garbage collector will notice and release the MyClass instance (together with e.g. any self.tcpServer = QTcpServer() member variables, so the QTcpServer() gets freed/destroyed. This is a good situation.

The problem comes when you use a lambda in a connect:

self.something.signal.connect(lambda: self.slot())

Now Python creates an "anonymous" function/method for a lambda. And Python sees that as a "live" reference to the self class instance. It is not clever enough to understand that "anonymous" function/method can be destroyed if the self instance is destroyed, unlike the case with a connect(self.method). So it thinks the instance is "still in use", and so does not release/destroy the instance.

This is a real problem. Basically for the code you showed so far you could have not used any lambdas and written member slot methods instead, then you would not have had this problem. If you do have to use a lambda for a slot you will experience this nasty issue.

If you need to read up about this try Googling for something about PyQt5 or PySide2 together with slot and something about instance not freed, there must be something out there for this.

[JonB]

@JesusM97
Start by appending line

print(self.senialSelected)

immediately after self.senialSelected = senial line.

[JesusM97]

@JonB
I modified the funcion as follows:

 def senialComboBoxChanged(self):
        senial = self.senialComboBox.currentText()
        self.senialSelected = senial
        print("Selected: " + senial)
        print("Selected: " + self.senialSelected)
        self.senialImage.setPixmap("./Resources/img/"+senial+"_image.png")

Console output:

[JonB]

@JesusM97
OK. I agree I am having trouble spotting how this is happening.

At the end of procesoTerminado() you do a self.close(). So how do you "get back into this instance" next time around? I am wondering if you somehow have two instances of additionalWindow and it is sending again on the previous instance? Does the first self.clientTcpSocket ever get disconnected/call self.clientTcpSocket.deleteLater()?

[JesusM97]

@JonB I have a QMainWindow which is the main window of my app. In this window I have a QListWidget with some QListItems. When I double click an item this additional window is created.

#This is the SLOT function when an item is double clicked
 if self.additionalWindow:
            self.additionalWindow.close()
self.additionalWindow = AdditionalWindow(parent=self, partida=self.partida, nombre="Editar señal",
                                                     tipoVentana=2, layout=QVBoxLayout(), width=500, height=300)

By now, all the data is processed inside this additional window and no data is stored in the main window.

Does the first self.clientTcpSocket ever get disconnected/call self.clientTcpSocket.deleteLater()?

How can I check that? Should I override that method?

[JonB]

@JesusM97
In that case you create a new instance of AdditionalWindow, and a new self.senialComboBox = QComboBox() each time. It certainly won't remember the previous value you set self.senial to.

Meanwhile let's have:

self.clientTcpSocket.disconnected.connect(lambda: print("self.clientTcpSocket.disconnected") : self.clientTcpSocket.deleteLater())

I don't know if the : between the two statements is correct for Python/lambda on one line, you'll have to sort that out. But I'd like to know whether the original/first self.clientTcpSocket ever gets disconnected....

[JesusM97]

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

hat case you create a new instance of AdditionalWindow, and a new self.senialComboBox = QComboBox() each time. It certainly won't remember the previous value you set self.senial to.

self.senial is a function variable, not the class one. self.senialSelected is the class variable. It default value is "Antorchas". The first selected value which works well is shown in the console screenshots (first two prints). I don't know if this is exactly what you mean.

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

I don't know if the : between the two statements is correct for Python/lambda on one line, you'll have to sort that out. But I'd like to know whether the original/first self.clientTcpSocket ever gets disconnected....

I created a function and I connected the disconnected SIGNAL with that function:

    self.clientTcpSocket.disconnected.connect(lambda: self.deleteTcpSocket())

    def deleteTcpSocket(self):
        print("self.clientTcpSocket.disconnected")
        self.clientTcpSocket.deleteLater()

This is the console output:

[JonB]

@JesusM97
OK.
Another thought: because of the way you are using lambdas for your connects I believe you are preventing Python from destroying your additionalWindow instances. I now wonder whether multiple instances of self.tcpServer = QTcpServer() are left listening for connections.

At the end of deleteTcpSocket(self) append

self.tcpServer.close()

Any change in behaviour?

[JesusM97]

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

Another thought: because of the way you are using lambdas for your connects I believe you are preventing Python from destroying your additionalWindow instances. I now wonder whether multiple instances of self.tcpServer = QTcpServer() are left listening for connections.

I tried to add the attribute DeleteOnClose to the additional window to ensure it is fully destroyed but sometimes I get an exit error which seems to be an access violation (l read it here) and I don't know why is happening so I eliminated that attribute.
self.setAttribute(Qt.WA_DeleteOnClose, True)

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

At the end of deleteTcpSocket(self) append
self.tcpServer.close()

Any change in behaviour?

Yes! Now the variable is beeing sending with the right value. Thank!

[JonB]

@JesusM97 said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

Yes! Now the variable is beeing sending with the right value. Thank!

Yes, I kind of had a feeling this might be the root cause of the problem.... What this is telling you is that when you closed the first additionalWindow it did not destroy that window/widget, it left it in existence but not visible. That meant the self.tcpServer = QTcpServer() did not get destroyed, and it was still "listening" for a new connection from self.tcpServer.listen(QHostAddress.Any, 5555). When you then created a fresh instance you executed that line a second time. You then had two instances of QTcpServer both listening for any incoming connections on port 5555. That is bad! It turned out that the first one created accepted the new client connection, and that instance still had the old self.senialSelected = senial value from the first self.senialComboBox = QComboBox(), which you could no longer see/interact with. Ugh!

You have a real problem with Python lambdas here, especially with Qt. Your problem comes from the fact that you are using lambdas as your slots on signals. This is an issue with Python/Qt. Let me explain.

If you have a class with:

class MyClass(QObject):
    def __init__(self):
        self.something.signal.connect(self.slot)

    def slot(self):
        pass

the signal is connected to a slot which is a method in self. If you remove all live references to a created MyClass instance (e.g. you close the window) Python will do what it normally does: once all references gone/closed, its garbage collector will notice and release the MyClass instance (together with e.g. any self.tcpServer = QTcpServer() member variables, so the QTcpServer() gets freed/destroyed. This is a good situation.

The problem comes when you use a lambda in a connect:

self.something.signal.connect(lambda: self.slot())

Now Python creates an "anonymous" function/method for a lambda. And Python sees that as a "live" reference to the self class instance. It is not clever enough to understand that "anonymous" function/method can be destroyed if the self instance is destroyed, unlike the case with a connect(self.method). So it thinks the instance is "still in use", and so does not release/destroy the instance.

This is a real problem. Basically for the code you showed so far you could have not used any lambdas and written member slot methods instead, then you would not have had this problem. If you do have to use a lambda for a slot you will experience this nasty issue.

If you need to read up about this try Googling for something about PyQt5 or PySide2 together with slot and something about instance not freed, there must be something out there for this.

[JesusM97]

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

This is a real problem. Basically for the code you showed so far you could have not used any lambdas and written member slot methods instead, then you would not have had this problem. If you do have to use a lambda for a slot you will experience this nasty issue.

I tried to use the connect method without lambda functions but I had problems. For example, in my main window class I have a QListWidget. I tried to connect the double click event to a funtion in the __init__() in this way:
self.listaFases.itemDoubleClicked.connect(self.editarItem(self.listaFases))
The problem was that the function self.editarItem() was called at this right moment, when I connected the signal to that function. And this happens to me with other SIGNALS. The solution I found on the Internet was to define the SLOT functions as lambda funtions. If there is any other way to solve it I can modify the code without problems, it is not very big yet.

[JonB]

@JesusM97 said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

self.listaFases.itemDoubleClicked.connect(self.editarItem(self.listaFases))

I do not understand this. It is (should not) be legal? You can do that (pass a parameter) with a lambda, but not with the non-lambda connection. A normal, non-lambda connection must look like:

self.object.signal.connect(self.slot_method)

No more and no less. See that there are no parentheses and no parameters allowed.

For this case, where the self.listaFases parameter is the sender of the signal (self.listaFases.itemDoubleClicked.connect()), you can use QObject.sender() in the slot to obtain that without using a lambda with a parameter. It's a shame because lambdas are preferable to that.

I did not say you cannot use lambdas in Python. I used them when I wrote Python/PyQt5 code, where I had to. What I said is you must be careful of the consequences, and understand them, because of this Python-no-release-instance-with-lambda-connection issue. Like I said if you want to understand the consequences you need to Google judiciously for this. Examples:

• https://stackoverflow.com/questions/47941743/lifetime-of-object-in-lambda-connected-to-pyqtsignal
• https://stackoverflow.com/questions/60771418/does-using-a-lambda-slot-function-in-a-signal-cause-a-memory-leak
• https://stackoverflow.com/questions/61871629/memory-profiler-while-using-lambda-expression-to-connect-slots
• https://stackoverflow.com/questions/35819538/using-lambda-expression-to-connect-slots-in-pyqt/48501804#48501804

but there are probably more discussions. I Googled pyqt5 lambda free instance, but there may be other hits if you try various similar terms.

UPDATE
Oohh, I discovered I myself raised this issue on this forum 4 years ago! Read through
https://forum.qt.io/topic/92825/python-pyqt-qdialog-with-connect-lambda-leaks
I see my last post there, https://forum.qt.io/topic/92825/python-pyqt-qdialog-with-connect-lambda-leaks/13, discusses answer/code I got back from PyQt5 mailing list, though I did not deploy it myself.

You can see the whole topic is "nasty"....

[JesusM97]

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

I do not understand this. It is (should not) be legal? You can do that (pass a parameter) with a lambda, but not with the non-lambda connection. A normal, non-lambda connection must look like:
self.object.signal.connect(self.slot_method)

No more and no less. See that there are no parentheses and no parameters allowed.

Maybe thats the problem. I was defining the SLOTS in a wrong way (including the parentheses and using parameters inside the connect) and python executed the function in that moment (when I connected it to the SIGNAL).
I can't try it now because I leave the computer but I will try as soon as possible and post the results.

Thanks!

[JonB]

@JesusM97 said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

I was defining the SLOTS in a wrong way (including the parentheses and using parameters inside the connect) and python executed the function in that moment (when I connected it to the SIGNAL).

Yes, that was what was happening the way you wrote it.

One final update on this whole Python lambda-slot thing, for you or anyone else reading this.

If I understand correctly, to allow destruction/garbage collection of a QObject which has lambda(s) on its signal(s), you can achieve that if you explicitly disconnect ALL connect()s which have a lambda-slot. But you must be very careful to do that.

In your example it was even worse: your additionalWindow has:

    self.tcpServer = QTcpServer()
    self.tcpServer.newConnection.connect(lambda: self.socketConnet())

So you had a member variable which itself had a lambda-connect, and that needs disconnecting too. I believe self.tcpServer.newConnection.disconnect() will disconnect all signals including lambdas? You should test this in place of the self.tcpServer.close() I got you to add to make it work. Does it still work if you do this? Then you will know what sort of thing is required.

[JesusM97]

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

So you had a member variable which itself had a lambda-connect, and that needs disconnecting too. I believe self.tcpServer.newConnection.disconnect() will disconnect all signals including lambdas? You should test this in place of the self.tcpServer.close() I got you to add to make it work. Does it still work if you do this? Then you will know what sort of thing is required.

It doesn't work. It takes the default value of self.senialSelected, not even the old one.
I will try now to replace lambda functions with normal SLOT functions and pass parameters through QObject.sender as you said.

@JonB said in Problem sending a class variable (string) via TCPSocket when it has been modified in a SLOT function:

Yes, that was what was happening the way you wrote it.

I changed some SLOTS and they work well. There are SLOTS that I can't change because sometimes I can't get the parameters through the sender. For example, I have a context menu when i right click over a QListWidget. This context menu has QActions that need the current item selected. The QSender is the QAction so I can't access any QListWidget function to get that item so I use lambda funcion to pass the paremeter through the connect method.

[SGaist]

Hi,

You don't need a lambda for that. The signal gives you the coordinates of the mouse. You can use the QListWidget::indexAt method to retrieve the index.

[JesusM97]

@SGaist Oh I didn't think on that. Thanks!