Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. Language Bindings
  4. Python/PyQt QDialog with connect lambda "leaks"
Forum Updated to NodeBB v4.3 + New Features

Python/PyQt QDialog with connect lambda "leaks"

Scheduled Pinned Locked Moved Solved Language Bindings
13 Posts 3 Posters 5.3k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • JonBJ JonB

    I have posted the following to the PyQt mailing list. I wait to see whether I get a reply. If there are any PyQt users here who can answer, or you're a PyQt user and you'd like to know, please read and/or comment.

    PyQt 5.7.

    I have a large body of existing UI code. I have spent two days commenting in & out bits of code to try to discover why some of its QDialogs "leak" after calling QDialog.exec().

    My definition of "leak" here is: after executing from somewhere else

    dlg = QDialog(self)
    QDialog.exec()
    

    the instance of the dialog stays in existence permanently (as long as the caller exists, which for me is till end of program). That means that every time that code gets executed, yet another new dialog is left around in memory, which adds up over time. All I do to test is go into the dialog and immediately close it.

    I discover this by inspecting QtWidgets.QApplication.allWidgets() and reporting all QDialogs which are still in existence. I see an ever-increasing number of these dialogs, one per each time it's constructed and executed, when & only when the code in the dialog is as follows.

    I have finally tracked down the problematic line in the dialog's __init__(). Some of them have:

    from elsewhere import ensureValidDecimal
    self.lineEdit = QLineEdit(self)
    self.lineEdit.editingFinished.connect(lambda: ensureValidDecimal(self))
    

    The vital bit is: they connect() to a lambda which references self.

    If the lambda does not need to pass self out as an argument, there will be no leak.

    If I go define (in this case) in the dialog (I actually sub-class from all my QDialogs so I can add stuff) a dedicated function to avoid the lambda:

        def selfEnsureValidDecimal(self)
            ensureValidDecimal(self)
    
        self.lineEdit.editingFinished.connect(self.selfEnsureValidDecimal)
    

    then there will also be no leak.

    I can see that at some deep level there must be a reference counting issue here. In some shape or form, the fact that we have a lambda which passes self to the outside world must mean Python/PyQt wants to keep a reference to the dialog and this must be preventing its destruction.

    But I don't know what to do about it. There is a lot of code with a lot of dialogs with all sorts of code attached. So I need some kind of explanation of what exactly can or cannot be done here, what to look for in code, etc. Note that I do not wish to use QDialog.setAttribute(QtCore.Qt.WA_DeleteOnClose, True) on all my dialogs (I believe that would solve the leak, but it's not the point). What must I not do if I do not expect such a self-reference to be left around preventing Python/PyQt from actually freeing up the dialog?

    EDIT

    Oh dear! Googling hard, I have just come across https://stackoverflow.com/a/48501804/489865, stating:

    ​Beware! As soon as you connect your signal to a lambda slot with a reference to self, your widget will not be garbage-collected! That's because lambda creates a closure with yet another uncollectable reference to the widget.
    Thus, self.someUIwidget.someSignal.connect(lambda p: self.someMethod(p)) is very evil :)

    which confirms exactly what I have spent days discovering and typing in :(

    So, that is indeed the problem? And so I am required to go through all code looking for exactly the pattern to change it? Does this only apply to QWidget.connect(), I only need to look at all connect() calls, or are there other similar cases?

    kshegunovK Offline
    kshegunovK Offline
    kshegunov
    Moderators
    wrote on last edited by
    #2

    Well, my friend, as you often like to point out how C++ is too complicated now is it time for me to gloat? :)
    Joking aside, is it possible that calling QObject::deleteLater after you've returned from the dialog would solve this particular conundrum?

    Read and abide by the Qt Code of Conduct

    JonBJ 1 Reply Last reply
    2
    • kshegunovK kshegunov

      Well, my friend, as you often like to point out how C++ is too complicated now is it time for me to gloat? :)
      Joking aside, is it possible that calling QObject::deleteLater after you've returned from the dialog would solve this particular conundrum?

      JonBJ Online
      JonBJ Online
      JonB
      wrote on last edited by
      #3

      @kshegunov
      I have never defended Python! I don't have a choice for my Qt work, if I did I'd prefer C++. I just like C# or plain C.

      deleteLater() is a possibility, requires code changing. It's not "one dialog" it's "the code [I didn't write] has loads & loads of dialogs, I have absolutely no idea which ones might have "connect-lambda-self"s (or maybe other idioms?) somewhere in them". If I deleteLater() after all of them (lots of places), seems I might just as well QDialog.setAttribute(QtCore.Qt.WA_DeleteOnClose, True) on all of them in initializer. That one worries because presumably then code must not be accessing any members of dialog after it's been closed, and I think some code does want to do that, and that would go wrong, right?

      Sigh.

      kshegunovK 1 Reply Last reply
      0
      • JonBJ JonB

        @kshegunov
        I have never defended Python! I don't have a choice for my Qt work, if I did I'd prefer C++. I just like C# or plain C.

        deleteLater() is a possibility, requires code changing. It's not "one dialog" it's "the code [I didn't write] has loads & loads of dialogs, I have absolutely no idea which ones might have "connect-lambda-self"s (or maybe other idioms?) somewhere in them". If I deleteLater() after all of them (lots of places), seems I might just as well QDialog.setAttribute(QtCore.Qt.WA_DeleteOnClose, True) on all of them in initializer. That one worries because presumably then code must not be accessing any members of dialog after it's been closed, and I think some code does want to do that, and that would go wrong, right?

        Sigh.

        kshegunovK Offline
        kshegunovK Offline
        kshegunov
        Moderators
        wrote on last edited by
        #4

        @JonB said in Python/PyQt QDialog with connect lambda "leaks":

        That one worries because presumably then code must not be accessing any members of dialog after it's been closed, and I think some code does want to do that, and that would go wrong, right?

        Right. I would prefer the deleteLater to the attribute, if it works at all, as you have a clear idea where is the deletion happening - just after you return control to the event loop.

        Read and abide by the Qt Code of Conduct

        JonBJ 1 Reply Last reply
        0
        • kshegunovK kshegunov

          @JonB said in Python/PyQt QDialog with connect lambda "leaks":

          That one worries because presumably then code must not be accessing any members of dialog after it's been closed, and I think some code does want to do that, and that would go wrong, right?

          Right. I would prefer the deleteLater to the attribute, if it works at all, as you have a clear idea where is the deletion happening - just after you return control to the event loop.

          JonBJ Online
          JonBJ Online
          JonB
          wrote on last edited by JonB
          #5

          @kshegunov
          Yes, I agree, but that requires me to go through code finding all cases where a dialog is shown (e.g. exec()) and appending that line. And I still don't really know when it's "safe", because I don't know whether there is a later code path which requires to access the dialog's information after closure.

          Since I haven't got time to do that, and since it's easy to do because I sub-class all my QDialogs via my own class, for now at least I have put QDialog.setAttribute(QtCore.Qt.WA_DeleteOnClose, True) into my derived class's constructor (if you can really call it that in Python, it's the __init__() method). Callers could then undo that immediately after construction if they want (i.e. it's just the default behaviour), if & when I ever get time to look through. I'll have to run with it and see....

          None of this actually addresses the fundamental "closure-leakage" situation I describe, and I don't know if that principle could be applying in other cases in code. I knew there was a good reason why I never want to use lambdas in languages (at least Python)! Does C have them? No, never needed them. Does C#? Yes, in later versions, but I already have avoided them deliberately. Does C++ have them? Yes in newer versions, but then in practice C++ programs leak like a sieve anyway, so I don't expect much ;) "KISS".

          Thanks for your input.

          kshegunovK 1 Reply Last reply
          0
          • JonBJ JonB

            @kshegunov
            Yes, I agree, but that requires me to go through code finding all cases where a dialog is shown (e.g. exec()) and appending that line. And I still don't really know when it's "safe", because I don't know whether there is a later code path which requires to access the dialog's information after closure.

            Since I haven't got time to do that, and since it's easy to do because I sub-class all my QDialogs via my own class, for now at least I have put QDialog.setAttribute(QtCore.Qt.WA_DeleteOnClose, True) into my derived class's constructor (if you can really call it that in Python, it's the __init__() method). Callers could then undo that immediately after construction if they want (i.e. it's just the default behaviour), if & when I ever get time to look through. I'll have to run with it and see....

            None of this actually addresses the fundamental "closure-leakage" situation I describe, and I don't know if that principle could be applying in other cases in code. I knew there was a good reason why I never want to use lambdas in languages (at least Python)! Does C have them? No, never needed them. Does C#? Yes, in later versions, but I already have avoided them deliberately. Does C++ have them? Yes in newer versions, but then in practice C++ programs leak like a sieve anyway, so I don't expect much ;) "KISS".

            Thanks for your input.

            kshegunovK Offline
            kshegunovK Offline
            kshegunov
            Moderators
            wrote on last edited by kshegunov
            #6

            @JonB said in Python/PyQt QDialog with connect lambda "leaks":

            Yes, I agree, but that requires me to go through code finding all cases where a dialog is shown (e.g. exec()) and appending that line. And I still don't really know when it's "safe", because I don't know whether there is a later code path which requires to access the dialog's information after closure.

            If you ask me, find a better code 'cause this one seems very suspicious ... ;)

            practice C++ programs leak like a sieve

            Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

            Read and abide by the Qt Code of Conduct

            JonBJ 1 Reply Last reply
            0
            • kshegunovK kshegunov

              @JonB said in Python/PyQt QDialog with connect lambda "leaks":

              Yes, I agree, but that requires me to go through code finding all cases where a dialog is shown (e.g. exec()) and appending that line. And I still don't really know when it's "safe", because I don't know whether there is a later code path which requires to access the dialog's information after closure.

              If you ask me, find a better code 'cause this one seems very suspicious ... ;)

              practice C++ programs leak like a sieve

              Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

              JonBJ Online
              JonBJ Online
              JonB
              wrote on last edited by
              #7

              @kshegunov

              If you ask me, find a better code 'cause this one seems very suspicious ... ;)

              32K lines of existing code I inherited. Whadda you want me to do about it? ;)

              Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

              I do not doubt that you are (pretty) perfect :) However, have a look through the code posts on this forum of users' real-world Qt/C++ code and send me £1 for each time you spot where they have left a leak....!

              J.HilkJ 1 Reply Last reply
              0
              • JonBJ JonB

                @kshegunov

                If you ask me, find a better code 'cause this one seems very suspicious ... ;)

                32K lines of existing code I inherited. Whadda you want me to do about it? ;)

                Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

                I do not doubt that you are (pretty) perfect :) However, have a look through the code posts on this forum of users' real-world Qt/C++ code and send me £1 for each time you spot where they have left a leak....!

                J.HilkJ Offline
                J.HilkJ Offline
                J.Hilk
                Moderators
                wrote on last edited by
                #8

                @JonB said in Python/PyQt QDialog with connect lambda "leaks":

                32K lines of existing code I inherited. Whadda you want me to do about it? ;)

                hopfuly it's at least decently structured and named/commented !?
                Because I remember legacy code that was handed to me with things like 3000 lines inside the main() function and stuff.

                Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

                I do not doubt that you are (pretty) perfect :) However, have a look through the code posts on this forum of users' real-world Qt/C++ code and send me £1 for each time you spot where they have left a leak....!

                I have to admit, early on in my days I had a slack stance on that as well. Usually with the mindset, "That class's instance will exist as long as the program is running anyway ".
                Until it didn't. Lessen learend, eventually.


                Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct


                Q: What's that?
                A: It's blue light.
                Q: What does it do?
                A: It turns blue.

                JonBJ kshegunovK 2 Replies Last reply
                0
                • J.HilkJ J.Hilk

                  @JonB said in Python/PyQt QDialog with connect lambda "leaks":

                  32K lines of existing code I inherited. Whadda you want me to do about it? ;)

                  hopfuly it's at least decently structured and named/commented !?
                  Because I remember legacy code that was handed to me with things like 3000 lines inside the main() function and stuff.

                  Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

                  I do not doubt that you are (pretty) perfect :) However, have a look through the code posts on this forum of users' real-world Qt/C++ code and send me £1 for each time you spot where they have left a leak....!

                  I have to admit, early on in my days I had a slack stance on that as well. Usually with the mindset, "That class's instance will exist as long as the program is running anyway ".
                  Until it didn't. Lessen learend, eventually.

                  JonBJ Online
                  JonBJ Online
                  JonB
                  wrote on last edited by
                  #9

                  @J.Hilk said in Python/PyQt QDialog with connect lambda "leaks":

                  hopfuly it's at least decently structured and named/commented !?

                  Don't be silly! :)

                  1 Reply Last reply
                  2
                  • J.HilkJ J.Hilk

                    @JonB said in Python/PyQt QDialog with connect lambda "leaks":

                    32K lines of existing code I inherited. Whadda you want me to do about it? ;)

                    hopfuly it's at least decently structured and named/commented !?
                    Because I remember legacy code that was handed to me with things like 3000 lines inside the main() function and stuff.

                    Hey! My programs don't leak, and if they ever do it's a little drip, no gushing.

                    I do not doubt that you are (pretty) perfect :) However, have a look through the code posts on this forum of users' real-world Qt/C++ code and send me £1 for each time you spot where they have left a leak....!

                    I have to admit, early on in my days I had a slack stance on that as well. Usually with the mindset, "That class's instance will exist as long as the program is running anyway ".
                    Until it didn't. Lessen learend, eventually.

                    kshegunovK Offline
                    kshegunovK Offline
                    kshegunov
                    Moderators
                    wrote on last edited by
                    #10

                    @J.Hilk said in Python/PyQt QDialog with connect lambda "leaks":

                    Because I remember legacy code that was handed to me with things like 3000 lines inside the main() function and stuff.

                    The pinnacle of programming for me when I was handed over 60k+ fortran "scientific code" with not one function in it - everything in the main. The globals (i.e. common blocks) were several pages long and everything was "controlled" through infinitely long ifs. Sufficed to say, no comments were thought to be needed ... So it's never the language, it's the programmer (or lack thereof) that molds the code into life.

                    I have to admit, early on in my days I had a slack stance on that as well. Usually with the mindset, "That class's instance will exist as long as the program is running anyway ".

                    Leave water dropping on a piece of stone and it'd eat at it eventually ... it wouldn't matter if it's granite, basalt or sandstone, it would eventually succumb to the simple droplet. The moral is: If you're smart and have a self-preservation instinct, don't leak stuff! :)

                    Read and abide by the Qt Code of Conduct

                    JonBJ 1 Reply Last reply
                    1
                    • kshegunovK kshegunov

                      @J.Hilk said in Python/PyQt QDialog with connect lambda "leaks":

                      Because I remember legacy code that was handed to me with things like 3000 lines inside the main() function and stuff.

                      The pinnacle of programming for me when I was handed over 60k+ fortran "scientific code" with not one function in it - everything in the main. The globals (i.e. common blocks) were several pages long and everything was "controlled" through infinitely long ifs. Sufficed to say, no comments were thought to be needed ... So it's never the language, it's the programmer (or lack thereof) that molds the code into life.

                      I have to admit, early on in my days I had a slack stance on that as well. Usually with the mindset, "That class's instance will exist as long as the program is running anyway ".

                      Leave water dropping on a piece of stone and it'd eat at it eventually ... it wouldn't matter if it's granite, basalt or sandstone, it would eventually succumb to the simple droplet. The moral is: If you're smart and have a self-preservation instinct, don't leak stuff! :)

                      JonBJ Online
                      JonBJ Online
                      JonB
                      wrote on last edited by
                      #11

                      @kshegunov , @J-Hilk
                      Don't tell anybody I said so, but this perfectly legitimate wikipedia link is actually my favorite language.

                      It doesn't mess about allowing any such thing as comments in the language in any shape or form, and the standard Hello World code is:

                      ++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.
                      

                      Programmers are too soft these days....

                      J.HilkJ 1 Reply Last reply
                      0
                      • JonBJ JonB

                        @kshegunov , @J-Hilk
                        Don't tell anybody I said so, but this perfectly legitimate wikipedia link is actually my favorite language.

                        It doesn't mess about allowing any such thing as comments in the language in any shape or form, and the standard Hello World code is:

                        ++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.
                        

                        Programmers are too soft these days....

                        J.HilkJ Offline
                        J.HilkJ Offline
                        J.Hilk
                        Moderators
                        wrote on last edited by
                        #12

                        @JonB said in Python/PyQt QDialog with connect lambda "leaks":

                        Programmers are too soft these days....

                        I'm just glad, that the EDSAC days are over :-)


                        Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct


                        Q: What's that?
                        A: It's blue light.
                        Q: What does it do?
                        A: It turns blue.

                        JonBJ 1 Reply Last reply
                        0
                        • J.HilkJ J.Hilk

                          @JonB said in Python/PyQt QDialog with connect lambda "leaks":

                          Programmers are too soft these days....

                          I'm just glad, that the EDSAC days are over :-)

                          JonBJ Online
                          JonBJ Online
                          JonB
                          wrote on last edited by JonB
                          #13

                          For the original problem, and https://stackoverflow.com/a/48501804/489865, stating "a connected lambda which references self will not be garbage collected", I received the following code function from the PyQt mailing list. I am not going to use it myself unless I have to (I prefer to rewrite code so as not to use self-referencing lambdas), but I paste it here (untested; and the author states it's for Python 2.7 but should work for Python 3) in case other readers in future might be interested.

                          def connect_lambda(bound_signal, self, func, **kw):
                              # It can be used like this:
                              # Instead of
                              #   self.editingFinished.connect(lambda: self.whatever())
                              # do
                              #   connect_lambda(self.editingFinished, self, lambda self: self.whatever())
                          
                              import weakref
                              r = weakref.ref(self)
                              del self
                              num_args = func.__code__.co_argcount - 1
                              if num_args < 0:
                                  raise TypeError('lambda must take at least one argument')
                          
                              def slot(*args):
                                  ctx = r()
                                  if ctx is not None:
                                      if len(args) != num_args:
                                          args = args[:num_args]
                                      func(ctx, *args)
                          
                              bound_signal.connect(slot, **kw)
                          
                          1 Reply Last reply
                          3

                          • Login

                          • Login or register to search.
                          • First post
                            Last post
                          0
                          • Categories
                          • Recent
                          • Tags
                          • Popular
                          • Users
                          • Groups
                          • Search
                          • Get Qt Extensions
                          • Unsolved