Code or tool for Qt memory usage/leaks

JonB · 7 Nov 2018, 13:09

I use PyQt/Python for Qt (desktop) development, under Linux. Given that, standard suggestions like valgrind for detecting memory leaks won't do me much good.

All I'd really like to get is a list of what QObjects (probably just QWidgets) have been allocated on the heap (Python should not do any stack objects...) and which are still in existence (i.e. not deleted). With some indication of what they are ("this is a QDialog", "this has objectName() == ...). In such a form that I can search through the undoubtedly large number of ones I don't care about to drill down to suspicious ones.

Does Qt keep a runtime, accessible list of currently existing objects it has created which I can get at from PyQt, or an external tool for this?

JonB · wrote on 20 Jul 2018, 08:57

For now I am getting what I need by walking QApplication.allWidgets() (as per @mrjj), checking for things like "no parent".

Thanks for all your suggestions though.

mrjj · 11 Jul 2018, 13:23

HI
Like
http://doc.qt.io/qt-5/qapplication.html#allWidgets
?

JonB · M mrjj 11 Jul 2018, 13:10

@mrjj
Hmm, quite possibly...!

So the first thing I was going to investigate is the use or non-use of QDialog.setAttribute(QtCore.Qt.WA_DeleteOnClose, True), which varies in existing code. I am unclear whether those which do not have this flag are left around persisting or not. Which would be very bad....!

JonB · M mrjj 11 Jul 2018, 13:10

@mrjj
As my app gets going, I discover len(QApplication.allWidgets()) == 896. I think I need to do some filtering before I start examining them... :)

kshegunov · 11 Jul 2018, 14:55

Maybe Gamma ray?

mrjj · 11 Jul 2018, 14:56

@kshegunov
Does that work with python bindings ?
@JonB wow, quite the amount of widgets :)

kshegunov · 11 Jul 2018, 14:57

@mrjj said in Code or tool for Qt memory usage/leaks:

Does that work with python bindings ?

I'm not sure, but I imagine so. In the end they are python bindings; you still get QObjects created in the background.

mrjj · wrote on 11 Jul 2018, 14:57

@kshegunov
Yeah, i assume that also as in the end, its same binary result but i just wondered.

JonB · 7 Nov 2018, 15:00

@kshegunov
Thanks. I've had a brief look. Apart from the fact that it probably does way more than I have in mind --- I only want to know what QWidgets currently exist --- I have tried to read the manual to understand. I do not install anything to find out, if I can't understand from its docs I don't bother!

What I don't see is what it actually "works on". The obvious approach would be attach to my running application. But, unlike you guys, my running application is always simply python3. I have no choice, that's how Python works! I don't know whether/see how it's going to recognise that is a Qt application....

JonB · 7 Nov 2018, 15:09

@mrjj

@JonB wow, quite the amount of widgets :)

Yes & no. This app has loads & loads of stacked widgets defined. These are effectively the many pages for the whole application (all accessed from sidebar).

That means they are all instantiated at start up time. All these pages' constructors are called as they are subjected to QStackedWidget.addWidget(). And the constructors create all the child widgets on the pages. It doesn't take long to reach 896 :(

These won't leak, and they're permanent, so I'm not worried about them. I'm more worried about the many dialogs which are created at various times, often without WA_DeleteOnClose. And figuring just how that relates to Python's reference counting and garbage collection....

mrjj · 11 Jul 2018, 17:07

@JonB
Yep all those have no choice as to die with the stack widget
So basically, you are looking for Widgets with no parent ? .
As those with no WA_DeleteOnClose , would still die with parent eventually.

I have no idea if pythons garbage collection also includes QObjects but i suspect it would be tricky if it does.

JonB · 11 Jul 2018, 17:25

@mrjj
I'm looking for repeated leaks. Whether that requires "no parent" I don't know.

I can't recall whether you are a pythonista. All our objects are allocated on the heap, there's no stack storage. w = QWidget() is the (implicit) new QWidget() C++ statement, but there's no such thing as delete! I assume(d) Python reference-counts like C# and frees when no more references, but then someone (@SGaist I think maybe) said this wasn't all there is to it and I should consider Qt's references separately, which I'm not sure about.

Yes all child widgets on the QStackedWidgets will eventually die with it on GUI exit. But I only create them once. I'm more worried about the (plentiful) dialogs which come & go all the time. They are instantiated in two ways:

QDialog(parent = None)

Yes, this dialog has no parent. You asked if I am interested in that? Having no parent like this is not a "leak". Does something give them a parent, perhaps when they're shown (I tend to use exec()). If not, or whether they are or not, what/when is deleting them?

QDialog(parent = <caller>)

This has a parent, and the parent will likely persist (e.g. it's a stacked widget page). But again, whether or not, is this dialog being created repeatedly without being freed each time it is called?

I think all I really must check for is whether the total number of existing widgets is seen to keep increasing each time I go in & out of a given dialog. Which would indeed be bad for app. And I ought be able to do this using your QApplication.allWidgets().

mrjj · 11 Jul 2018, 17:38

@JonB
Im only c++. So not sure how pythons integrates to the ownership systems
http://doc.qt.io/qt-5/objecttrees.html

The
QDialog(parent = None) and no WA_DeleteOnClose would leak in c++ but maybe python handles that ?
( on some platform, the leak would not be true. the OS would free on app termination )

QDialog(parent = <caller>) would eventually be freed but would might consume memory until app exit.

Being python noob i was wondering if its possible to use
http://www.gnosis.cx/publish/programming/metaclass_2.html
and bind a destructor to all object so you can dump objectnames of all being deallocated.
But no idea if that works with bindings :)

JonB · M mrjj 11 Jul 2018, 17:25

@mrjj
Yeah, we're both not sure how it interacts with Qt ownership :)

Not freeing till program exit is not in itself a problem to me. The problem would be if more than one instance of a dialog can be in existence at the same time. The point is that those QDialog() statements are being executed repeatedly: user clicks a button, dialog is created dynamically, shown and closed. One instance of that dialog. Then at a later date user clicks same button again. It will create dialog again. If previous one did not get deleted, that's a "leak" for me.

As I said, I think all I need --- for my simple checking --- is just the count and the types/names of the widgets from your QApplication.allWidgets(). If I see any permanent increase in that as I go in & out of various dialogs repeatedly I can investigate.

JonB · wrote on 20 Jul 2018, 08:57

For now I am getting what I need by walking QApplication.allWidgets() (as per @mrjj), checking for things like "no parent".

Thanks for all your suggestions though.