DSO Plugin Qt6 Widgets
-
Keywords: Modular Programming, Plugin, Plugins, DSO, dll, DLL, QApplication, system resource leaks
If this is a "repeat issue" please point me to a hyper link. I've failed to find it so far.
I'm developing program(s) and library API in C and C++ on Debian GNU/Linux 12 (and other linux distros) using APT to install Qt6 development packages. I wish to make the Qt6 widgets part of it, but not be required, and available as a run-time on-the-fly optional DSO (dynamic shared object) library (plugins). I have it working, but it's clear that Qt6, like most other like-systems, has issues with being used as a module. I have tried using GTK3 widgets as a module this way, and I have that working too; but it's more broken than the Qt6 version of it. I suspect that the first QApplication object leaks files even before it is destroyed. I could track that down and fix it for you, if you are receptive.
I'm very interested in what can be called "robust coding".
I'm just using the Qt widgets, and a QApplication object seems to be a required process wide singleton object; though it's not presented with a user interface that makes it a singleton object. I have found that a QApplication object leaks a lot of system resources; that is, it does not cleanup after itself. I was thinking that maybe it would be more robust if a QApplication object could cleanup after itself; or at least document it's limitations. For example, after destroying a QApplication object there are many file descriptors left open; but even so, I'm still able to get a QApplication restarted (recreated) many times. I have not been able to create more than one at a time in a process.
It is mostly by design that QApplication is the way it is; so this is a poke at core of Qt design.
I have a lot more to share about this, but at this point I feel like I'm just "poking the bear". Please take this critical analysis with some humor. Also, I do have small example codes that I can share, if there is interest. I can also find related "bugs" for you, if you like.
RAMBLING:
At a minimum, QApplication should at least document this usage limitation. Then I could call it robust. At best, add cleanup to QApplication objects. And even better, add a library destructor, and the ability to create and destroy any number of QApplication objects in a process. This is really not as crazy as it seems. I'd say that, in general, that code that does not leak system resources is better code; even in common use cases. There will be goodness spill-over effects that you can't see yet. I guarantee it. If you're not going to call this "bad use case" a bug, than you must document it in the QApplication documentation; otherwise it's a BUG by the UNIX-like standard. These kinds of (crazy) limitations (like in libc) are documented in man pages all the time. Example: thread safeness of functions.This is labeled as a Question so: Q: Is this of interest to "core" Qt developers?
-
Hi and welcome to devnet,
For the documentation side, QCoreApplication and its subclasses all state that there must be only one instance of it at any time in the Detailed Description of the classes.
On the improvement side that you are thinking about, you should bring the subject to the development mailing list where you will find Qt's developers/maintainers. This forum is more user oriented.
Interested in AI ? www.idiap.ch
Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct -
Hi and welcome to devnet,
For the documentation side, QCoreApplication and its subclasses all state that there must be only one instance of it at any time in the Detailed Description of the classes.
On the improvement side that you are thinking about, you should bring the subject to the development mailing list where you will find Qt's developers/maintainers. This forum is more user oriented.
@SGaist Thanks so much for replying. The response was much more positive than I expected. I'll reformulate this question given the new found wisdom that you have bestowed upon me; and then submit to "development mailing list" as you suggest.
For the documentation side, QCoreApplication and its subclasses all state that there must be only one instance of it at any time in the Detailed Description of the classes.
I was confused about that wording, when I saw it in the documentation in in both QCoreApplication and QApplication. I now take that to mean there can only be zero or one QCoreApplication object in a process at a given time. I guess most people take the zero as implied. It's an English statement not a math statement.
What I observe is that I can start with 0 and make 1, and then destroy it making something like a half of one (due to resource leaks), and then I can bring it back to 1, and then back to half; and I can not easily bring it back to 0.
Thanks again.
cheers
lance -
@SGaist Thanks so much for replying. The response was much more positive than I expected. I'll reformulate this question given the new found wisdom that you have bestowed upon me; and then submit to "development mailing list" as you suggest.
For the documentation side, QCoreApplication and its subclasses all state that there must be only one instance of it at any time in the Detailed Description of the classes.
I was confused about that wording, when I saw it in the documentation in in both QCoreApplication and QApplication. I now take that to mean there can only be zero or one QCoreApplication object in a process at a given time. I guess most people take the zero as implied. It's an English statement not a math statement.
What I observe is that I can start with 0 and make 1, and then destroy it making something like a half of one (due to resource leaks), and then I can bring it back to 1, and then back to half; and I can not easily bring it back to 0.
Thanks again.
cheers
lance@Lance-Arsenault No reason to be negative with well written and constructive question/criticism :-)
Out of curiosity, how did you discover the leaks ? Typically Valgrind is known to have a lot of false positives when it comes to Qt.
As for Qt base applications, the rule is: the QCoreApplication (or one of its subclass) must be the first Qt object created (it's usually done in the main thread) as it does the internal initialization/setup.
If you need to access the app object in your code there is the
qAppmacro that returns it. But indeed, you cannot create such object multiple times in a process.Interested in AI ? www.idiap.ch
Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct -
@Lance-Arsenault No reason to be negative with well written and constructive question/criticism :-)
Out of curiosity, how did you discover the leaks ? Typically Valgrind is known to have a lot of false positives when it comes to Qt.
As for Qt base applications, the rule is: the QCoreApplication (or one of its subclass) must be the first Qt object created (it's usually done in the main thread) as it does the internal initialization/setup.
If you need to access the app object in your code there is the
qAppmacro that returns it. But indeed, you cannot create such object multiple times in a process.Out of curiosity, how did you discover the leaks ?
Sorry, I have two flavors and multiple levels of answers to this question:
- From my experience I knew there would be leaks. I have too much experience, and this kind of thing just keeps showing up everywhere I look. I've worked on developing "valgrind-like" systems in the past.
2a. Short answer: Run a test program while looking in the linux /proc/ file system. This monitoring method is very linux (or UNIX) specific, seeing as the /proc file system does not exist on some operating systems (OSs) (pretty sure) . I think /proc is what commands like ps, top, and htop use. I think of /proc as a window into the current state of the OS processes (at least, if not more).
2b. More specific answer: I run a test program in one terminal window and in another I run at various times "ls -thlF --color=auto /proc/$PID/fd/" where $PID is the process ID of the first program.
I can send you the code if you like, but it's not so "neat/refined".
Also: I would have been very surprised if I found no system resource leaks in the QCoreApplication class object (et al.). In any case, it was better than I expected. I expected more leaks than what I found, and I do not expect the recreation to not crash the program.
cheers
lance -
Out of curiosity, how did you discover the leaks ?
Sorry, I have two flavors and multiple levels of answers to this question:
- From my experience I knew there would be leaks. I have too much experience, and this kind of thing just keeps showing up everywhere I look. I've worked on developing "valgrind-like" systems in the past.
2a. Short answer: Run a test program while looking in the linux /proc/ file system. This monitoring method is very linux (or UNIX) specific, seeing as the /proc file system does not exist on some operating systems (OSs) (pretty sure) . I think /proc is what commands like ps, top, and htop use. I think of /proc as a window into the current state of the OS processes (at least, if not more).
2b. More specific answer: I run a test program in one terminal window and in another I run at various times "ls -thlF --color=auto /proc/$PID/fd/" where $PID is the process ID of the first program.
I can send you the code if you like, but it's not so "neat/refined".
Also: I would have been very surprised if I found no system resource leaks in the QCoreApplication class object (et al.). In any case, it was better than I expected. I expected more leaks than what I found, and I do not expect the recreation to not crash the program.
cheers
lanceIf there are valid leaks then you should create a minimal, compilable example and report it in the bug tracker so they get fixed.
Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
Visit the Qt Academy at https://academy.qt.io/catalog -
If there are valid leaks then you should create a minimal, compilable example and report it in the bug tracker so they get fixed.
I built and installed the latest qt6 from source (builds very nicely). In looking at the source it looks like QCoreApplication uses static data structures (classes) very heavily. Which could explain the QCoreApplication objects ability to recreate itself without recreating all of itself. At present it is a big "can of worms" to me, and is near impossible to follow the code, at least in a short time. Qt's trend to wrap everything into Qt like cases makes code very hard to follow for a "non-Qt person". I understand that the point is to make Qt portable. Code portability has this complexity as a cost, at least in Qt.
I'm very doubtful that I could convince any of the "core: Qt developers" to consider fixing system resource leaks in QCoreApplication. As best I can tell, it's designed to be a little leaky, they know it and do not care. Their argument will be that it only leaks across object recreation which on average means that it does not leak over the long time scale of many recreate cycles. [Though that's another thing I could check; how it leaks over long times (many many recreates).] I'm still a little worried that it will leak memory, given it leaks files. /proc/PID would be my friend to do that check too.
I've had this "discussion" with the developers of GTK and they are adamant about not making gtk_init() robust, that is making a corresponding destructor for gtk_init() (that's their main loop exec() object constructor). I actually talked in person with one of the developers.
Qt6 is still beating the pants off of GTK in this loading widgets as a DSO module contest. GTK can't even recreate a main-loop thing. It just crashes the program and therefore it is not possible to unload a DSO that has GTK widgets, without not unloading the GTK libraries. So programs (processes) are burdened to keeping linked with the family of GTK libraries if they are ever used again. The common use case would be a server startup requiring a GUI and not requiring the GUI after that, as it runs forever like servers do.
One could imagine that the server when signaled pops up it's control GUI.
This is huge for server farms. The difference between being able to run 1000 servers or just 30 servers (just guessing numbers). I'm pretty sure it (DSO GUI plugins) has more use cases than just "common" servers.cheers
lance -
I built and installed the latest qt6 from source (builds very nicely). In looking at the source it looks like QCoreApplication uses static data structures (classes) very heavily. Which could explain the QCoreApplication objects ability to recreate itself without recreating all of itself. At present it is a big "can of worms" to me, and is near impossible to follow the code, at least in a short time. Qt's trend to wrap everything into Qt like cases makes code very hard to follow for a "non-Qt person". I understand that the point is to make Qt portable. Code portability has this complexity as a cost, at least in Qt.
I'm very doubtful that I could convince any of the "core: Qt developers" to consider fixing system resource leaks in QCoreApplication. As best I can tell, it's designed to be a little leaky, they know it and do not care. Their argument will be that it only leaks across object recreation which on average means that it does not leak over the long time scale of many recreate cycles. [Though that's another thing I could check; how it leaks over long times (many many recreates).] I'm still a little worried that it will leak memory, given it leaks files. /proc/PID would be my friend to do that check too.
I've had this "discussion" with the developers of GTK and they are adamant about not making gtk_init() robust, that is making a corresponding destructor for gtk_init() (that's their main loop exec() object constructor). I actually talked in person with one of the developers.
Qt6 is still beating the pants off of GTK in this loading widgets as a DSO module contest. GTK can't even recreate a main-loop thing. It just crashes the program and therefore it is not possible to unload a DSO that has GTK widgets, without not unloading the GTK libraries. So programs (processes) are burdened to keeping linked with the family of GTK libraries if they are ever used again. The common use case would be a server startup requiring a GUI and not requiring the GUI after that, as it runs forever like servers do.
One could imagine that the server when signaled pops up it's control GUI.
This is huge for server farms. The difference between being able to run 1000 servers or just 30 servers (just guessing numbers). I'm pretty sure it (DSO GUI plugins) has more use cases than just "common" servers.cheers
lanceMuch text and effort but still no valid reproducible bug report...
Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
Visit the Qt Academy at https://academy.qt.io/catalog -
Much text and effort but still no valid reproducible bug report...
Okay I'll submit one; but as I pointed out, I can't fix it yet. Thanks for the double encouragement. But oh boy, will it be out-right rejected? I think it will.
I'll make it a separate project; I looked into the test stuff in Qt and got totally lost. So ya, independent test, because, sorry, I'm a dumbass.
I just got it done. It's just one C++ file with comments in it as to how to compile it.I'll post a link to the "bug issue thingy" here soon.
BTW: This can produce more than just one bug. It's at least two, but maybe more. I'll start with just ~QCoreApplication (destructor) leaking files; and only on GNU/Linux because that's all I can test now. The whole thing could be just me being stupid about what /proc/ can do, but I don't think so... You see the Linux kernel is a moving target too, so what I knew about /proc/ ten years ago may no longer be valid today. Not likely. i.e. I'm always learning, and therefore I'm never 100% sure of anything. People that are 100% sure of anything are idiots. I'm 100% sure of that. Now laugh, ha, ha, ha. (The Three Stooges)
https://www.youtube.com/watch?v=4EQBbYoJXCU (same idea, 5 second video) -
Okay I'll submit one; but as I pointed out, I can't fix it yet. Thanks for the double encouragement. But oh boy, will it be out-right rejected? I think it will.
I'll make it a separate project; I looked into the test stuff in Qt and got totally lost. So ya, independent test, because, sorry, I'm a dumbass.
I just got it done. It's just one C++ file with comments in it as to how to compile it.I'll post a link to the "bug issue thingy" here soon.
BTW: This can produce more than just one bug. It's at least two, but maybe more. I'll start with just ~QCoreApplication (destructor) leaking files; and only on GNU/Linux because that's all I can test now. The whole thing could be just me being stupid about what /proc/ can do, but I don't think so... You see the Linux kernel is a moving target too, so what I knew about /proc/ ten years ago may no longer be valid today. Not likely. i.e. I'm always learning, and therefore I'm never 100% sure of anything. People that are 100% sure of anything are idiots. I'm 100% sure of that. Now laugh, ha, ha, ha. (The Three Stooges)
https://www.youtube.com/watch?v=4EQBbYoJXCU (same idea, 5 second video)
