Per Qobject Message Loop for Signal/Slot mechanism
-
From what I understand, The signal/slot message system is implemented per thread. Hence, every QObject needs to "belong" to a certain thread in order to operate correctly in a multithreaded environment.
This is great for scenarios in which every object is processed by one thread only. But, what happens when we have a thread pool processing various QObjects? The signal/slot mechanism will no longer work as the QObjects will be constantly migrating across threads.
What I propose, instead, is to move the message dispatch system to the QObject. This way, we can handle pending messages using QObject::ProcessMessages or similar.
What are your thoughts on this?
-Debdatta
-
The signal handling is similar to an event dispatching. The event is dispatched in the thread of the target object. If the signal is emitted in the same thread and the connection is direct, the same thread will handle the slot. If the connection is queued than the slot will be handled by the target thread (e.g., the application gui thread). The default connection accepts and implements both behavior. So I guess that queued connections is what you are looking for when dealing with multi-threading sources of events.
-
Processing QObjects from various threads AND using signal/slot with those is error prone. First of all, do you ensure, that only one thread at a time access this object? That you get no -raise- race conditions? QObject itself and it's methods are not thread safe.
-
It simply isn't possible, I think. Who or what would drive those eventloops exactly? Who makes sure that all eventloops of all objects in a thread get their turn to process? That would need a per-thread mechanism again, right? Or do you propose to run every QObject in its own thread, with all the horror that will result in? So instead of simplifying, you just complicate the process. Also, QObjects would become even heaver than they are now. That is not the direction to move into, IMHO.
-
You were the closest to understand what I was saying. :D Let me explain better.
Who or what would drive those eventloops exactly?
There would not be an event "LOOP" in the traditional sense. The programmer can elect to process pending events whenever he wants.That would need a per-thread mechanism again, right?
The current per-thread mechanism can be implemented as a layer above this. So, instead of the message passing code being in the QThread class, It would be in the QObject class, and the QThread class would simply call the event processing functions in its own loop.Or do you propose to run every QObject in its own thread, with all the horror that will result in?
Not AT ALL! :D I don't know how you reached this conclusion. :D
Not all asynchronous processing takes place using threads directly. In a lot of cases, We implement a task based system, where lots of tasks(Lets say QTask. :D) are scheduled in the same pool of threads(Mostly equal to the number of cores available). The (Hypothetical) QTask's Run() function is called when it is scheduled on a thread.
The current asynchronous signals architecture is centered around Threads, and will fail if the objects are processed in threads other than the one in which they were created. :D
The Proposed architecture will move the asynchronous message passing code to QObject. This will take care of the problem. :) In use cases where everything is in the same thread, the direct connection would not be affected. Only the Asynchronous connnection will.
@Gerolf.
Not Really.
http://doc.qt.nokia.com/stable/threads-qobject.html#signals-and-slots-across-threads -
The main problem is that you cannot pull a QObject into the current thread, but only push from the thread the object still belongs to. So you might work around this by sending a signal with the new worker thread as argument to your QTask which asks for moving it to the worker thread. Once that's done the QTask can tell the thread to continue the work. This is safe, just make sure to use queued connections.
I don't know if it's a good idea to make pulling an object to the current thread away from another thread is a good idea.
I strongly advise against messaging implemented in QObject. With thousands of those created in a real world application (all widgets, hidden helper objects, IO, networking...) that would blast size and complexity for an IMHO quite corner usage.
-
If I have understood your problem correctly I would propose the following solution: Subclass QThreadPool and add a startAndMove() method, which pushes the QRunnable to the selected thread before starting it (and possibly pushing it back as soon as it has finished if needed and not autoDeleted) and have your own event loop running there.
QThreadPool has some nasty limitations which render this class (partially) useless. Another one is that you cannot remove a QRunnable from the run queue as soon as it is passed to start() (and not scheduled yet). Thus you cannot flush pending jobs in the run queue (for example when your application shuts down). This limitation hits QtConcurrent::run() as well.
I think QThreadPool (and QtConcurrent) could need some love (and C++11 support) for Qt 5, at least with these changes proposed here included. Further bloating QObject is not the way to go. Quite contrary to I would actually love to see some cleanup and modularization to QObject which allows me to pick the support I need (as we already have with Q_GADGET for example).
-
I really don't see how making the application programmer manually drive an event loop (yes, you really do need such a thing if you want your application to stay responsive to user and system events) is going to simplify application development. You would end up with code like this:
@
forever {
ui->button1->processEvents();
ui->button2->processEvents();
ui->lineEdit->processEvents();
//... a few hundred more lines like this
myTcpSocket->processEvents();
}
@No, thanks, I think I prefer what we have.
I think that there are much simpler ways to solve your problem.
-
@everyone in support of moving Qtasks to the current thread. That has to be mighty inefficient as a lot of bookkeeping will be done over and over.
@Andre
You wont need to manually drive Anything. That will be an OPTION. The in-built message queue will call_forever{
ui->every QObject->ProcessEvents();
}_
for you. You don't need to implement it as long as you are doing standard stuff. In short, for most people, Nothing changes.
And if at all, This will INCREASE performance in multi-threaded situations, not decrease it. That is because LESS LOCKING will need to be done. It wont affect the single threaded case at all performance wise. :)
-
[quote author="Debdatta" date="1322652752"]@everyone in support of moving Qtasks to the current thread. That has to be mighty inefficient as a lot of bookkeeping will be done over and over.
[/quote]What kind of bookkeeping are we talking about here? QObject::moveToThread() overhead for a QObject with no pending events and without complex child structure (which should be true for all QRunnable) is marginal and virtually non-existent compared to context switch cost in multithreaded applications.
[quote author="Debdatta" date="1322652752"]
@Andre
And if at all, This will INCREASE performance in multi-threaded situations, not decrease it. That is because LESS LOCKING will need to be done. It wont affect the single threaded case at all performance wise. :)
[/quote]The primary role of the event loop (although it is used for signal/slots as well) is message dispatching from the operating system / windowing system - which has no such concept of QObject and QWidget. This means that every QObject would have to process every message, filtering out the ones not beeing responsible. I heavily doubt this won't have a significant performance impact in both, single-threaded and multi-threaded, applications.
-
[quote author="Debdatta" date="1322652752"]@Andre
You wont need to manually drive Anything. That will be an OPTION. The in-built message queue will call
@
_forever{ui->every QObject->ProcessEvents();
}_
@
for you. You don't need to implement it as long as you are doing standard stuff. In short, for most people, Nothing changes.
[/quote]This would result in reordering of events.
If you click one button and then the next, you expect the ui to behave like this, not in any other order. -
Let me put it like this: I am all but convincened. However, nothing is stopping you from creating a (demo) implementation, and proposing it via development@qt-project.org or via Gerrit.