Qt Event Loop
-
I'm working with 3rd party software that relies heavily on Qt, and learning as I go. This comment is in the API:
"If the Qt event loop gets backed up then the data for the slot calls gets queued up and memory explodes. So don't do a lot of work directly within signal calls. Quickly make a decision about whether you want the data, and if so, replace your previous reference and move onward. Handle the processing of the data within the regular thread loop."
In this case, the referenced slot is subscribing to a lot of data that comes at a pretty high rate. I am currently doing what they discourage, which is executing a lot of function calls on the data within that slot. Lots of nested loops. I'd rather not look at my Big O performance.
I get the concept, but I'm unclear of how to 'move' anything to the 'regular thread loop'. This is taking place inside a QThread. Should I use a QTimer? Should I turn all my functions into Events?
-
There's no shifting around.
Just to be sure I understand correctly, your plugin gets an awful lot of data from the main application and you are doing a pretty lengthy computation and generate a large amount of data to write to the disk, correct ?
At what speed does the data arrive ?
How long does it take for you to process it ?
How long does it take to write it down to the disk ? -
Hi
it's a bit hard to suggest on as it really depends on the overall design.
It sounds like you have a worker thread that processes incoming data from some other
thread/main thread.
And the rule is not do to too heavy so its busy when new data arrives.If an option, you could make your own worker thread and have all your nested loops there.
Then in the current thread ( worker 1) make it possible for it to send via signals data to your worker thread, so
it can always get the new data and send it for processing to worker 2.But where is data coming from ?
It's not possible to have the source queue it and send new data when worker 1 says its ok ? -
Data source is a larger 3rd party application. My QThread is a plugin that, at a minimum, listens to the main application and publishes a lot of runtime data. The data is published via the slot. Currently, I do all data manipulation within the slot of this publisher function.
I cannot control the way the source queues the data, unfortunately. I have to accept the application as a black box.
Everything is in a single thread right now. The plugin is created, I call run() and then exec() within run.
This comment from the API has me a little confused: "Handle the processing of the data within the regular thread loop."
Do slots have a separate event loop? Based on what I've read in the documentation, I don't think they do.
-
Hi,
No they don't.
I think the suggestion here is to know how much time it takes to process the data and drop some of it would overload the system.
-
No they don't.
Ok, good. I'm starting to catch on to Qt bit by bit. :)
I think the suggestion here is to know how much time it takes to process the data and drop some of it would overload the system.
Yes, clearly I'm filling up a memory buffer for the slot.
So to clarify: since the slot does not have a separate event loop, there is not a 'main' event loop that I need to shift work to within the same thread?
-
There's no shifting around.
Just to be sure I understand correctly, your plugin gets an awful lot of data from the main application and you are doing a pretty lengthy computation and generate a large amount of data to write to the disk, correct ?
At what speed does the data arrive ?
How long does it take for you to process it ?
How long does it take to write it down to the disk ? -
@SGaist said in Qt Event Loop:
Just to be sure I understand correctly, your plugin gets an awful lot of data from the main application and you are doing a pretty lengthy computation and generate a large amount of data to write to the disk, correct ?
Yes, but the write is separate.
At what speed does the data arrive ?
60 Hz.
How long does it take for you to process it ?
I haven't clocked it, but since 60Hz = 0.0167 seconds, and I crash after a period of time, I'll assume greater than 0.0167 seconds.
How long does it take to write it down to the disk ?
Also haven't clocked it, but the logging is handled in a separate thread (feeds to a catch all data logger).
There's no shifting around.
So it sounds like I need worker threads. Something like DataArrives -> Sort it to the right function -> Function executes in a separate thread.
-
Well, I would recommend timing all these stuff because at some point if you do not get faster you'll have bottlenecks anyway. You will likely need a queue from which you'll drop data it reaches a certain threshold.
