QThread::requestInterruption() cannot be undone
-
I do use Worker objects in my threads. Each therad have his own worker object according to the job it need to do.
I read the doc about QThreadPool, but it seems not convenient since every thread have to communicate betwwen each other and send result to the main thread.
QRunnable are doing some job in there side...
-
Well, may be I should explain my issue more concretely:
I have 3 worker objects that are meant to run in separate threads.
The first thread A have to read a file line by line and make few checks. It then pushes the QString to a queued shared buffer with thread B.
Thread B tkae those results and parse the QString into a QStringList. It pushes the QStringList in a queued shared buffer with thread C.
Thread C binds the QStringList parameters in a prepared SQL statement and execute it.An object in my main thread manages the worker threads synchronisation. It setup the worker objects, create QThread instance, move the objects to the correct thread and launch those threads. When the user click on a button, It calls a slot in the object Manager than manage to launch the differents methods in the worker threads using SIGNAL with parameters (since it is thread-safe).
If an error occured in any of the worker threads, the concern thread send an error SIGNAL to the object Manager and stop his job (return in a method). Then the object Manager executes requestInterruption() to other worker thread so they stop their job too. At this stage, the user can correct the input data and launch again the whole work.
So he clicks again on the button. The object Manager send SIGNAL to all thread so they start working again with new data. The problem now is that 2 of the 3 threads will have the isInterruptionRequested() method returning TRUE, so they will stop has they believe an error occured in an other thread. And then, I end up to a blocked situation where 2 threads ended their job, and the last one is waiting because the queued shared buffer are empty and it doesn't receive interruptionRequest() - because no error SINGAL have been sent to the object Manager since there was no errors.
So maybe my design is wrong. It's quite the first time I use threads. But having a method that can reset the requested interruption would have help me a lot!
-
I see.
What you've described is pausing your workers, rather than pausing the thread or the event loop.
[quote]the concern thread send an error SIGNAL to the object Manager and stop his job (return in a method).[/quote]How about doing the same thing with your worker objects? The manager can send an error signal to the workers and ask them to stop.
When the user clicks the button, let your manager send a signal to all the workers to ask them to continue.
-
Ok, but how to do that? While my worker object is running his job, the SIGNAL received will be put in queue, but not treated yet.
Does that means I'll have to hook into the event loop (I known there is a such method that can treat all in-queued events)
-
It depends a bit on your architecture. Can you share some code which shows how your jobs are carried out?
-
Well not realy as it is for my job.
The thing is I use shared buffer between thread, so each thread uses a method to pull and push an element from the shared buffer (thread safe of course). If the buffer is empty (or full depending on which side of the buffer you are), the method return false. If so, I check if the job have to stop there by checking interruptionRequested() method, if not, I put the thread in sleep for 1 ms and then try to read (/ write) again from (/ in) the buffer. etc.
Otherwise, if the buffer is never empty nor full (which means the thread are working well in quite the same speed) the interruptionRequested() is never called.
A kind of equivalent code:
@while (buffer->push(element))
{
if (interruptionRequested()) return;
msleep(1);
}@Maybe I should use a kind of shared ressource between all threads that tell them to stop.
-
Ok, it sounds like you are mixing an event loop with an infinite-while-loop. These two kinds of loops don't usually work well together -- it's best to choose one or the other.
If you want to use long while loops:
- Subclass QThread. Don't start event loops.
- Communicate using shared buffers and flags only.
- Use "low-level primitives":http://qt-project.org/doc/qt-5/threads-synchronizing.html#low-level-synchronization-primitives to synchronize threads and access shared buffers/flags.
If you want to use event loops:
- Don't subclass QThread.
- Communicate using signals/slots only.
- Don't use shared buffers. Use signals and slots to transfer your QStrings/QStringLists between threads.
** Example: When Thread A reads a line, it emits a signal to send the QString to Thread B. When Thread B receives this signal, it parses the string and emits a signal to pass the QStringList to Thread C. When Thread C receives the signal, it runs the SQL query.
-
Ok, I see your point! Indeed, I mix the 2 features because for example, the database thread need some preparation before entreing he loop...
But yes, I see better how to design the thing. I used shared buffer because I thought it would be faster than using signal to pass data. But if you said you can use them for that, I'll work on that!
Thanks for you help.
Just a question, if you are using infinite loop way, how can you send regularly to the main thread the progression of the task or when the job actually finish? Because in fact, I use SIGNAL from worker threads mainly in that purpose: send the progression percentage and tell when the task is finished.
-
You're welcome :)
[quote author="Vinorcola" date="1406476906"]I used shared buffer because I thought it would be faster than using signal to pass data. But if you said you can use them for that, I'll work on that![/quote]The only reliable way to find out if something is fast enough is to profile it.
However, I have done live audio processing before using signals and slots -- read bytes from file, then process bytes, then play the audio -- on a small BeagleBoard C4. Signals and slots are fast enough for this, so I'm sure it's fast enough for database queries on a PC.
[quote author="Vinorcola" date="1406534871"]Ok, just thought longer about that, and the thing is that my worker function can send SIGNALs without having an event loop running, doesn't it? The important thing is that the receiver has an event loop.
Is that right?[/quote]100% correct!
-
Ok, just another thing. Currently, my thread that read and parse are faster than the one querying database.
By using SIGNALs, I'm a bit afraid the queue of event in my database thread grows very hight.
By using the shared buffer, I can put the threads in pause for a while when the buffer in full (as I show you before).
So as I can read some files that can contains few million lines, I'm just afraid that if the database thread is slower, I'll queued many and many SIGNAL with QStringList inside, so the memory usage can grow up. Is there kind of safety in Qt that can avoid this situation?
Maybe using shared buffer would be more appropriate than using SIGNAL.
-
There are different possible solutions; both shared buffers and signals are valid solutions. I don't know the details of your app though, so I can't tell you which approach is better.
One alternative is to let your SQL thread "pull" data, instead of letting your read thread "push" data -- emit a signal when it wants the read thread to read another line.
By the way, you might consider combining your read thread and parse thread into one, to simplify your code. Is there any benefit in keeping them separate?
These examples might be useful too:
-
Well, thank you JKSH for your help!
Yes I think I can combine file reading and parsing in the same thread. Initially, I wanted the parsing thread to bind the parsed string directly into the prepared statement so that the database thread only have to execute the query. But it was impossible, or very very complicated.
