Is it safe to have QThreads launching QProcesses

mbruel

Hello,
I'm going to develop an application that does several calculations (that can take few minutes or more). As it is possible to parallelize some of them I was thinking to have a Pool of working Threads (their number being dynamic) that would consume the CalculRequest from a shared queue.
My problem is that some of the calculation will use an external program and thus would need to use a QProcess.
I've been told that I'll have problems if I was mixing processes and threads and that I should have one dedicated process that would deal with all the subprocesses.
Is it the case with QProcess or is the class handling totally the process and thus could be launched even from a thread?
I know a QProcess wouldn't really need a thread but I then need to load the result of the process (from a file) in that thread. Does this make sense?

SGaist

Hi,

From an architectural point of view, does your process need to be started separately for each calculation or can you feed it with them ?

mbruel

Yes it needs to be started each time. You call it with the path of a file where all its required parameters have been serialized in the format it understands and the name of the output file you'll then parse to get the result.

People were telling me to not mix threads and processes that it would be a mess to maintain and could explode...
Is it really the case when using QT wrappers or does it hide the complexity (checking process state, know when/how it is finished, prevent zombie process...)?
That is why those persons where advising me to create one process that would manage all the calculation processes and keep the main process of the app for the HMI and other type of threads (Post treatments)
What do you think about that? It kind of brings more complexity than my initial design...

SGaist

Prevent zombie process ? No, that's not Qt's role at all.

QProcess does signal when the application started has excited.

AFAIK, you can move QProcess objects to another thread.

You might be interested by this KDAB article about the new QThread::create method in Qt 5.10.

Depending on what you want to do with the output of the application you are calling through QProcess, you might want to consider a multi-stage pipeline:

Creating the requests
Handling these request through QProcess (no need of QThread)
Handle the newly created results with for example QFileSystemWatcher.

mbruel

Thanks for your reply,
As I said, sometimes I'll need to use an external program to do the calculation, sometimes not, the code will be embedded and thus run in a Thread.
So I would like to make the nature of the task transparent to the user and even to the Simulation (object in my design) that launches the calculations (by filling a shared queue).
I don't find much documentation on the web about the behaviour of a multi-Threads AND multi-Processes application.
I've been told that it should be a NO go... (from C/ C++ Unix developers)
I'd like to make sure that I could do such thing using QT: mixing QThreads and QProcesses in a same application or if there could be some risks.
What do you think?
Any links about mixing both?

PS: @SGaist :

I'll have a look at this new way to create threads with QT 5.10, normally I should be able to use that version.
Idem for QFileSystemWatcher but I'm not sure to like the idea to dedicate a watcher when I can be signaled at the end of the process and thus know the exact moment I can do the reading. I could first run the QProcess then run a QThread to do read the result but I'd like to have the execution of both in one single task... Do you see my problem?

stevemcgf

If you will wait on that thread until the process finish the operation I don't see any problem. Running external process multiple times very quickly is highly inefficient but you stated that each run will take at least one minute so the bottleneck wouldn't be on launching process.