Run parts of an Application in separate process



  • I'm facing this issue of how to run different parts of an Application in separate process as the member function for QProcess that sets a process name requires a full path.

    void setProgram(const QString &program)
    

    But the process i'm trying to run is basically a module of the Main Application. So let me explain what i'm trying to achieve basically.

    The application has QMainWindow as the central window which will controls all the different modules, monitor them, stop/restart them, communicate with them via IPC mechanism.

    • So for example one of the modules has a QWebEngineView that will direct the user to a certain website and manage session information( cookies, authentications, etc) for that user, and will let the user use the website like he would normally via a browser.

    • Another module will just continuous perform CPU intensive computations, without ever blocking any part of the application.

    • Another module will render a full fledged Javascript charting app.

    And so on. There are a number of such heavy weight modules in this application so using a threading approach is not desirable as basically these modules needs to run in their separate memory space and will act as child processes of the MainWindow process. The communication between the processes will be nicely handled using D-Bus, SharedMemory.

    But what i can't figure out is how to actually create these child processes when the user clicks on the specific actions/buttons in the MainWindow. Again these modules are very much intergrated in one app and can't be called externally due to security and integrity constraints. So please let me know of any way to achieve this.


  • Moderators

    Well, if you only have one executable, then you will need to use parameters to tell the executable which module to use. Something like:

    path_to_your_executable -w
    

    Lets say -w is for the web engine part.
    In your application you then check the parameters and decide which module to use.

    setProgram() does not require an absolute path. But if you use a relative path or just executable name then you should be in the right directory or the directory containing your exe must be in PATH.


  • Moderators

    To get the path to your executable you can use http://doc.qt.io/qt-5/qcoreapplication.html#applicationFilePath



  • @jsulm Yes that is the regular or the standard way of forking a process using a executable path. And this is already quite straightforward and done you could say. But i was looking for a more integrated solution meaning a way to fork a process using function name ( the modules ) rather than a separate executable. Just like a thread is spawned by providing it a callable function name. I'm not sure that this is even possible but i wonder. Thanks for your input , really appreciate it.


  • Moderators

    Actually you could use fork(), yes. And execute another code path in the child process. But I never used fork() in a Qt GUI application, so I don't know whether you could have any issues. But you can try.



  • @jsulm So far your advice is used which is to call the modules separately upon button click using the module specific command line arguments. And its working perfectly.

    But right now the main challenge is to run this module in a separate tab. The MainWindow has a tabbed interface with the 1st tab being the actual main tab while the modules will be spawned in their own tabs separately. So when the user will click the appropriate button a separate tab will open up and in that the Webengine process will be called which will be the child process of the Mainwindow. Can this be done in a straightforward manner ? Or is there any other way to show multiple processes as s separate tabs from within the MainWindow.


  • Qt Champions 2016

    @jsulm

    Actually you could use fork(), yes. And execute another code path in the child process. But I never used fork() in a Qt GUI application, so I don't know whether you could have any issues. But you can try.

    QProcess::startDetached already double-forks the process, so it's not necessary to go to the linux API directly. For non-detached processes QProcess::start is possible.

    @Maxx-Dovahkiin

    But i was looking for a more integrated solution meaning a way to fork a process using function name ( the modules ) rather than a separate executable.

    Then you are looking for threads, not processes. As processes do not share their address space, opposed to threads, they can not know what's in other processes' memory (beside the obvious memory being exposed manually). If you need functions or objects from another process, you need to make that process a thread (which is quite similar to how threads are implemented on linux).

    And so on. There are a number of such heavy weight modules in this application so using a threading approach is not desirable as basically these modules needs to run in their separate memory space and will act as child processes of the MainWindow process.

    Why? On the one hand you want them to run separately, but then, on the other hand, you want them (in your following post) to share their memory addressing. Both are not possible at the same time. You either can run them as separate entities, pass the arguments through the command line and control/notify through IPC (D-Bus), or you create them in the same address space (i.e. as threads) and communicate/notify through standard means - signals/slots, mutexes/semaphores etc.

    But right now the main challenge is to run this module in a separate tab.

    You can't! As the said process doesn't share the address space of the parent process you can't "run it in a tab" directly. You have to implement the whole infrastructure by yourself. That includes a QWidget that'd be a proxy for a running process and the D-Bus communication on both sides.

    Kind regards.



  • @kshegunov Yes using threads was the most obvious choice at first, but later when assessing the requirements i found that these each of these modules were heavy-weight and actually are separate subprograms that fulfills a particular need. And each of these sub modules will have separate threads of its own in order to parallelize computations. So weighing in all this factor i decided to go with QProcess. These processes won't share their memory address space at all. But only share the required data to and from each other via QSharedMemory.

    And hence i was looking for ways to "embed" this subprograms which will run as separate processes in separate tabs all integrated in the QMainWindow. I understand your point that as these processes will be actually separate programs each with their own address space so its not possible to directly integrate them in a separate tab. But the thing is i am willing to actually take the leap of faith and actually code the infrastructure that you mentioned with proxing QWidget for the running process and establishing a IPC mechanism on both sides. Can you post some examples maybe or any reference that would hint on these techniques more elaborately.

    Anyways thanks for your answer, i really appreciate it.


  • Qt Champions 2016

    @Maxx-Dovahkiin said:

    Yes using threads was the most obvious choice at first, but later when assessing the requirements i found that these each of these modules were heavy-weight and actually are separate subprograms that fulfills a particular need.

    Fair enough.

    And each of these sub modules will have separate threads of its own in order to parallelize computations.

    As a bit of an offtopic - note that threads on Linux are already pretty heavy (as opposed to Windows). Also, consider using as small number of threads as possible, as otherwise each process will be "fighting" with the others for time allocation by the OS's scheduler. The best choice would be to have as many threads (altogether in all of your processes) minus 1 (to allow for responsive GUI), as there are cores. So if you go out of your way and create many threads you can end up actually slowing down your processing or even "hijacking" the time allocated for X11 (or Qt's GUI).

    But the thing is i am willing to actually take the leap of faith and actually code the infrastructure that you mentioned with proxing QWidget for the running process and establishing a IPC mechanism on both sides.

    This would involve (after getting D-Bus going) having a custom widget that is subscribed to the D-Bus events (or signals) and displaying whatever it is to display. If the data the different processes are generating is heterogeneous, then you might end up coding a separate widget for each of them. Also you should think about synchronizing the processes. That's possible with D-Bus as well as by subscribing to the QProcess::finished() if you started the child process with QProcess::start. But that's not quite enough, you may actually need to wait for the child process (if for example the parent process is required to exit), so you can have a nice clean shutdown.

    Can you post some examples maybe or any reference that would hint on these techniques more elaborately.

    I don't have anything for a GUI app, but there's nothing specific about it either. The only difference between GUI and non-GUI app is that you have to expose more through the D-Bus and provide a way to visualize the running processes. I have recently coded a daemon for Linux (rather a library for daemons), so you can look at it for some inspiration. As for the GUI the D-Bus documentation features some (mostly) good examples as well. You could look at the remote-controlled car example for more details.

    Kind regards.



  • @kshegunov

    Also, consider using as small number of threads as possible, as otherwise each process will be "fighting" with the others for time allocation by the OS's scheduler.

    Yes the threads will be managed by QConcurrent with a hard limit on the maximum number of threads possible. The threads will be there mainly to parallelize many computations which are inherently SIMD parallel instructions.

    This would involve (after getting D-Bus going) having a custom widget that is subscribed to the D-Bus events (or signals) and displaying whatever it is to display

    I understand this. So basically i need to establish a D-Bus layer from the QMainWindow central app that will declare the signals/slots available for each sub programs and communicate accordingly. Anyways i think i have enough of an idea of what to do thanks to your wonderful help. Will take a look at qtdaemon for ideas about QProcess and IPC in general. Thanks again for your help.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.