[Solved] How to separate OpenGL display and heavy calculations via threads?
-
Hi everybody!
I've got a class CMain, derived from QObject. It has a slot called mainloop, with, well, my main loop running in it :-). A global event loop is running (QApplication).
Now, in nmy main loop, I'm doing heavy computer vision processing, and want to maintain 100 FPS, which barely works out. I also want to display the results to the user in a nice 3D view, I've got a class CGLPointCloudView which is derived from QGLWidget. I create an instance of this class without a parent. The 3D scene has to be repainted when I update the data or when the user does mouse / keyboard input (to change the viewpoint, for example). It all works out nicely and currently looks like this:
@forever {
// do big calculation
newPoints = BigAlgo.processFrame();// display nice 3D data PointCloudView.updatePoints(newPoints); // do framecounting and other bookkeeping}//end forever
@Now my problem: The OpenGL painting is not optimized at all, and I'm no OpenGL expert. I also don't have the time to tweak the OpenGL code. It currently runs at about 60 FPS, and that's totally fine. Problem is, it slows down my computer vision algorithm! And this is time critical, as I have to send out the data at 100 FPS to another application.
So I would like to do the OpenGL display update "whenever there is time", preferrably in a background thread. How could I achieve this?
I read http://www.develer.com/promo/workshop/enhancing_with_multi_threading.pdf and the documentation on QThread and QRunnable and QtConcurrent. I also found this example http://git.zx2c4.com/qt/tree/tests/auto/qglthreads/tst_qglthreads.cpp?h=master-stable&id=7cd7785deb5d2cccaa712699c1dda1980a5983e5 .
What I'd like to do is have something like "CGLPointCloudView.moveToThread(DisplayThread);". How could I do this? Would it work? How would the code for my "DisplayThread : public QThread" class look?
I also thought about just moving the BigAlgo.processFrame() calculation into another thread (e.g. by simply using QtConcurrent::run()). There are two problems with this approach:
- The main loop would still get blocked during display updates, I'm afraid? So if a display update takes e.g. 12 or more milliseconds, it's really a problem, since my processing is time critical and I only have exactly 10ms to do this and every ms counts. While the calculation would've been finished asynchronously, I still waste time during display updates, right?
- Conceptually, the main "thing" to do is the image processing, and not the display. So the important thing should run in the main thread, for the sake of clarity. The displaying could run "in the background, whenver there is time".
If you could give me any idea or your experience with rendering to OpenGL from a separate thread (and only from this one thread), then I'd be very grateful.
Thanks, Linus
-
Perhaps "this":http://doc.qt.nokia.com/qq/qq06-glimpsing.html#writingmultithreadedglapplications is what you are looking
-
[quote author="cincirin" date="1311059518"]Perhaps "this":http://doc.qt.nokia.com/qq/qq06-glimpsing.html#writingmultithreadedglapplications is what you are looking[/quote]
Thanks, I had already seen that.The thing is, this looks a bit complicated. It involves moving the gl*() rendering commands to the run-method of a QThread. I'd really like to avoid this. I have several helper-functions for rendering (all protected member methods of my subclassed QGLWidget), as well as subclasses of my custom QGLWidgets. Could I leave these helpers in place of the QGLWidget, or would I have to protect them all with mutexes? I'm not to sure in which thread my QGLWidget "lives", if I do the rendering commands, that call this widget's methods, from the rendering-thread (ok, I could move the helper-methods to global functions).
Anyway, the thing is, I've got several complete QGLWidgets. They only use paintGL() to draw. Can't I use
@
QGLWidget::moveToThread(targetThread);
@
? How would that targetThread look like?(In future, of course, I shall design my GLWidgets properly with "disconnected" render-threads in mind).
-
Good news, I found this: http://labs.qt.nokia.com/2010/06/17/youre-doing-it-wrong/
This might be exactly what I looked for. I should be able to do something like this now:@
void CMain::mainLoop() {
QThread DisplayThread;
CGLPointCloudView GLView;GLView.moveToThread(DisplayThread);
DisplayThread.start();forever {
// do big calculation
newPoints = BigAlgo.processFrame();// display nice 3D data GLView.updatePoints(newPoints);}
}
@I'm afraid, I'll have to change the communication from my unprotected method-call to queued connections now, like this
@
forever {
// do big calculation
newPoints = BigAlgo.processFrame();// signal connected to CGLPointCloudView's according slot... emit NewPointsReady(newPoints);}
}
@
and with a "UpdatePoints" slot in GLView. I think I also need to call qRegisterMetaType(), as my "points" are a QVector of a custom class...I'll try to follow the Mandelbrot example ( http://doc.qt.nokia.com/latest/threads-mandelbrot.html ) and make sure I don't step into these pitfalls: http://developer.qt.nokia.com/forums/viewthread/2884 and http://stackoverflow.com/questions/1276967/qt-4-5-how-do-i-perform-a-queued-connection-with-a-template-type
I think QGLWidget::paintGL() automatically calls makeCurrent(), so this souldn't be a problem (or I simply add the call).
I'll keep you posted.
-
Ok, I did a very simple test no. I stripped down the interaction between algorithm and GLView inside the forever-mainloop. So basically all I wanted for now (to test) is having a QGLWidget running in the background, updating its display whenever the user interacts with it (mouse move, for example).
I tried to move it to a new thread:
@
void CMain::mainLoop() {
QThread DisplayThread;
CGLPointCloudView GLView;GLView.moveToThread(&DisplayThread;);
DisplayThread.start();
while (!DisplayThread.isRunning()) {
doEventsAndSleep(1);
}//end whileforever {
// do big calculation
newPoints = BigAlgo.processFrame();// count and display framerate doEventsAndSleep(0);}
}
@
doEventsAndSleep just calls qApp->processEvents(); and Win-API Sleep(int milliseconds).The result is: Nothing. It all works, but when I interact with the GLView, the repainting starts and the frame-rate drops.
Any ideas why this "trick" I wan't to do isn't possible?
If I remove the "doEventsAndSleep()", the GLView isn't painted/updated at all btw. (i.e. looks like a broken window).The way I understood it, the QThread has its own event loop (default implementation of run() just calls exec()). The events of my GLWidget (mouse move etc.) should be processed in that event loop, no?
I also tried to create an additional thread and move CMain into it (as said, it's a QObject with just the mainloop). But again, the result made no difference.
@void CMain::mainLoop() {
QThread MainThread;
this->moveToThread(&MainThread;);
MainThread.start();
// and wait until running etc.
@ -
I finally solved my problem now, or I worked around it.
My original intend to get QGLWidget into a separate thread failed. I still don't get why. I tried to do exactly as in an example from the link I posted before (the whole "you're doing it wrong" thing). See here. http://chaos.troll.no/~bhughes/producerconsumer2.tar.gz
Also this site here explains how to do it right and how it should work: http://blog.exys.org/entries/2010/QThread_affinity.html
Anyway, I played around with my display thread some more. Put it into the real main(), before my CMain object is created, and before the QApplication::exec() call. It always worked, but I could never notice an effect.
I finally went the other way round, and made a QThread for my algorithm. The classic way (overridign run() in a subclass). It all works perfectly now. Still, if anybody finds a way to do it the other way round -- please let me know.
Thanls, Linus
-