QOpenGLWidget and Threads

[Kent-Dorfman]

@rtavakko Unfortunately my opengl experience is old school immediate mode programming. I have yet to do anything real using the modern shaper/pipeline architecture. I think it's a given that the framework properly handles multiple QOpenGLWidgets, but as you've discovered, when you try to offload to worker threads there are some gotchas: the big one being that you now have to explicitly give the correct worker thread the context (and it probably cannot be simultaneously shared). I suspect that the modern pipeline architecture cannot handle multiple threads using the pipeline concurrently.

[wesblake]

I've got a similar question. I am trying to get our current (threaded) QGLWidget migrated to a QOpenGLWidget. So far based on an example it's successfully working meaning it runs, the glClear is working from the other thread, as is the "WTF" message so I know it's rendering at X fps.
What I can't figure out is how to get a QPainter to work!! I've tried everything. Our current QGLWidget based class renders QIMages at about 60 fps threaded, np using QPainter. I've read many things that seem to indicate I can still use QPainter in a separate thread as we are doing with QGLWidget, but I just can't get it to work. No matter how/where I create QPainter, it crashes. (cpp line 197)
Am I forced to use opengl directly? How would I draw a QImage like that? Thanks!

#ifndef GLWIDGET_H
#define GLWIDGET_H

#include "framequeue.h"

#include <QOpenGLWidget>
#include <QOpenGLContext>
#include <QImage>
#include <QThread>
#include <QMutex>
#include <QList>
#include <QWaitCondition>

class GLWidget;

class Renderer : public QObject
{
    Q_OBJECT

public:
    Renderer(GLWidget *);
    ~Renderer();
    void lockRenderer() { m_renderMutex.lock(); }
    void unlockRenderer() { m_renderMutex.unlock(); }
    QMutex *grabMutex() { return &m_grabMutex; }
    QWaitCondition *grabCond() { return &m_grabCond; }
    void prepareExit() { m_exiting = true; m_grabCond.wakeAll(); }

    FrameQueue *selfView;
    QList<FrameQueue*> remoteViewList;

signals:
    void contextWanted();

public slots:
    void start();
    void stop();
    void pause();
    void resume();
    void resumeToClose();
    void render();

private:
    static const int animationFPS = 30;
    static const int pipWidth = 160;
    static const int pipHeight = 120;

    bool m_inited;
    GLWidget *m_glwidget;
    QMutex m_renderMutex;
    QMutex m_grabMutex;
    QWaitCondition m_grabCond;
    bool m_exiting;

    bool pauseRendering;
};

class GLWidget : public QOpenGLWidget
{
    Q_OBJECT
public:
    GLWidget(QWidget *parent);
    ~GLWidget();
    void stopRendering();
    void startRendering();
    void pauseRendering();
    void resumeRendering();
    void resumeForClose();
    bool isStarted();

protected:
    void paintEvent(QPaintEvent *) Q_DECL_OVERRIDE { }

signals:
    void renderRequested();

public slots:
    void grabContext();

private slots:
    void onAboutToCompose();
    void onFrameSwapped();
    void onAboutToResize();
    void onResized();

private:
    QThread *m_thread;
    Renderer *m_renderer;
    bool m_isStarted;
};

#endif // GLWIDGET_H

#include "glwidget.h"

#include <QGuiApplication>
#include <QPainter>
#include <QDebug>

//Leak detection
#ifdef Q_OS_WIN32
#ifdef QT_DEBUG
#include "debugtools/reportinghook.h"
#include "debugtools/setdebugnew.h"
#define new DEBUG_NEW
#endif
#endif

GLWidget::GLWidget(QWidget *parent)
    : QOpenGLWidget(parent)
    , m_isStarted(false)
{
    connect(this, &QOpenGLWidget::aboutToCompose, this, &GLWidget::onAboutToCompose);
    connect(this, &QOpenGLWidget::frameSwapped, this, &GLWidget::onFrameSwapped);
    connect(this, &QOpenGLWidget::aboutToResize, this, &GLWidget::onAboutToResize);
    connect(this, &QOpenGLWidget::resized, this, &GLWidget::onResized);

    m_thread = new QThread;
    m_renderer = new Renderer(this);
    m_renderer->moveToThread(m_thread);
    connect(m_thread, &QThread::finished, m_renderer, &QObject::deleteLater);

    connect(this, &GLWidget::renderRequested, m_renderer, &Renderer::render);
    connect(m_renderer, &Renderer::contextWanted, this, &GLWidget::grabContext);
}

GLWidget::~GLWidget()
{
//    stopRendering();
}

void GLWidget::stopRendering()
{
    m_renderer->prepareExit();
    m_thread->quit();
    m_thread->wait();
    delete m_thread;
}

void GLWidget::startRendering()
{
    m_thread->start();
    m_isStarted = true;
    QMetaObject::invokeMethod(m_renderer, "start");
}

void GLWidget::pauseRendering()
{
    QMetaObject::invokeMethod(m_renderer, "pause");
}

void GLWidget::resumeRendering()
{
    QMetaObject::invokeMethod(m_renderer, "resume");
}

void GLWidget::resumeForClose()
{
    QMetaObject::invokeMethod(m_renderer, "resumeToClose");
}

bool GLWidget::isStarted()
{
    return m_isStarted;
}

void GLWidget::onAboutToCompose()
{
    //We are on the gui thread here. Composition is about to begin, wait until the render thread finishes.
    m_renderer->lockRenderer();
}

void GLWidget::onFrameSwapped()
{
    m_renderer->unlockRenderer();
    //Assuming a blocking swap, our animation is driven purely by the vsync in this example.
    emit renderRequested();
}

void GLWidget::onAboutToResize()
{
    m_renderer->lockRenderer();
}

void GLWidget::onResized()
{
    m_renderer->unlockRenderer();
}

void GLWidget::grabContext()
{
    m_renderer->lockRenderer();
    QMutexLocker lock(m_renderer->grabMutex());
    context()->moveToThread(m_thread);
    m_renderer->grabCond()->wakeAll();
    m_renderer->unlockRenderer();
}

Renderer::Renderer(GLWidget *w)
    : m_inited(false),
      m_glwidget(w),
      m_exiting(false),
      pauseRendering(false)
{
    selfView = new FrameQueue("selfView");
    long lineCount = 4;
    int iRet = HAPIGetPropertyInt(catPhone, propPhoneCallsLimit, &lineCount);
    QString remoteName;
    for(int i=0; i<lineCount; i++){
        remoteViewList.append(new FrameQueue(remoteName.sprintf("remoteView%i",i)));
    }
}

Renderer::~Renderer()
{
    if(selfView)
        delete selfView;
    for(int i=0; i<remoteViewList.size(); i++){
        FrameQueue *curQueue = remoteViewList.at(i);
        if(curQueue)
            delete curQueue;
    }
}

void Renderer::start()
{
    HAPIRegisterSelfView(selfView);
}

void Renderer::stop()
{
    HAPIUnregisterSelfView(selfView);
}

void Renderer::pause()
{
    pauseRendering = true;
    HAPIUnregisterSelfView(selfView);
}

void Renderer::resume()
{
    pauseRendering = false;
    HAPIRegisterSelfView(selfView);
}

void Renderer::resumeToClose()
{
    pauseRendering = false;
}

void Renderer::render()
{
    if(m_exiting)
        return;

    QOpenGLContext *ctx = m_glwidget->context();
    if(!ctx)//QOpenGLWidget not yet initialized
        return;

    //Grab the context.
    m_grabMutex.lock();
    emit contextWanted();
    m_grabCond.wait(&m_grabMutex);
    QMutexLocker lock(&m_renderMutex);
    m_grabMutex.unlock();

    if(m_exiting)
        return;

    Q_ASSERT(ctx->thread() == QThread::currentThread());

    //Make the context (and an offscreen surface) current for this thread.
    //The QOpenGLWidget's fbo is bound in the context.
    m_glwidget->makeCurrent();

    if(!m_inited) {
        m_inited = true;
//        initializeOpenGLFunctions();
    }

    glClearColor(0.27f, 0.26f, 0.26f, 1.0f);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    if(pauseRendering){
        return;
    } else {
        //TODO HERE
        //How do I get QPainter here?! The context is moved yet I'm getting errors!
//            QPainter p(m_glwidget);

        qDebug() << "WTF";
    }

    //Make no context current on this thread and move the QOpenGLWidget's context back to the gui thread.
    m_glwidget->doneCurrent();
    ctx->moveToThread(qGuiApp->thread());

    //Schedule composition. Note that this will use QueuedConnection, meaning that update() will be invoked on the gui thread.
    QMetaObject::invokeMethod(m_glwidget, "update");
}

[wesblake]

Bump. I've done everything and can't figure this out. I'm about to try a non-threaded QOpenGLWidget because it's my only choice at this point but I'm afraid it will impact performance too much.

[rtavakko]

@wesblake The topics are not directly related so it would've probably helped to start a new one.

What error are you getting?

I'm coming more and more to the conclusion that a single thread is the way to go most of the time.

[wesblake]

@rtavakko said in QOpenGLWidget and Threads:

@wesblake The topics are not directly related so it would've probably helped to start a new one.

What error are you getting?

I'm coming more and more to the conclusion that a single thread is the way to go most of the time.

Sorry, seemed similar but I can move it. Here is the error:

[rtavakko]

@wesblake said in QOpenGLWidget and Threads:

QPainter p(m_glwidget);

Try replacing:

QPainter p(m_glwidget);

with:

QPainter p(this);

[wesblake]

Thanks. I found a known bug in QOpenGLWidget now preventing me from using it. :( So, starting over. New trial, a regular QWidget, worker thread to resize and composite frames first.

[rtavakko]

@wesblake Hmm. Which bug is this?

[wesblake]

https://bugreports.qt.io/browse/QTBUG-49657
Duplicated ticket says fixed but it's not. It occurs exactly as described.

[Christian Ehrlicher]

I'm pretty sure this is not your bug as long as you don't use qt5.6 and fullscreen mode. Please provide a compilable, minimal example to show your problem.