Proper way to pass QImage from thread through signal?

wesblake

Hi all. So I've looked at every post and example I can find and I can't figure this out. First of all, I've simplified the code below but the thread is needed for the heavy computations I've stripped out just for now. I get up to 3 "frames" of video at once (local camera and 2 remote), which then need to be resized and composited before sent back to the widget to be displayed. (Thus the QPainter on QImage in the thread)
For now, I'm using a local resource image to get the basics going. I commented in the code, as you can see, I know the thread is running as expected, I know I'm getting to the paintEvent and can draw as expected (directly using the image there), however when I try to pass the image from the thread, I get nothing. Just a blank widget. My conclusion is I'm passing the QImage wrong. Ideally a copy would pass since I want to avoid tearing or thread issues (the thread trying to work on the image while I'm trying to display it).
So far, this is all working fast, no performance issues, but I don't know since I'm not actually getting anything passed if that will be my bottleneck!
Any help is appreciated, thanks!

vwidget.h:

#ifndef VWIDGET_H
#define VWIDGET_H

#include <QWidget>
#include <QImage>
#include <QThread>

class VWidget;

class Renderer : public QObject
{
    Q_OBJECT

public:
    Renderer(VWidget *);
    ~Renderer();

signals:
    void finished();
    void frameReady(QImage img);

public slots:
    void stop();
    void render();

private:
    VWidget *m_vwidget;
    bool m_exiting;

    QImage privacyImage;
};

class VWidget : public QWidget
{
    Q_OBJECT
public:
    VWidget(QWidget *parent);
    ~VWidget();
    void stopRendering();
    void startRendering();
    void pauseRendering();
    void resumeRendering();
    bool isStarted();
    void scaleUp();
    void scaleDown();

protected:
    void paintEvent(QPaintEvent *event) override;

signals:
    void renderRequested();

public slots:
    void requestRender();
    void displayFrame(QImage img);

private:
    QThread *m_thread;
    Renderer *m_renderer;
    bool m_isStarted;
    QImage dFrame;
    QTimer *timer;
};

#endif // VWIDGET_H

vwidget.cpp:

#include "vwidget.h"

#include <QPainter>
#include <QTimer>
#include <QDebug>

VWidget::VWidget(QWidget *parent)
    : QWidget(parent)
    , m_isStarted(false)
{
    m_thread = new QThread;
    m_renderer = new Renderer(this);
    m_renderer->moveToThread(m_thread);
    connect(m_renderer, SIGNAL(finished()), m_thread, SLOT(quit()));
    connect(m_renderer, SIGNAL(finished()), m_renderer, SLOT(deleteLater()));
    connect(m_thread, SIGNAL(finished()), m_thread, SLOT(deleteLater()));

    timer = new QTimer(this);
    connect(timer, &QTimer::timeout, this, &VWidget::requestRender);

    connect(this, &VWidget::renderRequested, m_renderer, &Renderer::render);
    connect(m_renderer, SIGNAL(frameReady(QImage)), this, SLOT(displayFrame(QImage)));
}

VWidget::~VWidget()
{
    stopRendering();
}

void VWidget::stopRendering()
{
    if(timer->isActive())
        timer->stop();
    m_renderer->stop();
}

void VWidget::startRendering()
{
    m_thread->start();
    m_isStarted = true;
    timer->start(20);//50fps
}

void VWidget::pauseRendering()
{
    timer->stop();
}

void VWidget::resumeRendering()
{
    timer->start(20);//50fps
}

bool VWidget::isStarted()
{
    return m_isStarted;
}

void GLWidget::requestRender()
{
    emit renderRequested();
}

void VWidget::scaleUp()
{

}

void VWidget::scaleDown()
{

}

void VWidget::paintEvent(QPaintEvent *event)
{
    QPainter painter;
    painter.begin(this);
    painter.setRenderHint(QPainter::Antialiasing);

    //Know I'm getting signal/update because logs are full of this
//    qDebug() << "Painting frame";

    //This works, so painter is working here
//    QImage test(":/resources/img/PrivacyMode.png");
//    painter.drawImage(this->rect(),test);

    //This does not! I'm not getting anything from thread, or at least nothing visible
    painter.drawImage(this->rect(),dFrame);

    painter.end();
}

void VWidget::displayFrame(QImage img)
{
    dFrame = img;
    update();
}

Renderer::Renderer(VWidget *w)
    : m_vwidget(w),
      m_exiting(false),
      privacyImage(":/resources/img/PrivacyMode.png")
{
}

Renderer::~Renderer()
{
    qDebug() << "Cleaning up Renderer...";
}

void Renderer::stop()
{
    m_exiting = true;
}

void Renderer::render()
{
    if(m_exiting){
        emit finished();
        return;
    }

    QImage videoFrame;
    bool haveFrame = false;

    QPainter p(&videoFrame);
    //Same image that works directly in the widget, no errors, no crashes, just doens't get through signal?
    p.drawImage(0, 0, privacyImage);
    haveFrame = true;

    if(haveFrame)
        emit frameReady(videoFrame);
}

SGaist

Hi,

The calls to paintEvent are completely unrelated to your displayFrame slot being called. There are a lot of reasons for which paintEvent is called. If you want basic debugging, you should put your qDebug statement in the slot there.

By the way, why not use the new syntax when connecting m_renderer ?

VRonin

I'm sorry to spoil the party but I'm afraid it's a no-go.

From https://doc.qt.io/qt-5/thread-basics.html#gui-thread-and-worker-thread

All widgets and several related classes, for example QPixmap, don't work in secondary threads.

QPainter::drawImage ultimately calls QPaintEngine::drawImage that creates a QPixmap from that image and draws it.

P.S.

You call VWidget::requestRender when the timer timeouts but your code shows GLWidget::requestRender instead (class name is different)
VWidget::stopRendering() is currently a race condition. Renderer::m_exiting should be declared as type std::atomic_bool to make it safe

SGaist

@VRonin said in Proper way to pass QImage from thread through signal?:

QPainter::drawImage ultimately calls QPaintEngine::drawImage that crates a QPixmap from that image and draws it.

Nice one ! I forgot about that low level detail.

wesblake

So I can't do this?! I've tried EVERYTHING and always seem to run into an issue. :( I saw some examples where something similar was done, supposedly successfully. The GLWidget is a leftover but not that way in my code.
I'm out of options.
So what I have ultimately is a video relay app built on an SDK. The SDK supplies up to 3 raw images at a time, I need to display them on my video surface. Currently, the app uses 2 classes I wrote. On Mac and Linux, a QGLWidget based class, renders completely, using QPainter and QImage on a separate thread. Performs great, but has 3 issues. Known bug where styles are lost after some time on Windows, Mac is dropping OpenGL, and it's deprecated, running into issues with Catalina, works for now. For Windows, I use D2D but it's not threaded so there are some performance issues.
So I'm looking for a replacement, preferably threaded, and preferably one widget for all OS now.
I have tried and failed with (using Qt 5.12.X and 5.13.X):

QOpenGLWidget, ran into this bug preventing me from using it since we must run full screen: https://bugreports.qt.io/browse/QTBUG-49657
RHI, no idea where to start. Apparently it's beta, there's one example, and it uses QML while our app is a QWidget based desktop app.
Tried QWidget + QPainter with NO threading. I stopped half way because I got a test working and it seems it impacted UI performance enough it was a no go.
I feel like I've gone down many other rabbit holes too. QGraphicsView/Scene, etc. Each time I get half way and something just won't work.

Any bright ideas for me then? I really thought I was close this time!

wesblake

Actually, this should still be possible, this is where I started:
https://doc.qt.io/qt-5/threads-modules.html

"QPainter can be used in a thread to paint onto QImage, QPrinter, and QPicture paint devices. Painting onto QPixmaps and QWidgets is not supported. On macOS the automatic progress dialog will not be displayed if you are printing from outside the GUI thread."

I put a qDebug in displayFrame and it is firing off around 50 times a second. So, I think I still am passing the QImage incorrectly?

SGaist

Your issue is specific to drawImage with the default paint engine. To test, create a simple QImage, fill it with one colour and draw a rect if another's colour on it.

By the way, why are you loading your image and then calling drawImage ? Why not directly load the image itself in the target QImage ?

Asperamanca

Have you debugged the signal-slot connection? Do you arrive in the displayFrame slot? What size does the QImage you sent when emitting the signal as opposed the image you receive in the displayFrame slot?

wesblake

@SGaist said in Proper way to pass QImage from thread through signal?:

Your issue is specific to drawImage with the default paint engine. To test, create a simple QImage, fill it with one colour and draw a rect if another's colour on it.

By the way, why are you loading your image and then calling drawImage ? Why not directly load the image itself in the target QImage ?

Because in my final code I won't be loading an image, I will be drawing the composite image. That was just a test with a known image so I could see if anything is getting through. I already tried the rectangles too and nothing comes through.

@Asperamanca I did by putting a qDebug in the slot and I see it getting called. I will add some more in and check image size, etc. Thanks.

VRonin

@wesblake said in Proper way to pass QImage from thread through signal?:

I already tried the rectangles too and nothing comes through.

Change connect(m_renderer, SIGNAL(frameReady(QImage)), this, SLOT(displayFrame(QImage))); to connect(m_renderer, &Renderer::frameReady, this, &VWidget::displayFrame); so that we are sure the connection is working

change Renderer::render to

void Renderer::render()
{
    if(m_exiting){
        emit finished();
        return;
    }
const auto colors[] = {Qt::red,Qt::blue,Qt::black,Qt::green};
    QImage videoFrame;
videoFrame.fill(colors[std::uniform_int_distribution<int>(0,3)(std::default_random_engine)]);
     frameReady(videoFrame);
}

and see if anything changed

nothing comes through.

Define "nothing comes through"

wesblake

Thanks all for your help! How do I mark solved?
I feel dumb, it was simple. After adding in qDebug on the size of the image, both in Renderer and VWidget, both sides were 0.
So, it was simple, I was not initializing the QImage to paint into correctly in Renderer::render! I needed:

QImage videoFrame(m_glwidget->width(), m_glwidget->height(), QImage::Format_ARGB32_Premultiplied);

And now everything is working beautifully!

mrjj

@wesblake
Hi. Good you found it.
you can mark it solved on your first post int eh Topic Tools button
alt text