Pixel framebuffer data in C++ to QML application using the GPU as much as possible

tobias_m

Hi!

I'm trying out different alternatives to feed a framebuffer from the C++ layer to QML. I was hoping to leverage the GPU as much as possible to avoid doing too much drawing and pixel manipulation on the CPU side.

My current architecture looks like this:

I manipulate a QImage framebuffer from C++ using the QPainter APIs like drawLine and drawPolygon. This is fine and the pixel manipulations are not that expensive, although I was hoping to leverage a GPU backend for drawing lines and filled polygons.

The QImage is sent over to the QML layer via a signal (which is pumped by a 60 FPS timer) to a custom derived QQuickPaintedItem which sets the image and calls update().
The image is then drawn by implementing the overloaded "paint" method using drawImage().

I've noticed that by just calling update() on the QQuickPaintedItem incurs a direct CPU penalty, even though no new image was set.

From my understanding using QPainter on a QImage and using a custom QQuickPaintedItem will use the software rasterizer, and I don't think that's what I want.

So my questions are:

How can I utilize a QPainter GPU backend for direct manipulation of my framebuffer?
How can I incorporate that "thing" into a QML item for a near 60 FPS guarantee, or at least by not using the CPU for anything but the calculations needed before drawing the lines and polygons?

My thought would be:

Draw "stuff" into a texture and use that by Qt3D perhaps in QML? Using OpenGL directly seems like a hazzle and doesn't really fit nicely into the app.
Use some other backend for QPainter to use against so that the GPU is used instead of the rasterizer.

Please advice! :)

P.S. For full details on my implementation check https://github.com/tobiasmarciszko/qt_raytracer_challenge/blob/master/raytracer/main/raytracerbackend.cpp and follow the wireframe() method to see the flow from C++ to QML. This current code doesn't use the 60 FPS timer, but you'll get the idea hopefully :)

Thanks!
Tobias

tobias_m

One alternative that I found is to subclass a QQuickItem (instead of a QQuickPainterItem) and draw the QImage as a texture. But it doesn't really give me a huge performance boost anyway. Constructing the raw QImage from QPainter calls and the texture conversion is still "too heavy"...

QSGNode *ImageItem::updatePaintNode(
    QSGNode *oldNode, QQuickItem::UpdatePaintNodeData *updatePaintNodeData) {

    auto node = dynamic_cast<QSGSimpleTextureNode *>(oldNode);

    if (!node) {
        node = new QSGSimpleTextureNode();
    }

    QSGTexture *texture = window()->createTextureFromImage(m_image, QQuickWindow::TextureIsOpaque);
    node->setOwnsTexture(true);
    node->setRect(boundingRect());
    node->markDirty(QSGNode::DirtyForceUpdate);
    node->setTexture(texture);
    return node;
}

Maybe the next step is to "convert" all my QPainter calls into corresponding "things" in the updatePaintNode method?

tobias_m

Allright, I have got a few answers from other places that are potential options, so here's a list of potential ways to do what I want.

QQuickPaintedItem but will use software rasterizer (or an FBO which will be a texture upload on every frame).
QQuickItem and updatePaintNode. More low level then QQuickPaintedItem which will use the SceneGraph API. How this performs is entirely up to how painting is "done".
QQuickFrameBufferObject https://doc.qt.io/qt-5/qquickframebufferobject.html lets you use OpenGL directly in your custom item.
The QML Canvas element can draw primitives using the Canvas API known from HTML5. Can also use an FBO to draw.
"Shapes" in QML can draw lines and, Shapes. But requires dynamically creating the elements and then updating them from C++.
QNanoPainter which is an OpenGL accelerated C++ vector drawing library for Qt: https://github.com/QUItCoding/qnanopainter

I haven't explored the suggested options in code yet. But this list looks pretty exhaustive to me. I'm marking this question as solved since I don't expect more alternatives to arise :)