[Solved] Best way to display/update live image data in a QML GUI

[serdef]

Hi,

I'm developing an application which has a qml gui, and some C++ backend.
The backend receives udp frames with image data from a camera (line by line), and colorizes some pixels.
The backend processing is no problem, I can construct/update a qimage. But how do I show this in my GUI without much overhead ?
I have been looking at QQuickImageProvider, but I don't see how I could trigger
an on screen update when the backend frame has come in (will be between 2 and 10 frames/s, depending on the backend).

What is the most efficient and/or cleanest way to show and update custom frames of bitmap data in qml ?

Tnx
Serge

[dasRicardo]

Hmm, i think the best way to do this is the "SceneGraph":http://qt-project.org/doc/qt-5/qtquick-visualcanvas-scenegraph.html it's a separate thread and the performance on an update seams well.

A "QSGGeometryNode":http://qt-project.org/doc/qt-5/qsggeometrynode.html with "QSGTextureMaterial":http://qt-project.org/doc/qt-5/qsgtexturematerial.html should be fine

[serdef]

Hi Ricardo,
thanks for the advice.

I looked into scenegraph, but this looks quite complex, couldn't find a simple example with variable image data, that goes all the way up to displaying in QML. I'd love to try it though, it would allow some hardware accelerated effects, that would be fun.

Now I tried a more simple way first, deriving from QQuickPaintedItem,
and this seems to work (performance still to be tested).

@MaQCamVisual::MaQCamVisual(QQuickItem *parent)
: QQuickPaintedItem(parent)
{
....
}
@

then when data is changed, for example, camera number is changed,
i just call update(). I will do so when I get a new frame too.
@void MaQCamVisual::setCameraNr(int cameranr)
{
if(m_cameranr != cameranr)
{
m_cameranr = cameranr;
update();
}

}

@
The paint routine then draws the image :
@void MaQCamVisual::paint(QPainter *painter)
{
painter->drawImage(boundingRect().adjusted(1, 1, -1, -1),*pCamImg[m_cameranr]);
}
@

in the C startup routines
@ qmlRegisterType<MaQCamVisual>("MaQCamVisual", 1, 0, "MaQCamVisual");@

then finally in QML:
@import MaQCamVisual 1.0
....

MaQCamVisual {
    id: mainCam
    x: 1
    y: 1
    width: 800
    height:  600

cameranr: 1
}@

[dasRicardo]

Yes, great if it's work for you. I had played around with QQuickPaintedItem but was very annoyed because you can call update() but it doesn't force an repaint.

bq. void QQuickPaintedItem::update(const QRect & rect = QRect())
Schedules a redraw of the area covered by rect in this item. You can call this function whenever your item needs to be redrawn, such as if it changes appearance or size.
This function does not cause an immediate paint; instead it schedules a paint request that is processed by the QML Scene Graph when the next frame is rendered. The item will only be redrawn if it is visible.
Note that calling this function will trigger a repaint of the whole scene.

Yeah, scenegraph is a pain in the ass for me to. :)

bq. I’d love to try it though, it would allow some hardware accelerated effects, that would be fun.

Yes, but with ShaderEffect you can did the same direct in qml you doesn't need scenegraph for nice effects.

[serdef]

It does the trick for now.
I will delve into the scenegraph stuff later, that may give a better and more consistent frame rate.
Thanks for your help and inspiration.

[sletta]

bq. Yeah, scenegraph is a pain in the ass for me to. No really good or simple examples and some of them are buggy especially things like the ShaderEffects. Works on Desktop but not on mobile because OpenGL ES is very straight :)

Could you please elaborate on which ones are not working properly?

[sletta]

As for the original question, your best option is to use a QSGSimpleTextureNode and QSGTexture as already suggested. Going through QQuickPaintedItem is going to cost you a lot of performance.

An example on how to create textures from images can be found here http://qt.apidoc.info/5.2.0/qtquick/qtquick-scenegraph-graph-noisynode-cpp.html.

[dasRicardo]

[quote author="sletta" date="1410458827"]
Could you please elaborate on which ones are not working properly?

[/quote]

Sorry, my fault. I take a look into the simple material example and it looks good. In my mind there was no types like lowp in the variables definitions but there are. Maybe in another example. If i found it i will report it and make a patch.