Sending OpenCV images continuously from C++ to Qml

[Yunus]

Hi all, I am stuck in a subject and I need help. I have a while loop in C++ part and it is taking frames from the camera continuously using OpenCV library and at the same time my qml UI is also working. Both are working in different threads so there is no problem up to now.

I want to send my OpenCV images to Qml part continuously. I can convert OpenCV images to QImage. For example, I can send QString to qml easily using emit function. Is there a simple way like this for images. I looked at several examples but I couldnt find an efficient one. Here is the examples which I tried:

Here
and here

[sierdzio]

@yunus said in Sending OpenCV images continuously from C++ to Qml:

I looked at several examples but I couldnt find an efficient one

So even the image provider was too slow?

Try subclassing QQuickItem then, and paint the image directly there as a texture, it should be faster. An example of this (warning: very old code! But should work) https://github.com/sierdzio/closecombatfree/blob/master/src/qmlBase/ccfqmlbasemap.cpp. Once you register the item you can use it in QML as if it was a built-in component.

Alternatively, there is also QNanoPainter.

[Yunus]

@sierdzio said in Sending OpenCV images continuously from C++ to Qml:
@sierdzio Thanks for ur reply. I couldnt even try image provider. I can send image to qml but not continuously. I am new on Qt and I need a simple example to do that.

[sierdzio]

Look at the documentation, then. It contains several examples https://doc.qt.io/qt-5/qquickimageprovider.html

[GrecKo]

You should implement an object capable of adding frames on a QAbstractVideoSurface, here is a quick explanation on how to do it : https://forum.qt.io/post/474269 (ignore the QScreen::grabWindow part). The non trivial part is converting your frames to QVideoFrame.

You could then use this object as a source of a QML VideoOutput.

[Yunus]

@sierdzio Okey I tried to use ImageProvider example in here. Okey I can send my image but how can i update images when I changed the QImage in cpp part. Here is my files:

LiveImageProvider.cpp

// LiveImageProvider.cpp
#include "liveimageprovider.h"
#include <QDebug>
/**
* @brief Image provider that is used to handle the live image stream in the QML viewer.
 */
LiveImageProvider::LiveImageProvider() : QQuickImageProvider(QQuickImageProvider::Image)
{
    this->no_image = QImage(":/images/no_image.png");
    this->blockSignals(false);
}


QImage LiveImageProvider::requestImage(const QString &id, QSize *size, const QSize &requestedSize)
{
    QImage result = this->image;

    if(result.isNull()) {
        result = this->no_image;
    }

    if(size) {
        *size = result.size();
    }

    if(requestedSize.width() > 0 && requestedSize.height() > 0) {
        result = result.scaled(requestedSize.width(), requestedSize.height(), Qt::KeepAspectRatio);
    }

    return result;
}

void LiveImageProvider::updateImage(const QImage &image)
{
    if(this->image != image) {
        this->image = image;
        emit imageChanged();
    }
}

LiveImageProvider.h

// LiveImageProvider.h
#ifndef LIVEIMAGEPROVIDER_H
#define LIVEIMAGEPROVIDER_H

#include <QImage>
#include <QQuickImageProvider>

class LiveImageProvider : public QObject, public QQuickImageProvider
{
    Q_OBJECT
public:
    LiveImageProvider();

    QImage requestImage(const QString &id, QSize *size, const QSize &requestedSize) override;

public slots:
    void updateImage(const QImage &image);
    int ulan();

signals:
    void imageChanged();

private:
    QImage image;
    QImage no_image;
};

#endif // LIVEIMAGEPROVIDER_H

main.qml

import QtQuick 2.9
import QtQuick.Window 2.2
import QtQuick.Controls 1.0


Window {
    visible: true
    width: 1200
    height: 900
    title: qsTr("Hello World")
    Image {
      id: liveImage
      property bool counter: false
    
      asynchronous: true
      source: "image://live/image"
      anchors.fill: parent
      fillMode: Image.PreserveAspectFit
      cache: false
    
    
      function reload() {
        counter = !counter
        source = "image://live/image?id=" + counter
      }
      }
}

[SGaist]

Hi,

Out of curiosity, do you do any processing with OpenCV ?
If not, what about using QtMultimedia and the Camera type ?

[sierdzio]

Hm indeed, looks like the image provider API does not contain any useful API to update the view continuously. You could hack around it with the solution you linked (https://forum.qt.io/topic/38978/solved-send-qimage-from-c-to-qml-via-qquickimageprovider-or-using-a-signal/4), which requests a new image periodically using a timer. But probably a better solution is to prepare a video surface like @GrecKo suggests, or to reimplement QQuickItem. Sorry for leading you down a suboptimal path!

[Yunus]

@sgaist Yeah, I m doing lots of task in Opencv and I need to send opencv images to qml continuously. I used this example. But I can only send images triggering with qml buttons. I want to send the images continuously like video.

[Yunus]

@sierdzio Thanks for ur help. I will try the example using timer I hope it works :(

[J.Hilk]

@yunus

You can also take a look at one of my projects:
https://github.com/DeiVadder/Sindepile-Challenge-QML

I used that one to experiment with the QImageProvoder class and I update the ui each time a new image is available, not based on a timer.

If you don't really care about performance, you could also simply force an update each time possible, by reacting to the frameSwapped signal

[Yunus]

@j-hilk @sierdzio Thank you so much. I am very glad to both. I think I fixed my problem. I solved using Imageprovider and QTimer using example here. And also @J-Hilk project is working well. I ll use one of these. Thank you Qt Forum, I am very happy to be able to fix this problem

[GrecKo]

QQuickImageProvider is not the correct solution to display videos. Implementing a source for a VideoOutput is the sanest thing to do.
If we are advising convoluted ways to display a video, I'd like to propose writing a QAbstractTableModel containing each pixel value in its cells, and display that with a TableView and 1x1 Rectangle delegates.

[Yunus]

@grecko I also want the right way but I am new on Qt. Do you have an example to advise me about your method

[J.Hilk]

@grecko awesome idea, may I suggest an add-on .

Will need some work on the model, but I suggest only updating every other row, for a 90's era interlacing effect!
😉

[GrecKo]

@Yunus Here's a post where I explain it textually : https://forum.qt.io/post/474269

Here is some sample code I wrote some time ago : https://gist.github.com/oKcerG/a5b59c7583c85ff03bdec238eed8bcd8

The import part is the videoSurface setter which starts the videoSurface, and then I periodically present a frame on it.

Make sure to chose a correct QVideoSurfaceFormat, and don't do like I did for generating a QVideoFrame ( QPixmap -> QImage -> QImage in the correct format -> QVideoFrame), that's way too much conversions, but I did that only as a quick proof of concept.

There must be a more direct way to convert cv::Mat to QVideoFrame, there might even be a copyless solution but that's not my domain of expertise.

[Yunus]

Thank you I ll try

[Muhammet Ali Asan]

As far as I understand you want to read video from camera, process frames using OpenCV then display output in QML. This solution you used is not good. It uses timer with a second interval and each timeout event will send image to QML.
2 Disadvantages :

• You are hardcoding your processing speed to 1 fps. Even camera provides more frames you are reading only 1 frame. To catch everything your timer ticks should be more frequent than your video reader.

• In a low power system or with a camera with low fps there will be synchronization problems.

You should use QAbstractVideoFilter . There is a good example in the page. Basically it allows you to process image while it is being sent to VideoOutput by Camera in Qml. And it will be definitely more easier and efficient than your solution.