Sending OpenCV images continuously from C++ to Qml
-
wrote on 8 Aug 2019, 11:19 last edited by Yunus 8 Aug 2019, 11:20
Hi all, I am stuck in a subject and I need help. I have a while loop in C++ part and it is taking frames from the camera continuously using OpenCV library and at the same time my qml UI is also working. Both are working in different threads so there is no problem up to now.
I want to send my OpenCV images to Qml part continuously. I can convert OpenCV images to QImage. For example, I can send QString to qml easily using emit function. Is there a simple way like this for images. I looked at several examples but I couldnt find an efficient one. Here is the examples which I tried:
-
Hi all, I am stuck in a subject and I need help. I have a while loop in C++ part and it is taking frames from the camera continuously using OpenCV library and at the same time my qml UI is also working. Both are working in different threads so there is no problem up to now.
I want to send my OpenCV images to Qml part continuously. I can convert OpenCV images to QImage. For example, I can send QString to qml easily using emit function. Is there a simple way like this for images. I looked at several examples but I couldnt find an efficient one. Here is the examples which I tried:
@yunus said in Sending OpenCV images continuously from C++ to Qml:
I looked at several examples but I couldnt find an efficient one
So even the image provider was too slow?
Try subclassing QQuickItem then, and paint the image directly there as a texture, it should be faster. An example of this (warning: very old code! But should work) https://github.com/sierdzio/closecombatfree/blob/master/src/qmlBase/ccfqmlbasemap.cpp. Once you register the item you can use it in QML as if it was a built-in component.
Alternatively, there is also QNanoPainter.
-
wrote on 8 Aug 2019, 11:38 last edited by Yunus 8 Aug 2019, 11:39
@sierdzio said in Sending OpenCV images continuously from C++ to Qml:
@sierdzio Thanks for ur reply. I couldnt even try image provider. I can send image to qml but not continuously. I am new on Qt and I need a simple example to do that. -
Look at the documentation, then. It contains several examples https://doc.qt.io/qt-5/qquickimageprovider.html
-
You should implement an object capable of adding frames on a
QAbstractVideoSurface
, here is a quick explanation on how to do it : https://forum.qt.io/post/474269 (ignore theQScreen::grabWindow
part). The non trivial part is converting your frames toQVideoFrame
.You could then use this object as a source of a QML
VideoOutput
. -
Look at the documentation, then. It contains several examples https://doc.qt.io/qt-5/qquickimageprovider.html
wrote on 8 Aug 2019, 13:40 last edited by Yunus 8 Aug 2019, 13:41@sierdzio Okey I tried to use ImageProvider example in here. Okey I can send my image but how can i update images when I changed the QImage in cpp part. Here is my files:
LiveImageProvider.cpp
// LiveImageProvider.cpp #include "liveimageprovider.h" #include <QDebug> /** * @brief Image provider that is used to handle the live image stream in the QML viewer. */ LiveImageProvider::LiveImageProvider() : QQuickImageProvider(QQuickImageProvider::Image) { this->no_image = QImage(":/images/no_image.png"); this->blockSignals(false); } QImage LiveImageProvider::requestImage(const QString &id, QSize *size, const QSize &requestedSize) { QImage result = this->image; if(result.isNull()) { result = this->no_image; } if(size) { *size = result.size(); } if(requestedSize.width() > 0 && requestedSize.height() > 0) { result = result.scaled(requestedSize.width(), requestedSize.height(), Qt::KeepAspectRatio); } return result; } void LiveImageProvider::updateImage(const QImage &image) { if(this->image != image) { this->image = image; emit imageChanged(); } }
LiveImageProvider.h
// LiveImageProvider.h #ifndef LIVEIMAGEPROVIDER_H #define LIVEIMAGEPROVIDER_H #include <QImage> #include <QQuickImageProvider> class LiveImageProvider : public QObject, public QQuickImageProvider { Q_OBJECT public: LiveImageProvider(); QImage requestImage(const QString &id, QSize *size, const QSize &requestedSize) override; public slots: void updateImage(const QImage &image); int ulan(); signals: void imageChanged(); private: QImage image; QImage no_image; }; #endif // LIVEIMAGEPROVIDER_H
main.qml
import QtQuick 2.9 import QtQuick.Window 2.2 import QtQuick.Controls 1.0 Window { visible: true width: 1200 height: 900 title: qsTr("Hello World") Image { id: liveImage property bool counter: false asynchronous: true source: "image://live/image" anchors.fill: parent fillMode: Image.PreserveAspectFit cache: false function reload() { counter = !counter source = "image://live/image?id=" + counter } } }
-
Hi,
Out of curiosity, do you do any processing with OpenCV ?
If not, what about using QtMultimedia and the Camera type ? -
Hm indeed, looks like the image provider API does not contain any useful API to update the view continuously. You could hack around it with the solution you linked (https://forum.qt.io/topic/38978/solved-send-qimage-from-c-to-qml-via-qquickimageprovider-or-using-a-signal/4), which requests a new image periodically using a timer. But probably a better solution is to prepare a video surface like @GrecKo suggests, or to reimplement QQuickItem. Sorry for leading you down a suboptimal path!
-
Hi,
Out of curiosity, do you do any processing with OpenCV ?
If not, what about using QtMultimedia and the Camera type ?wrote on 9 Aug 2019, 06:42 last edited by Yunus 8 Sept 2019, 06:51@sgaist Yeah, I m doing lots of task in Opencv and I need to send opencv images to qml continuously. I used this example. But I can only send images triggering with qml buttons. I want to send the images continuously like video.
-
Hm indeed, looks like the image provider API does not contain any useful API to update the view continuously. You could hack around it with the solution you linked (https://forum.qt.io/topic/38978/solved-send-qimage-from-c-to-qml-via-qquickimageprovider-or-using-a-signal/4), which requests a new image periodically using a timer. But probably a better solution is to prepare a video surface like @GrecKo suggests, or to reimplement QQuickItem. Sorry for leading you down a suboptimal path!
-
You can also take a look at one of my projects:
https://github.com/DeiVadder/Sindepile-Challenge-QMLI used that one to experiment with the QImageProvoder class and I update the ui each time a new image is available, not based on a timer.
If you don't really care about performance, you could also simply force an update each time possible, by reacting to the
frameSwapped
signal -
You can also take a look at one of my projects:
https://github.com/DeiVadder/Sindepile-Challenge-QMLI used that one to experiment with the QImageProvoder class and I update the ui each time a new image is available, not based on a timer.
If you don't really care about performance, you could also simply force an update each time possible, by reacting to the
frameSwapped
signalwrote on 9 Aug 2019, 08:05 last edited by@j-hilk @sierdzio Thank you so much. I am very glad to both. I think I fixed my problem. I solved using Imageprovider and QTimer using example here. And also @J-Hilk project is working well. I ll use one of these. Thank you Qt Forum, I am very happy to be able to fix this problem
-
QQuickImageProvider
is not the correct solution to display videos. Implementing a source for aVideoOutput
is the sanest thing to do.
If we are advising convoluted ways to display a video, I'd like to propose writing aQAbstractTableModel
containing each pixel value in its cells, and display that with a TableView and 1x1 Rectangle delegates. -
QQuickImageProvider
is not the correct solution to display videos. Implementing a source for aVideoOutput
is the sanest thing to do.
If we are advising convoluted ways to display a video, I'd like to propose writing aQAbstractTableModel
containing each pixel value in its cells, and display that with a TableView and 1x1 Rectangle delegates. -
QQuickImageProvider
is not the correct solution to display videos. Implementing a source for aVideoOutput
is the sanest thing to do.
If we are advising convoluted ways to display a video, I'd like to propose writing aQAbstractTableModel
containing each pixel value in its cells, and display that with a TableView and 1x1 Rectangle delegates.@grecko awesome idea, may I suggest an add-on .
Will need some work on the model, but I suggest only updating every other row, for a 90's era interlacing effect!
😉 -
@Yunus Here's a post where I explain it textually : https://forum.qt.io/post/474269
Here is some sample code I wrote some time ago : https://gist.github.com/oKcerG/a5b59c7583c85ff03bdec238eed8bcd8
The import part is the videoSurface setter which starts the videoSurface, and then I periodically present a frame on it.
Make sure to chose a correct
QVideoSurfaceFormat
, and don't do like I did for generating aQVideoFrame
( QPixmap -> QImage -> QImage in the correct format -> QVideoFrame), that's way too much conversions, but I did that only as a quick proof of concept.There must be a more direct way to convert cv::Mat to QVideoFrame, there might even be a copyless solution but that's not my domain of expertise.
-
wrote on 9 Aug 2019, 08:34 last edited by
Thank you I ll try
-
wrote on 5 Nov 2019, 15:51 last edited by
As far as I understand you want to read video from camera, process frames using OpenCV then display output in QML. This solution you used is not good. It uses timer with a second interval and each timeout event will send image to QML.
2 Disadvantages :-
You are hardcoding your processing speed to 1 fps. Even camera provides more frames you are reading only 1 frame. To catch everything your timer ticks should be more frequent than your video reader.
-
In a low power system or with a camera with low fps there will be synchronization problems.
You should use QAbstractVideoFilter . There is a good example in the page. Basically it allows you to process image while it is being sent to VideoOutput by Camera in Qml. And it will be definitely more easier and efficient than your solution.
-