QT 6.3: Streaming camera image

[Kimosabe]

Hi everyone, I am new to QT, so please forgive me if I make some stupid question.
I am creating simple QT application for streaming camera image.
This is done on embedded Linux device.
QML code (main.qml) follows:

import QtQuick
import QtQuick.Controls
import QtMultimedia

Item {
    width: 1000
    height: 800

    MediaDevices {
        id: devices
    }

    function setDevice(id)
    {
        for (var i = 0; i < 5; i++)
        {
            if (devices.videoInputs[i].id == id)
            {
                return devices.videoInputs[i];
            }
        }
        console.log("Setting default device!");
        return devices.defaultVideoInput;
    }

    CaptureSession {
        objectName: "session"
        camera: Camera {
            objectName: "camera"
            id: camera
            cameraDevice: setDevice("/dev/video3")
        }
        imageCapture: ImageCapture {
            objectName: "capture"
            id: imageCapture
        }
        videoOutput: videoOutput
    }

    VideoOutput {
        width: parent.width
        height: parent.height
        id: videoOutput
        anchors.fill: parent
    }
}

C++ code follows:

#include <QGuiApplication>
#include <QQuickView>
#include <QQmlEngine>

int main(int argc, char* argv[])
{
    QGuiApplication app(argc,argv);
    QQuickView view;
    view.setResizeMode(QQuickView::SizeRootObjectToView);
    QObject::connect(view.engine(), &QQmlEngine::quit,
                     qApp, &QGuiApplication::quit);
    view.setSource(QUrl(u"qrc:/CameraStream/main.qml"_qs));
    view.show();
    return app.exec();
}

I have 5 video inputs available on device, /dev/video0 - /dev/video4.
Thing is that stream from camera sometimes work, but sometimes I get only empty window. Same thing is happening on all 5 inputs.

Am I doing something wrong, or missing something in code, so stream works randomly?

NOTE: Also I noticed that my setDevice function is called multiple times, meaning
every time I start application it triggers setDevice function six times.

NOTE: Alo I tried to print camera errors using signals for it, but no error is produced even when stream does not work.

Thanks in advance
Regards

[SGaist]

Hi and welcome to devnet,

If you want to do streaming (and not just show the camera feed on your application), then you should use a GStreamer pipeline.

[Kimosabe]

Well plan is to expand this application to have buttons for capturing camera image and also for video recording.
I know there is a camera example for that, but it is using widgets, which I do not prefer.
I executed gst-launch tool over specified camera files and stream works every time, but again I need a GUI application with multiple features.

Regards

[EinarJonG]

@SGaist said in QT 6.3: Streaming camera image:

If you want to do streaming (and not just show the camera feed on your application), then you should use a GStreamer pipeline.

Didn't QT6 remove all support for gstreamer pipelines in QML? How would that be done now?

[SGaist]

Yes, the custom pipeline has been removed. You will have to build it in C++.

[Kimosabe]

@SGaist
qmlglsink is not supported anymore in QT 6.
Is there some generic QT 6 sink solution for this?
Any suggestion what shall I use to create custom pipeline?

In past user could use GstGLVideoItem from qt part of gst-plugins-good.
Setup gstream stuff (pipeline, src, caps. etc...) using gstream API and then assign those stuff to GstGLVideoItem.

GstGLVideoItem is not supported in QT 6, meaning qt part of gst-plugins-good is not supported for QT 6 anymore.

[JoeCFD]

@Kimosabe Is qmlglsink not a support from gstreamer(not from Qt) for QML? Is it dropped for Qt6? I doubt it. Do you have a link for the info?

[JoeCFD]

@JoeCFD I googled a bit and found out it seems that qmlglsink does not support Qt6 now. But I think it will later.

[EinarJonG]

@JoeCFD

In order for this to work, both qmlglsink and the QML "glue" org.freedesktop.gstreamer.GLVideoItem need to be compiled.

The whole setup is littered with QT5 references, and it seems unmaintained.

[JoeCFD]

@EinarJonG Did you notice in your link?
qt5_element_init (GstPlugin * plugin)

There is no qt6_element_init (GstPlugin * plugin)

[Kimosabe]

@JoeCFD @EinarJonG
I looked now at "gst-plugins-good-1.22.1", by that I mean gst-plugins-good version 1.22.1 and there is a qt6 directory inside of it. Inside of that directory there is a "qt6_element_init (GstPlugin * plugin)" and new element called "GstGLQt6VideoItem", so I assume this is support for QT6, meaning "qml6glsink" is now available.

I was looking gst-plugins-good source code at https://gstreamer.freedesktop.org/src/gst-plugins-good/ and it seems like qt6 support was introduced starting from 1.21.3 version. Please correct me if I am wrong.

[JoeCFD]

@Kimosabe good to know. Thanks. Install and try it out.
Try to get streamer source from
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
I remember I had some issues when I downloaded the source from the link
https://gstreamer.freedesktop.org/src/gst-plugins-good/
to build it.

It can be very messy to upgrade gstreamer since you may have to rebuild Qt against the version of gstreamer you install. Also some other apps on Linux may be dependent on the specific version of gstreamer as well. You may need to rebuild all of them.

[Kimosabe]

@JoeCFD thanks for advice. Yes I am aware that updating gstreamer can break a lot of other things on Linux OS that are depending on gstreamer.
If you manage to stream camera image via QT using new gst-plugins-good please leave a comment, so I have information if it is worth of effort to implement it on my side.

[JoeCFD]

@Kimosabe I am building the latest gstreamer now because of delay and memory leak in RTSP streaming. I noticed there may be only one module QMultimediaGstTools and QMultimedia-plugins in Qt which need gstreamer. If you do not use them, simply ignore Qt rebuild. I could be wrong. But you may need to get the latest libva
https://github.com/intel/libva.git
to build with gstreamer. Build libva first and then gstreamer with the new streamer.

[prettyQt]

Hi I'm trying to get a custom gstreamer pipeline going.
I am building gstreamer 1.23 from source:

git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git
meson setup builddir
meson compile -C builddir

Once this builds, how would I then point my .pro file at this?

For reference I am trying to build this example project.

The current .pro file looks like this:

TEMPLATE = app

QT += qml quick widgets

QT_CONFIG -= no-pkg-config
CONFIG += link_pkgconfig debug
PKGCONFIG = \
    gstreamer-1.0 \
    gstreamer-video-1.0

DEFINES += GST_USE_UNSTABLE_API

INCLUDEPATH += ../lib

SOURCES += main.cpp

RESOURCES += qmlsink.qrc

# Additional import path used to resolve QML modules in Qt Creator's code model
QML_IMPORT_PATH =

How would I modify this .pro file to point to my gstreamer build?

Currently the line

#include <gst/gst.h>

in main.cpp references my system's gstreamer at /usr/include/gstreamer-1.0/gst/gst.h.

The line

import org.freedesktop.gstreamer.Qt6GLVideoItem 1.0

in main.qml throws an error.

[JoeCFD]

@prettyQt
do not install it now. Otherwise, you will mess it up with the installed gstreamer version.
rm -rf builddir
meson setup -prefix=/opt/gstreamer builddir /* install it to a place which does not affect the installed one */
ninja -C builddir
sudo ninja install -C builddir

[prettyQt]

@JoeCFD thanks that makes sense. Once this is done, how would I then utilize this build in the Qt project?

[JoeCFD]

@prettyQt
set the right path for compiling with the built gstreamer

export PKG_CONFIG_PATH=/opt/gstreamer/lib/pkgconfig/:/opt/gstreamer/share/pkgconfig:$PKG_CONFIG_PATH

to test and run, the following is also needed

export PATH=/opt/gstreamer/bin:$PATH
export LD_LIBRARY_PATH=/opt/gstreamer/lib:$LD_LIBRARY_PATH
which gst-inspect-1.0 /* to check the right bin path */
ldd /opt/gstreamer/bin/gst-inspect-1.0 /* to check the right lib path */ 

Not sure if you need to rebuild qmultimedia module with the new gstreamer version. Also the video driver may need to be rebuilt as well.

[prettyQt]

@JoeCFD I got an error when running

sudo ninja install -C builddir/

ninja: Entering directory `builddir/'
[1/2] Installing files.
Traceback (most recent call last):
  File "/home/robby/.local/bin/meson", line 5, in <module>
    from mesonbuild.mesonmain import main
ModuleNotFoundError: No module named 'mesonbuild'
FAILED: meson-internal__install 
/home/robby/.local/bin/meson install --no-rebuild
ninja: build stopped: subcommand failed.

Any ideas?

[prettyQt]

It looks like the version of meson sudo was using was wrong. It worked after I ran

sudo python3 -m pip install meson

This post talks about it.