Qt Android gstreamer , qtquick2videosink slow down video.

DkDkDk

Hi All need help,

Here is my current setup:

Host Windows 10
Target Android API Level 28, NDK 21.3.6
Platform QT 5.14.1
Gstreamer 1.82
Gstremer Plugin - gstqtvideosink (https://github.com/GStreamer/qt-gstreamer/tree/master/elements/gstqtvideosink)

I have taken Gstremer and Video render code from https://github.com/CubePilot/qgroundcontrol-herelink

I have pipeline where its reads rtsp stream then render on video screen and same stream goes to server using rtmp.

Code is working good but when there is more pixel change in video frame, video rendering on screen got slow.

here is my pipe line

gst_bin_add_many(GST_BIN(_pipeline), dataSource,srcqueue,_teeStream,prequeue, demux, parser, decoder, queue1, _videoSink, nullptr);

is there is any other option to render gstremer video in QML?

SGaist

Hi,

You can use a custom GStreamer pipeline with QMediaPlayer. Provided that you have the QtMultimedia GStreamer backend built for Androïd.

DkDkDk

@SGaist thaks for quick reply.
I looked into it and fount that is It is hard to manipulate pipeline for other use.
I am using same pipeline for rtmp stream. I have to carry pipeline or tee pointer for that.

SGaist

So your device does both streaming over the network and visualization on screen ? All in the same application ?

DkDkDk

Yes. streaming is working good.
but video render on App UI creating problem.
I have disabled streaming still same problem is there. video start getting slow as I get my hand near camera view(more pixel changes).
this is raw pipeline I have used.

rtspsrc location="rtsp://127.0.0.1:8554/fpv_stream" latency=41 udp-reconnect=1 timeout=0 do-retransmission=false ! rtph264depay ! h264parse ! avdec_h264 ! queue ! qtquick2videosink sync=true update="onUpdateThunk"

DkDkDk

Hey @SGaist thanks for your valuable time. it was minor changes in code fixed that problem.
in video sink "sync" property was true. I make it false and its work with minor video lag but it is acceptable.

        if ((videoSink = gst_element_factory_make("qtquick2videosink", NULL)) == NULL) {
                qCritical("Failed to create qtquick2videosink. Make sure it is installed correctly");
                return NULL;
            }   
            g_object_set(G_OBJECT(videoSink), "sync", gboolean(false), NULL);
            g_signal_connect(videoSink, "update", G_CALLBACK(onUpdateThunk), (void* )this);

SGaist

Great ! Glad you found a solution even if not ideal.

Do you know if there's any hardware you could maybe take advantage of in your device ?

DkDkDk

I am using this device https://docs.cubepilot.org/user-guides/herelink/herelink-overview. It have Mali T860 GPU. As per the Gstremer documentation, qtquick2videosink uses internally OpenGL/OpenGLES means its already use hardware acceleration. I think this issue is due to qtquick2videosink plugin compatibility with the latest Gstreamer(gstreamer-1.0-android-armv7-1.18.1/ndk21).

https://github.com/CubePilot/qgroundcontrol-herelink/blob/388aa2aaa3536a3f0870e7e9004617833adc5e75/src/VideoStreaming/VideoSurface.cc#L53
Here they have used gstreamer-1.0-android-armv7-1.5.2/ndk15c and its working on same device.

SGaist

Sorry, I meant for video encoding.

In any case, I would separate the streaming and the preview if possible. Generating the network stream for one or more device can be very heavy CPU wise.