QtMultimedia, QtQuick and vaapisink
we are developing an application that plays an RTSP stream from a ip camera. We are using Qt5.8, QtMultimedia and QtQuick2. The PC running the application has an intel GPU chipset, so we have enabled vaapi all over gstreamer 1.0 and we made it work smoothly.
Before any Qt development we have used gstreamer directly and it works with this:
DISPLAY=:0 gst-launch-1.0 playbin uri=rtsp://XXX:YYY@192.168.1.103:554/profile3/media.smp
Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Progress: (open) Opening Stream Progress: (connect) Connecting to rtsp://XXXXX:YYYYY@192.168.1.103:554/profile3/media.smp Progress: (open) Retrieving server options Progress: (open) Retrieving media info Progress: (request) SETUP stream 0 Progress: (request) SETUP stream 1 Progress: (request) SETUP stream 2 Progress: (open) Opened Stream Setting pipeline to PLAYING ... New clock: GstSystemClock Progress: (request) Sending PLAY request Progress: (request) Sending PLAY request Progress: (request) Sent PLAY request libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 libva info: va_openDriver() returns 0 Got context from element 'vaapidecode': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL; Redistribute latency... AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse" Missing element: VND.ONVIF.METADATA RTP depayloader WARNING: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'application/x-rtp, media=(stint)4019201696, seqnum-base=(uint)36341, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1'. Additional debug info: gsturidecodebin.c(939): unknown_type_cb (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0 Got context from element 'playsink': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL; AL lib: (EE) UpdateDeviceParams: Failed to set 8000hz, got 48000hz instead Redistribute latency... Redistribute latency... ^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:01.868192155 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...
As I said this plays smoothly and the cpu consumption is almost 0. Then we develop a test application with Qt, but the visualization is jumping, as if it would loose some frames. We have debug gstreamer in order to see what components is QtMultimedia using and we see something strange:
vaapisink pad is created at the beginning and released at the the end, but it's not linked anywhere else.
vaapidecode and vaapipostproc is used through the entire run.
There is a video-output-bin created and used, and this is linked to qgstvideorenderersink, but I have no clue what output is this sink using (vaapisink / xvimagesink / ...)
We have used also qt-gstreamer which have some classes using vaapi (qtquick2videosink, qtvideosink) and we see vaapisink used all over the execution (using GST_DEBUG).
So after all this, our question is somewhat: Are we doing anything wrong? Is vaapisink used by QtMultimedia 5.8? Is there something we must change? Any help would be much appreciated. And if there is need for any code or more explanation I will be glad to answer.
Thanks in advance. Regards, Álvaro
Hi and welcome to devnet,
From a quick look at the GStreamer backend it seems that it will try to use the vaapi sink if available and possible. You should enable the
qt.multimedialogging to get more information about what is happening.