QT and GSTREAMER on buildroot

[Koax]

Hello,

I am work on boundary device board :

https://boundarydevices.com/product/nitrogen6x-board-imx6-arm-cortex-a9-sbc/

And I build an embedded system with buildroot ( tag 2017-05 ).

When I play a video like this :

//play video on h264
# gst-play-1.0 /root/surfer_h264.mp4 
Press 'k' to see a list of keyboard shortcuts.
Now playing /root/surfer_h264.mp4
CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.09.00_CORTEX-A8  build on Jul 13 2016 18:15:25.
mxc_sdc_fb fb@0: 1280x720 h_sync,r,l: 40,110,220  v_sync,l,u: 5,5,20 pixclock=74250000 Hz
imx-ipuv3 2800000.ipu: try ipu internal clk
imx-ipuv3 2800000.ipu: disp=0, pixel_clk=74250000 74250000 parent=74250000 div=1
[INFO]  bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0
Redistribute latency...
mxc_sdc_fb fb@0: 1280x720 h_sync,r,l: 40,110,220  v_sync,l,u: 5,5,20 pixclock=74250000 Hz
imx-ipuv3 2800000.ipu: try ipu internal clk
imx-ipuv3 2800000.ipu: disp=0, pixel_clk=74250000 74250000 parent=74250000 div=1
mxc_vpu 2040000.vpu_fsl: VPU interrupt received.

All is fine, playback is smooth as it would have been without any player

But when I play with player like QT example : /usr/lib/qt/examples/multimediawidgets/player/player

The video is very slow.

I would like to know how to use vpu acceleration ( imx ) with GSTREAMER on QT ?

I tried to play with :

# ./multimedia/video/qmlvideo/qmlvideo /root/surfer_h264.mp4 
mxc_sdc_fb fb@0: 1280x720 h_sync,r,l: 40,110,220  v_sync,l,u: 5,5,20 pixclock=74250000 Hz
imx-ipuv3 2800000.ipu: try ipu internal clk
imx-ipuv3 2800000.ipu: disp=0, pixel_clk=74250000 74250000 parent=74250000 div=1
QEglFSVivIntegration will set environment variable FB_MULTI_BUFFER=2 to enable double buffering and vsync.
 If this is not desired, you can override this via: export QT_EGLFS_IMX6_NO_FB_MULTI_BUFFER=1
Unable to query physical screen size, defaulting to 100 dpi.
To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.09.00_CORTEX-A8  build on Jul 13 2016 18:15:25.
[INFO]  bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0
CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.09.00_CORTEX-A8  build on Jul 13 2016 18:15:25
[INFO]  bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0

And the video is fluent.

I build with "qt-webkit-kiosk" package :
https://git.buildroot.net/buildroot/tree/package/qt-webkit-kiosk/qt-webkit-kiosk.mk
https://github.com/sergey-dryabzhinsky/qt-webkit-kiosk

And when I play a video with this application, I have :

# qt-webkit-kiosk -u /root/surfer_h264.mp4
QEglFSVivIntegration will set environment variable FB_MULTI_BUFFER=2 to enable double buffering and vsync.
 If this is not desired, you can override this via: export QT_EGLFS_IMX6_NO_FB_MULTI_BUFFER=1
Unable to query physical screen size, defaulting to 100 dpi.
To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
No signal BREAK defined
0:00:00.000887343   566   0xbfc400 INFO                GST_INIT gst.c:510:init_pre: Initializing GStreamer Core Library version 1.10.4
0:00:00.001393349   566   0xbfc400 INFO                GST_INIT gst.c:511:init_pre: Using library installed in /usr/lib
0:00:00.001473017   566   0xbfc400 INFO                GST_INIT gst.c:522:init_pre: Linux buildroot 4.1.15 #1 SMP PREEMPT Thu Jun 8 14:22:12 CEST 2017 armv7l
0:00:00.004414385   566   0xbfc400 INFO                GST_INIT gstmessage.c:126:_priv_gst_message_initialize: init messages
0:00:00.007911426   566   0xbfc400 INFO                GST_INIT gstcontext.c:83:_priv_gst_context_initialize: init contexts
0:00:00.009048439   566   0xbfc400 INFO      GST_PLUGIN_LOADING gstplugin.c:316:_priv_gst_plugin_initialize: registering 0 static plugins
0:00:00.009877449   566   0xbfc400 INFO      GST_PLUGIN_LOADING gstplugin.c:224:gst_plugin_register_static: registered static plugin "staticelements"
0:00:00.009941783   566   0xbfc400 INFO      GST_PLUGIN_LOADING gstplugin.c:226:gst_plugin_register_static: added static plugin "staticelements", result: 1
0:00:00.010125451   566   0xbfc400 INFO            GST_REGISTRY gstregistry.c:1738:ensure_current_registry: reading registry cache: /root/.cache/gstreamer-1.0/registry.arm.bin
/......
....../
0:00:28.819865074   580 0x63101b50 WARN                basesink gstbasesink.c:2834:gst_base_sink_is_too_late:<webkitvideosink0> warning: A lot of buffers are being dropped.
0:00:28.819990076   580 0x63101b50 WARN                basesink gstbasesink.c:2834:gst_base_sink_is_too_late:<webkitvideosink0> warning: There may be a timestamping problem, or this computer is too slow.
0:00:28.820121411   580 0x63101b50 INFO        GST_ERROR_SYSTEM gstelement.c:1890:gst_element_message_full_with_details:<webkitvideosink0> posting message: A lot of buffers are being dropped.
0:00:28.820320080   580 0x63101b50 INFO        GST_ERROR_SYSTEM gstelement.c:1917:gst_element_message_full_with_details:<webkitvideosink0> posted warning message: A lot of buffers are being dropped.
0:00:44.666170255   580 0x63101b50 WARN                basesink gstbasesink.c:2834:gst_base_sink_is_too_late:<webkitvideosink0> warning: A lot of buffers are being dropped.
0:00:44.666294256   580 0x63101b50 WARN                basesink gstbasesink.c:2834:gst_base_sink_is_too_late:<webkitvideosink0> warning: There may be a timestamping problem, or this computer is too slow.
0:00:44.666415257   580 0x63101b50 INFO        GST_ERROR_SYSTEM gstelement.c:1890:gst_element_message_full_with_details:<webkitvideosink0> posting message: A lot of buffers are being dropped.
0:00:44.666606926   580 0x63101b50 INFO        GST_ERROR_SYSTEM gstelement.c:1917:gst_element_message_full_with_details:<webkitvideosink0> posted warning message: A lot of buffers are being dropped.

Please, What is the signification of "A lot of buffers are being dropped' ?

Thanks

[Hung Tran]

@Koax Did you play by gst-launch ?

[Koax]

@Hung-Tran I tried with gst-launch and I have the same result of gst-play. the video is fluid all the time.

But, more clearly I would like play an video on browser with hardware acceleration ( video as fluid like gst-play ) .

[Hung Tran]

@Koax You can see, gst-launch can play smoothly your clip in your device but Qt is not. So, Qt does not support good gstreamer API and for my understand, QMediaplayer is under development and it is not fully good.

[Koax]

@Hung-Tran : OK , I see.
But in my project I worked with a board of boundary ( Nitrogen6X ).
=> https://boundarydevices.com/product/nitrogen6x-board-imx6-arm-cortex-a9-sbc/
That is why I want to use framework GSTREAMER for use hardware acceleration.

According to your post, I would like to know where we can see the roadmap of GSTREAMER on QT ?

Thanks for you support.

[jsulm]

@Koax Sorry, but the comment from @Hung-Tran isn't correct - at least I would like to hear what Qt does not support from "good gstreamer API" (what ever it is). QMediaPlayer is official part of Qt and is well tested ("not fully good" - some more details would be good, what is not good?).

"But, more clearly I would like play an video on browser with hardware acceleration" - this is a good information. Maybe you should ask Qt developers (developers mailing list) - they know better how all this is implemented in Qt.

[Koax]

@jsulm : Thanks to you for all this information.
Humm, so please where can I find the mailing list for GSTREAMER and QTWEBKIT ?

[jsulm]

@Koax http://lists.qt-project.org/mailman/listinfo/development

[Hung Tran]

@jsulm Sr if my information is not correct. However, I have asked the same question before and the answer I got that "decode using GPU not use Hardware IP so performance is not good on Qt in that time". So I chose the way that I use Qt to control Gstreamer directly, not using QMediaplayer of Qt for playback. It worked Ok but there is some problem with UI.
By the way, I heard that Qt4.7 support Phonon support good for media playback but it's not available on Qt5.