Simple app to show webcam output via GStreamer pipeline
-
I am trying to build a simple app on macOS that can use a GStreamer pipeline to open the webcam as a video source and output that to the Qt app.
I have looked at a few examples to draw inspiration:
- https://doc.qt.io/qtforpython-6/examples/example_multimedia_player.html
- https://doc.qt.io/qt-5/qmediaplayer.html#setMedia (specifically the gst-pipeline example)
I have the following program, but the output is a black content inside the app window .. this is different from a "blank" Qt app that has a brownish background that seems to be default theme color. How can I get my program to output the webcam video to the Qt app?
One interesting data point is that macOS does NOT show the "camera in use" icon in the toolbar with this program.
Black background with example code:
Default app with nothing inside:
import sys from PySide6.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget from PySide6.QtCore import QSize, Qt, QUrl, Slot from PySide6.QtMultimedia import QMediaPlayer from PySide6.QtMultimediaWidgets import QVideoWidget GST_PIPELINE = ('avfvideosrc device-index=1 ! ' '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! ' 'rawvideoparse width=1280 height=720 format=yuy2 ! ' 'queue ! ' 'autovideoconvert ! ' 'qtvideosink') class MainWindow(QMainWindow): def __init__(self): super().__init__() self._player = QMediaPlayer() self._video_widget = QVideoWidget() self.setCentralWidget(self._video_widget) self._player.setVideoOutput(self._video_widget) def closeEvent(self, event): self._ensure_stopped() event.accept() @Slot() def _ensure_stopped(self): if self._player.playbackState() != QMediaPlayer.StoppedState: self._player.stop() def _play(self): self._player.setSource(QUrl(f'gst-pipeline: {GST_PIPELINE}')) self._player.play() if __name__ == '__main__': app = QApplication(sys.argv) window = MainWindow() window.show() window._play() sys.exit(app.exec())@HanSooloo
FFmpeg as the default backend from here:
https://doc.qt.io/qt-6/qtmultimedia-index.htmlYou need to change backend to gstreaner to run your pipeline
export QT_MEDIA_BACKEND=gstreamerOn iOS, I am not sure.
Check if the qml6glsink is available since it may not be built in the gstreamer installed.
run command:
gst-inspect-1.0 | grep qml
to see if qml6 sink is availableIf yes, do
export GST_DEBUG=5
to see what is wrong with gstreamer -
@HanSooloo
FFmpeg as the default backend from here:
https://doc.qt.io/qt-6/qtmultimedia-index.htmlYou need to change backend to gstreaner to run your pipeline
export QT_MEDIA_BACKEND=gstreamerOn iOS, I am not sure.
Check if the qml6glsink is available since it may not be built in the gstreamer installed.
run command:
gst-inspect-1.0 | grep qml
to see if qml6 sink is availableIf yes, do
export GST_DEBUG=5
to see what is wrong with gstreamer@JoeCFD said in Simple app to show webcam output via GStreamer pipeline:
@HanSooloo
FFmpeg as the default backend from here:
https://doc.qt.io/qt-6/qtmultimedia-index.htmlYou need to change backend to gstreaner to run your pipeline
export QT_MEDIA_BACKEND=gstreamerOn iOS, I am not sure.
Check if the qml6glsink is available since it may not be built in the gstreamer installed.
run command:
gst-inspect-1.0 | grep qml
to see if qml6 sink is availableIf yes, do
export GST_DEBUG=5
to see what is wrong with gstreamergst-inspect-1.0doesn't show anything related toqml... Do I need to install GStreamer with QML support then?Is there a documented way to do this via Homebrew on macOS?
-
@JoeCFD said in Simple app to show webcam output via GStreamer pipeline:
@HanSooloo
FFmpeg as the default backend from here:
https://doc.qt.io/qt-6/qtmultimedia-index.htmlYou need to change backend to gstreaner to run your pipeline
export QT_MEDIA_BACKEND=gstreamerOn iOS, I am not sure.
Check if the qml6glsink is available since it may not be built in the gstreamer installed.
run command:
gst-inspect-1.0 | grep qml
to see if qml6 sink is availableIf yes, do
export GST_DEBUG=5
to see what is wrong with gstreamergst-inspect-1.0doesn't show anything related toqml... Do I need to install GStreamer with QML support then?Is there a documented way to do this via Homebrew on macOS?
@HanSooloo install gstreamer good plugin which has qml gl sink. Unluckily, there is not much doc for you to refer to.
-
@HanSooloo install gstreamer good plugin which has qml gl sink. Unluckily, there is not much doc for you to refer to.
@JoeCFD I configured the GStreamer source using the
meson configure -Dqt6=enabled builddirconfiguration option and built with regularninjacommand, and finally installed viameson install.I was able to confirm Qt6 support with the following:
$ gst-inspect-1.0 | grep qml qml6: qml6glmixer: Qt6 Video Mixer qml6: qml6gloverlay: Qt Video Overlay qml6: qml6glsink: Qt6 Video Sink qml6: qml6glsrc: Qt Video SourceAfter updating my pipeline as follows, I am still getting the black screen in the window. Any thoughts on where to go from here?
GST_PIPELINE = ('avfvideosrc device-index=1 ! ' '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! ' 'rawvideoparse width=1280 height=720 format=yuy2 ! ' 'queue ! ' 'autovideoconvert ! ' 'qml6glsink') -
@JoeCFD I configured the GStreamer source using the
meson configure -Dqt6=enabled builddirconfiguration option and built with regularninjacommand, and finally installed viameson install.I was able to confirm Qt6 support with the following:
$ gst-inspect-1.0 | grep qml qml6: qml6glmixer: Qt6 Video Mixer qml6: qml6gloverlay: Qt Video Overlay qml6: qml6glsink: Qt6 Video Sink qml6: qml6glsrc: Qt Video SourceAfter updating my pipeline as follows, I am still getting the black screen in the window. Any thoughts on where to go from here?
GST_PIPELINE = ('avfvideosrc device-index=1 ! ' '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! ' 'rawvideoparse width=1280 height=720 format=yuy2 ! ' 'queue ! ' 'autovideoconvert ! ' 'qml6glsink')@HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.
You can check this example out:
https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
BTW, QMultimedia module does not offer enough controls over video play. -
@HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.
You can check this example out:
https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
BTW, QMultimedia module does not offer enough controls over video play.@JoeCFD said in Simple app to show webcam output via GStreamer pipeline:
@HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.
You can check this example out:
https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
BTW, QMultimedia module does not offer enough controls over video play.The pipeline with
autovideosinkworks on the command line. However, the one withqml6glsinkdoesn't work, since it is expecting a QGuiApplication context, e.g.,(pipeline modified to a super simple video test pattern for brevity)
$ gst-launch-1.0 -v -e videotestsrc ! autovideoconvert ! qml6glsink Setting pipeline to PAUSED ... ERROR: from element /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0: Failed to connect to Qt Additional debug info: ../subprojects/gst-plugins-good/ext/qt6/gstqml6glsink.cc(353): gst_qml6_gl_sink_change_state (): /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0: Could not retrieve QGuiApplication instance ERROR: pipeline doesn't want to preroll. Failed to set pipeline to PAUSED. Setting pipeline to NULL ... Freeing pipeline ... -
@JoeCFD said in Simple app to show webcam output via GStreamer pipeline:
@HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.
You can check this example out:
https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
BTW, QMultimedia module does not offer enough controls over video play.The pipeline with
autovideosinkworks on the command line. However, the one withqml6glsinkdoesn't work, since it is expecting a QGuiApplication context, e.g.,(pipeline modified to a super simple video test pattern for brevity)
$ gst-launch-1.0 -v -e videotestsrc ! autovideoconvert ! qml6glsink Setting pipeline to PAUSED ... ERROR: from element /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0: Failed to connect to Qt Additional debug info: ../subprojects/gst-plugins-good/ext/qt6/gstqml6glsink.cc(353): gst_qml6_gl_sink_change_state (): /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0: Could not retrieve QGuiApplication instance ERROR: pipeline doesn't want to preroll. Failed to set pipeline to PAUSED. Setting pipeline to NULL ... Freeing pipeline ...@HanSooloo you can not do it. From command line, gstreamer has nothing to do with Qt.
gst-launch-1.0 -v -e videotestsrc ! autovideoconvert ! autovideosink
qml6glsink is attached to a qt widget or qml item. Check the example out. -
@HanSooloo you can not do it. From command line, gstreamer has nothing to do with Qt.
gst-launch-1.0 -v -e videotestsrc ! autovideoconvert ! autovideosink
qml6glsink is attached to a qt widget or qml item. Check the example out.@JoeCFD as you suggested, understand that Qt6 with GStreamer backend on macOS is not an option.
So, decided to switch back to Qt5 just to validate a basic GStreamer pipeline. Even with the below code that uses a simple video test source, I am still getting a black window content.
Any thoughts on where to go from here?
import sys from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget from PyQt5.QtCore import QSize, Qt, QUrl from PyQt5.QtMultimedia import QMediaPlayer, QMediaContent from PyQt5.QtMultimediaWidgets import QVideoWidget GST_PIPELINE = 'videotestsrc ! autovideoconvert ! autovideosink' class MainWindow(QMainWindow): def __init__(self): super().__init__() self._player = QMediaPlayer() self._video_widget = QVideoWidget() self.setCentralWidget(self._video_widget) self._player.setVideoOutput(self._video_widget) def _play(self): self._player.setMedia(QMediaContent(QUrl(f'gst-pipeline: {GST_PIPELINE}'))) self._player.play() if __name__ == '__main__': app = QApplication(sys.argv) window = MainWindow() window.show() window._play() sys.exit(app.exec()) -
@JoeCFD as you suggested, understand that Qt6 with GStreamer backend on macOS is not an option.
So, decided to switch back to Qt5 just to validate a basic GStreamer pipeline. Even with the below code that uses a simple video test source, I am still getting a black window content.
Any thoughts on where to go from here?
import sys from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget from PyQt5.QtCore import QSize, Qt, QUrl from PyQt5.QtMultimedia import QMediaPlayer, QMediaContent from PyQt5.QtMultimediaWidgets import QVideoWidget GST_PIPELINE = 'videotestsrc ! autovideoconvert ! autovideosink' class MainWindow(QMainWindow): def __init__(self): super().__init__() self._player = QMediaPlayer() self._video_widget = QVideoWidget() self.setCentralWidget(self._video_widget) self._player.setVideoOutput(self._video_widget) def _play(self): self._player.setMedia(QMediaContent(QUrl(f'gst-pipeline: {GST_PIPELINE}'))) self._player.play() if __name__ == '__main__': app = QApplication(sys.argv) window = MainWindow() window.show() window._play() sys.exit(app.exec()) -
J JoeCFD referenced this topic on
-
@JoeCFD I configured the GStreamer source using the
meson configure -Dqt6=enabled builddirconfiguration option and built with regularninjacommand, and finally installed viameson install.I was able to confirm Qt6 support with the following:
$ gst-inspect-1.0 | grep qml qml6: qml6glmixer: Qt6 Video Mixer qml6: qml6gloverlay: Qt Video Overlay qml6: qml6glsink: Qt6 Video Sink qml6: qml6glsrc: Qt Video SourceAfter updating my pipeline as follows, I am still getting the black screen in the window. Any thoughts on where to go from here?
GST_PIPELINE = ('avfvideosrc device-index=1 ! ' '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! ' 'rawvideoparse width=1280 height=720 format=yuy2 ! ' 'queue ! ' 'autovideoconvert ! ' 'qml6glsink')@HanSooloo said in Simple app to show webcam output via GStreamer pipeline:
qml6glsrc
Hi,
Any idea on how to use qml6glsrc?
Since QT6 Multimedia does not support gstreamer I don't know how qml6glsrc is supposed to work. -
@HanSooloo said in Simple app to show webcam output via GStreamer pipeline:
qml6glsrc
Hi,
Any idea on how to use qml6glsrc?
Since QT6 Multimedia does not support gstreamer I don't know how qml6glsrc is supposed to work. -
@malikcis some made an example here.
https://forum.qt.io/topic/155422/parallel-pipelines-with-qml6glsink?_=1712071475549
