Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. Qt for Python
  4. Simple app to show webcam output via GStreamer pipeline
Forum Updated to NodeBB v4.3 + New Features

Simple app to show webcam output via GStreamer pipeline

Scheduled Pinned Locked Moved Unsolved Qt for Python
13 Posts 3 Posters 3.2k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • H Offline
    H Offline
    HanSooloo
    wrote on last edited by
    #1

    I am trying to build a simple app on macOS that can use a GStreamer pipeline to open the webcam as a video source and output that to the Qt app.

    I have looked at a few examples to draw inspiration:

    1. https://doc.qt.io/qtforpython-6/examples/example_multimedia_player.html
    2. https://doc.qt.io/qt-5/qmediaplayer.html#setMedia (specifically the gst-pipeline example)

    I have the following program, but the output is a black content inside the app window .. this is different from a "blank" Qt app that has a brownish background that seems to be default theme color. How can I get my program to output the webcam video to the Qt app?

    One interesting data point is that macOS does NOT show the "camera in use" icon in the toolbar with this program.

    Black background with example code:
    627e7c0e-03b0-496b-83d9-c87589ebe260-image.png

    Default app with nothing inside:
    ecf6e070-de0c-483e-b9c2-44de01a8af7d-image.png

    import sys
    
    from PySide6.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget
    from PySide6.QtCore import QSize, Qt, QUrl, Slot
    from PySide6.QtMultimedia import QMediaPlayer
    from PySide6.QtMultimediaWidgets import QVideoWidget
    
    GST_PIPELINE = ('avfvideosrc device-index=1 ! '
                    '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! '
                    'rawvideoparse width=1280 height=720 format=yuy2 ! '
                    'queue ! '
                    'autovideoconvert ! '
                    'qtvideosink')
    
    class MainWindow(QMainWindow):
      def __init__(self):
        super().__init__()
        self._player = QMediaPlayer()
        self._video_widget = QVideoWidget()
        self.setCentralWidget(self._video_widget)
        self._player.setVideoOutput(self._video_widget)
      
      def closeEvent(self, event):
        self._ensure_stopped()
        event.accept()
        
      @Slot()
      def _ensure_stopped(self):
          if self._player.playbackState() != QMediaPlayer.StoppedState:
              self._player.stop()
    
      def _play(self):
        self._player.setSource(QUrl(f'gst-pipeline: {GST_PIPELINE}'))
        self._player.play()
    
    
    if __name__ == '__main__':
      app = QApplication(sys.argv)
      window = MainWindow()
      window.show()
      window._play()
      sys.exit(app.exec())
    
    JoeCFDJ 1 Reply Last reply
    0
    • H HanSooloo

      I am trying to build a simple app on macOS that can use a GStreamer pipeline to open the webcam as a video source and output that to the Qt app.

      I have looked at a few examples to draw inspiration:

      1. https://doc.qt.io/qtforpython-6/examples/example_multimedia_player.html
      2. https://doc.qt.io/qt-5/qmediaplayer.html#setMedia (specifically the gst-pipeline example)

      I have the following program, but the output is a black content inside the app window .. this is different from a "blank" Qt app that has a brownish background that seems to be default theme color. How can I get my program to output the webcam video to the Qt app?

      One interesting data point is that macOS does NOT show the "camera in use" icon in the toolbar with this program.

      Black background with example code:
      627e7c0e-03b0-496b-83d9-c87589ebe260-image.png

      Default app with nothing inside:
      ecf6e070-de0c-483e-b9c2-44de01a8af7d-image.png

      import sys
      
      from PySide6.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget
      from PySide6.QtCore import QSize, Qt, QUrl, Slot
      from PySide6.QtMultimedia import QMediaPlayer
      from PySide6.QtMultimediaWidgets import QVideoWidget
      
      GST_PIPELINE = ('avfvideosrc device-index=1 ! '
                      '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! '
                      'rawvideoparse width=1280 height=720 format=yuy2 ! '
                      'queue ! '
                      'autovideoconvert ! '
                      'qtvideosink')
      
      class MainWindow(QMainWindow):
        def __init__(self):
          super().__init__()
          self._player = QMediaPlayer()
          self._video_widget = QVideoWidget()
          self.setCentralWidget(self._video_widget)
          self._player.setVideoOutput(self._video_widget)
        
        def closeEvent(self, event):
          self._ensure_stopped()
          event.accept()
          
        @Slot()
        def _ensure_stopped(self):
            if self._player.playbackState() != QMediaPlayer.StoppedState:
                self._player.stop()
      
        def _play(self):
          self._player.setSource(QUrl(f'gst-pipeline: {GST_PIPELINE}'))
          self._player.play()
      
      
      if __name__ == '__main__':
        app = QApplication(sys.argv)
        window = MainWindow()
        window.show()
        window._play()
        sys.exit(app.exec())
      
      JoeCFDJ Offline
      JoeCFDJ Offline
      JoeCFD
      wrote on last edited by JoeCFD
      #2

      @HanSooloo
      FFmpeg as the default backend from here:
      https://doc.qt.io/qt-6/qtmultimedia-index.html

      You need to change backend to gstreaner to run your pipeline
      export QT_MEDIA_BACKEND=gstreamer

      On iOS, I am not sure.

      Check if the qml6glsink is available since it may not be built in the gstreamer installed.
      run command:
      gst-inspect-1.0 | grep qml
      to see if qml6 sink is available

      If yes, do
      export GST_DEBUG=5
      to see what is wrong with gstreamer

      H 1 Reply Last reply
      0
      • JoeCFDJ JoeCFD

        @HanSooloo
        FFmpeg as the default backend from here:
        https://doc.qt.io/qt-6/qtmultimedia-index.html

        You need to change backend to gstreaner to run your pipeline
        export QT_MEDIA_BACKEND=gstreamer

        On iOS, I am not sure.

        Check if the qml6glsink is available since it may not be built in the gstreamer installed.
        run command:
        gst-inspect-1.0 | grep qml
        to see if qml6 sink is available

        If yes, do
        export GST_DEBUG=5
        to see what is wrong with gstreamer

        H Offline
        H Offline
        HanSooloo
        wrote on last edited by HanSooloo
        #3

        @JoeCFD said in Simple app to show webcam output via GStreamer pipeline:

        @HanSooloo
        FFmpeg as the default backend from here:
        https://doc.qt.io/qt-6/qtmultimedia-index.html

        You need to change backend to gstreaner to run your pipeline
        export QT_MEDIA_BACKEND=gstreamer

        On iOS, I am not sure.

        Check if the qml6glsink is available since it may not be built in the gstreamer installed.
        run command:
        gst-inspect-1.0 | grep qml
        to see if qml6 sink is available

        If yes, do
        export GST_DEBUG=5
        to see what is wrong with gstreamer

        gst-inspect-1.0 doesn't show anything related to qml ... Do I need to install GStreamer with QML support then?

        Is there a documented way to do this via Homebrew on macOS?

        JoeCFDJ 1 Reply Last reply
        0
        • H HanSooloo

          @JoeCFD said in Simple app to show webcam output via GStreamer pipeline:

          @HanSooloo
          FFmpeg as the default backend from here:
          https://doc.qt.io/qt-6/qtmultimedia-index.html

          You need to change backend to gstreaner to run your pipeline
          export QT_MEDIA_BACKEND=gstreamer

          On iOS, I am not sure.

          Check if the qml6glsink is available since it may not be built in the gstreamer installed.
          run command:
          gst-inspect-1.0 | grep qml
          to see if qml6 sink is available

          If yes, do
          export GST_DEBUG=5
          to see what is wrong with gstreamer

          gst-inspect-1.0 doesn't show anything related to qml ... Do I need to install GStreamer with QML support then?

          Is there a documented way to do this via Homebrew on macOS?

          JoeCFDJ Offline
          JoeCFDJ Offline
          JoeCFD
          wrote on last edited by JoeCFD
          #4

          @HanSooloo install gstreamer good plugin which has qml gl sink. Unluckily, there is not much doc for you to refer to.

          H 1 Reply Last reply
          0
          • JoeCFDJ JoeCFD

            @HanSooloo install gstreamer good plugin which has qml gl sink. Unluckily, there is not much doc for you to refer to.

            H Offline
            H Offline
            HanSooloo
            wrote on last edited by
            #5

            @JoeCFD I configured the GStreamer source using the meson configure -Dqt6=enabled builddir configuration option and built with regular ninja command, and finally installed via meson install.

            I was able to confirm Qt6 support with the following:

            $ gst-inspect-1.0 | grep qml
            qml6:  qml6glmixer: Qt6 Video Mixer
            qml6:  qml6gloverlay: Qt Video Overlay
            qml6:  qml6glsink: Qt6 Video Sink
            qml6:  qml6glsrc: Qt Video Source
            

            After updating my pipeline as follows, I am still getting the black screen in the window. Any thoughts on where to go from here?

            GST_PIPELINE = ('avfvideosrc device-index=1 ! '
                            '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! '
                            'rawvideoparse width=1280 height=720 format=yuy2 ! '
                            'queue ! '
                            'autovideoconvert ! '
                            'qml6glsink')
            
            JoeCFDJ malikcisM 2 Replies Last reply
            0
            • H HanSooloo

              @JoeCFD I configured the GStreamer source using the meson configure -Dqt6=enabled builddir configuration option and built with regular ninja command, and finally installed via meson install.

              I was able to confirm Qt6 support with the following:

              $ gst-inspect-1.0 | grep qml
              qml6:  qml6glmixer: Qt6 Video Mixer
              qml6:  qml6gloverlay: Qt Video Overlay
              qml6:  qml6glsink: Qt6 Video Sink
              qml6:  qml6glsrc: Qt Video Source
              

              After updating my pipeline as follows, I am still getting the black screen in the window. Any thoughts on where to go from here?

              GST_PIPELINE = ('avfvideosrc device-index=1 ! '
                              '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! '
                              'rawvideoparse width=1280 height=720 format=yuy2 ! '
                              'queue ! '
                              'autovideoconvert ! '
                              'qml6glsink')
              
              JoeCFDJ Offline
              JoeCFDJ Offline
              JoeCFD
              wrote on last edited by JoeCFD
              #6

              @HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.

              You can check this example out:
              https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
              BTW, QMultimedia module does not offer enough controls over video play.

              H 1 Reply Last reply
              0
              • JoeCFDJ JoeCFD

                @HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.

                You can check this example out:
                https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
                BTW, QMultimedia module does not offer enough controls over video play.

                H Offline
                H Offline
                HanSooloo
                wrote on last edited by
                #7

                @JoeCFD said in Simple app to show webcam output via GStreamer pipeline:

                @HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.

                You can check this example out:
                https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
                BTW, QMultimedia module does not offer enough controls over video play.

                The pipeline with autovideosink works on the command line. However, the one with qml6glsink doesn't work, since it is expecting a QGuiApplication context, e.g.,

                (pipeline modified to a super simple video test pattern for brevity)

                $ gst-launch-1.0 -v -e  videotestsrc ! autovideoconvert ! qml6glsink
                Setting pipeline to PAUSED ...
                ERROR: from element /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0: Failed to connect to Qt
                Additional debug info:
                ../subprojects/gst-plugins-good/ext/qt6/gstqml6glsink.cc(353): gst_qml6_gl_sink_change_state (): /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0:
                Could not retrieve QGuiApplication instance
                ERROR: pipeline doesn't want to preroll.
                Failed to set pipeline to PAUSED.
                Setting pipeline to NULL ...
                Freeing pipeline ...
                
                JoeCFDJ 1 Reply Last reply
                0
                • H HanSooloo

                  @JoeCFD said in Simple app to show webcam output via GStreamer pipeline:

                  @HanSooloo Does your pipeline run from command line? Wait, Qt6 may have removed support for running gstreamer pipeline in Qt. You may need to find it out. If my memory serves me right, Qt5 supports it. I do not use QMultimedia module for video display. Instead, I use raw gstreamer code to do it.

                  You can check this example out:
                  https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt/qmlsink
                  BTW, QMultimedia module does not offer enough controls over video play.

                  The pipeline with autovideosink works on the command line. However, the one with qml6glsink doesn't work, since it is expecting a QGuiApplication context, e.g.,

                  (pipeline modified to a super simple video test pattern for brevity)

                  $ gst-launch-1.0 -v -e  videotestsrc ! autovideoconvert ! qml6glsink
                  Setting pipeline to PAUSED ...
                  ERROR: from element /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0: Failed to connect to Qt
                  Additional debug info:
                  ../subprojects/gst-plugins-good/ext/qt6/gstqml6glsink.cc(353): gst_qml6_gl_sink_change_state (): /GstPipeline:pipeline0/GstQml6GLSink:qml6glsink0:
                  Could not retrieve QGuiApplication instance
                  ERROR: pipeline doesn't want to preroll.
                  Failed to set pipeline to PAUSED.
                  Setting pipeline to NULL ...
                  Freeing pipeline ...
                  
                  JoeCFDJ Offline
                  JoeCFDJ Offline
                  JoeCFD
                  wrote on last edited by
                  #8

                  @HanSooloo you can not do it. From command line, gstreamer has nothing to do with Qt.
                  gst-launch-1.0 -v -e videotestsrc ! autovideoconvert ! autovideosink
                  qml6glsink is attached to a qt widget or qml item. Check the example out.

                  H 1 Reply Last reply
                  0
                  • JoeCFDJ JoeCFD

                    @HanSooloo you can not do it. From command line, gstreamer has nothing to do with Qt.
                    gst-launch-1.0 -v -e videotestsrc ! autovideoconvert ! autovideosink
                    qml6glsink is attached to a qt widget or qml item. Check the example out.

                    H Offline
                    H Offline
                    HanSooloo
                    wrote on last edited by
                    #9

                    @JoeCFD as you suggested, understand that Qt6 with GStreamer backend on macOS is not an option.

                    So, decided to switch back to Qt5 just to validate a basic GStreamer pipeline. Even with the below code that uses a simple video test source, I am still getting a black window content.

                    Any thoughts on where to go from here?

                    import sys
                    
                    from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget
                    from PyQt5.QtCore import QSize, Qt, QUrl
                    from PyQt5.QtMultimedia import QMediaPlayer, QMediaContent
                    from PyQt5.QtMultimediaWidgets import QVideoWidget
                    
                    GST_PIPELINE = 'videotestsrc ! autovideoconvert ! autovideosink'
                    
                    
                    class MainWindow(QMainWindow):
                      def __init__(self):
                        super().__init__()
                        self._player = QMediaPlayer()
                        self._video_widget = QVideoWidget()
                        self.setCentralWidget(self._video_widget)
                        self._player.setVideoOutput(self._video_widget)
                    
                      def _play(self):
                        self._player.setMedia(QMediaContent(QUrl(f'gst-pipeline: {GST_PIPELINE}')))
                        self._player.play()
                    
                    
                    if __name__ == '__main__':
                      app = QApplication(sys.argv)
                    
                      window = MainWindow()
                      window.show()
                      window._play()
                    
                      sys.exit(app.exec())
                    
                    JoeCFDJ 1 Reply Last reply
                    0
                    • H HanSooloo

                      @JoeCFD as you suggested, understand that Qt6 with GStreamer backend on macOS is not an option.

                      So, decided to switch back to Qt5 just to validate a basic GStreamer pipeline. Even with the below code that uses a simple video test source, I am still getting a black window content.

                      Any thoughts on where to go from here?

                      import sys
                      
                      from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton, QWidget
                      from PyQt5.QtCore import QSize, Qt, QUrl
                      from PyQt5.QtMultimedia import QMediaPlayer, QMediaContent
                      from PyQt5.QtMultimediaWidgets import QVideoWidget
                      
                      GST_PIPELINE = 'videotestsrc ! autovideoconvert ! autovideosink'
                      
                      
                      class MainWindow(QMainWindow):
                        def __init__(self):
                          super().__init__()
                          self._player = QMediaPlayer()
                          self._video_widget = QVideoWidget()
                          self.setCentralWidget(self._video_widget)
                          self._player.setVideoOutput(self._video_widget)
                      
                        def _play(self):
                          self._player.setMedia(QMediaContent(QUrl(f'gst-pipeline: {GST_PIPELINE}')))
                          self._player.play()
                      
                      
                      if __name__ == '__main__':
                        app = QApplication(sys.argv)
                      
                        window = MainWindow()
                        window.show()
                        window._play()
                      
                        sys.exit(app.exec())
                      
                      JoeCFDJ Offline
                      JoeCFDJ Offline
                      JoeCFD
                      wrote on last edited by
                      #10

                      @HanSooloo https://forum.qt.io/topic/141301/understanding-qt-documentation-about-using-qtmultimedia-with-lgpl-gstreamer-on-android/8

                      the sink is qtvideosink

                      1 Reply Last reply
                      0
                      • JoeCFDJ JoeCFD referenced this topic on
                      • H HanSooloo

                        @JoeCFD I configured the GStreamer source using the meson configure -Dqt6=enabled builddir configuration option and built with regular ninja command, and finally installed via meson install.

                        I was able to confirm Qt6 support with the following:

                        $ gst-inspect-1.0 | grep qml
                        qml6:  qml6glmixer: Qt6 Video Mixer
                        qml6:  qml6gloverlay: Qt Video Overlay
                        qml6:  qml6glsink: Qt6 Video Sink
                        qml6:  qml6glsrc: Qt Video Source
                        

                        After updating my pipeline as follows, I am still getting the black screen in the window. Any thoughts on where to go from here?

                        GST_PIPELINE = ('avfvideosrc device-index=1 ! '
                                        '"video/x-raw, width=1280, height=720, format=(string)YUY2, texture-target=rectangle" ! '
                                        'rawvideoparse width=1280 height=720 format=yuy2 ! '
                                        'queue ! '
                                        'autovideoconvert ! '
                                        'qml6glsink')
                        
                        malikcisM Offline
                        malikcisM Offline
                        malikcis
                        wrote on last edited by
                        #11

                        @HanSooloo said in Simple app to show webcam output via GStreamer pipeline:

                        qml6glsrc

                        Hi,
                        Any idea on how to use qml6glsrc?
                        Since QT6 Multimedia does not support gstreamer I don't know how qml6glsrc is supposed to work.

                        JoeCFDJ 1 Reply Last reply
                        0
                        • malikcisM malikcis

                          @HanSooloo said in Simple app to show webcam output via GStreamer pipeline:

                          qml6glsrc

                          Hi,
                          Any idea on how to use qml6glsrc?
                          Since QT6 Multimedia does not support gstreamer I don't know how qml6glsrc is supposed to work.

                          JoeCFDJ Offline
                          JoeCFDJ Offline
                          JoeCFD
                          wrote on last edited by
                          #12

                          @malikcis some made an example here.
                          https://forum.qt.io/topic/155422/parallel-pipelines-with-qml6glsink?_=1712071475549

                          JoeCFDJ 1 Reply Last reply
                          0
                          • JoeCFDJ JoeCFD

                            @malikcis some made an example here.
                            https://forum.qt.io/topic/155422/parallel-pipelines-with-qml6glsink?_=1712071475549

                            JoeCFDJ Offline
                            JoeCFDJ Offline
                            JoeCFD
                            wrote on last edited by
                            #13

                            @JoeCFD gstreamer 1.24 build includes qml6glsink automatically if qt6 is installed in Ubuntu. No extra settings is needed in the build of gstreamer 1.24.

                            1 Reply Last reply
                            0

                            • Login

                            • Login or register to search.
                            • First post
                              Last post
                            0
                            • Categories
                            • Recent
                            • Tags
                            • Popular
                            • Users
                            • Groups
                            • Search
                            • Get Qt Extensions
                            • Unsolved