Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Pyqt5 and Gstreamer, how to setup layout and draw over the video.



  • Hi, I'm quite new to qt, but not to code. I need to play a Gstreamer pipeline in a window, with some buttons around and the possibility to draw and write text on the video (an ip camera stream).

    what I have working is just playing the video inside a window, but since the video is linked to the window by a WindowId, I cannot put it inside a layout to show other gui widgets.
    Any suggestion on how to solve that?

    my code:

    import sys
    import gi
    gi.require_version('Gst', '1.0')
    gi.require_version('GstVideo', '1.0')
    from gi.repository import GObject, Gst, GstVideo
    from PyQt5.QtWidgets import *
    
    Gst.init(sys.argv)
    
    
    class MainWindow(QMainWindow):
        def __init__(self, parent = None):
            super(MainWindow, self).__init__(parent)
    
            self.pipeline = Gst.parse_launch(
                'videotestsrc name=source ! videoconvert name = convert ! xvimagesink name=sink')  # xvimagesink, ximagesink
            self.source = self.pipeline.get_by_name("source")
            self.videoconvert = self.pipeline.get_by_name("convert")
            self.sink = self.pipeline.get_by_name("sink")
    
            self.display = QWidget()
    
            self.setGeometry(100, 100, 640, 480)
            self.setWindowTitle("Prova_gst_qt5")
    
            self.windowId = self.winId()
    
        def setup_pipeline(self):
    
            self.state = Gst.State.NULL
            self.source.set_property('pattern', 0)
    
            if not self.pipeline or not self.source or not self.videoconvert or not self.sink:
                print("ERROR: Not all elements could be created")
                sys.exit(1)
    
            # instruct the bus to emit signals for each received message
            # and connect to the interesting signals
            bus = self.pipeline.get_bus()
    
            bus.add_signal_watch()
            bus.enable_sync_message_emission()
            bus.connect('sync-message::element', self.on_sync_message)
    
        def on_sync_message(self, bus, msg):
            if msg.get_structure().get_name() == 'prepare-window-handle':
                msg.src.set_window_handle(self.windowId)
    
        def start_pipeline(self):
            self.pipeline.set_state(Gst.State.PLAYING)
    
    
    if __name__ == '__main__':
        app = QApplication(sys.argv)
        window = MainWindow()
        window.setup_pipeline()
        window.start_pipeline()
        window.show()
        sys.exit(app.exec_())
    
    

  • Lifetime Qt Champion

    Hi and welcome to devnet,

    If you are using a recent enough version of Qt, you might be able to simplify your code using a custom pipeline like describe here.


  • Banned

    @spiderdab while I could not test this to make sure it works as I do not have the gi library I have restructured your code to perhaps allow for it and/or render what you were trying to do. If it does not work it will at least give you a leg up on doing what you need done if you need help troubleshooting it I can teach you how to effectively trouble shoot this so that you can find out how to implement what you are trying to implement as well.

    # This should be declared first as it can affect how the other 
    # imports are referenced
    from PyQt5.QtWidgets import *
     
    import sys
    import gi
    gi.require_version('Gst', '1.0')
    gi.require_version('GstVideo', '1.0')
    from gi.repository import GObject, Gst, GstVideo
    
    # I am thinking is wrong but have to admit I do not know much about
    # it. Still global anything is generally a bad thing
    Gst.init(sys.argv)
    
    class GstDisplay(QWidget):
        def __init__(self, parent):
            QWidget.__init__(self)
            self.CntrPne = parent
    
            self.pipeline = Gst.parse_launch(
                'videotestsrc name=source ! videoconvert name = convert ! xvimagesink name=sink')  # xvimagesink, ximagesink
            self.source = self.pipeline.get_by_name("source")
            self.videoconvert = self.pipeline.get_by_name("convert")
            self.sink = self.pipeline.get_by_name("sink")
    
            self.WinId = self.winId()
            self.setup_pipeline()
            self.start_pipeline()
    
        def setup_pipeline(self):
            self.state = Gst.State.NULL
            self.source.set_property('pattern', 0)
    
            if not self.pipeline or not self.source or not self.videoconvert or not self.sink:
                print("ERROR: Not all elements could be created")
                sys.exit(1)
    
            # instruct the bus to emit signals for each received message
            # and connect to the interesting signals
            bus = self.pipeline.get_bus()
    
            bus.add_signal_watch()
            bus.enable_sync_message_emission()
            bus.connect('sync-message::element', self.on_sync_message)
    
        def on_sync_message(self, bus, msg):
            if msg.get_structure().get_name() == 'prepare-window-handle':
                msg.src.set_window_handle(self.windowId)
    
        def start_pipeline(self):
            self.pipeline.set_state(Gst.State.PLAYING)
    
    class CentralPanel(QWidget):
        def __init__(self, parent):
            QWidget.__init__(self)
            self.MainWin = parent
    
            self.btnPush = QPushButton('Pusher')
            self.btnPush.clicked.connect(self.Pushed)
            
            HBox = QHBoxLayout()
            HBox.addWidget(self.btnPush).
            HBox.addStretch(1)
    
            self.GstDsply = GstDisplay()
    
            VBox = QVBoxLayout()
            VBox.addWidget(self.GstDsply)
            VBox.addLayout(HBox)
            
            self.setLayout(HBox)
    
        @Slot()
        def Pushed(self):
            print('Quit pushing me down.')
    
    class MainWindow(QMainWindow):
    # Do not use Parent unless you are actually going to possibly 
    # supply it and due to the nature of where you are calling this
    # from that would never happen
    #    def __init__(self, parent = None):
        def __init__(self):
    # Do not use super( ) unless you fully understand the 3 issues it
    # creates that you must protect against within your code further
    # unless you need it for the very rare case it was created to fix
    # then you are adding more issues to your program than you are 
    # removing
    #        super(MainWindow, self).__init__(parent)
            QMainWindow.__init__(self)
            self.setWindowTitle("Prova Gst Qt5")
            Top = 100; Left = 100; Wdth = 640; Hght = 480
            self.setGeometry(Left, Top, Wdth, Hght)
            
            self.CenterPane = CentralPanel()
            self.setCentralWidget(self.CenterPane)
    
    if __name__ == '__main__':
    # If you are not going to use Command Line arguments then do not code 
    # for them but IF you are then looking into the argparser library as 
    # it handles Command Line arguments much more cleanly
        MainEvntHndlr = QApplication([])
    
        MainApp = MyForm()
        MainApp.show()
    
    # This is the old PyQt4 way of doing this
    #    sys.exit(app.exec_())
    # Here is the PyQt5 way of doing this
        MainEvntHndlr.exec()
    
      # If anyone wants more extensive free help I run an online lab-like classroom-like 
      # message server feel free and drop by you will not be able to post until I clear 
      # you as a student as this prevents spammers so if interested here is the invite
      # https://discord.gg/3D8huKC
    


  • I've tested the example ad it seems to be working fine.

    Do you, by any chance have similar example for QtQuick (QML) form?


  • Banned

    @yossiovcharik glad to hear this fixed your problem but sorry I am not currently using QtQuick (QML) but some of my informal students might be just shoot me a PM and I can perhaps get you hooked up with them. Otherwise maybe someone else on here is familiar with QML and can speak to this.


Log in to reply