QMediaPlayer inside QGraphicsView with gstreamer

JokZiz1

I have tried to launch a gstreamer pipeline in Python using QGraphicsView with QMediaPlayer.

It works if I use QVideoWidget, however, I am unable to run it with QGraphicsView. I need QGraphicsView, because I need functionality of it, as I will need to be able to draw on it, move around and etc.

Python: 3.10.12
PySide6: 6.6.1
GStreamer: 1.20.3
OS: Ubuntu 22.04

I have tried to use the example from QMediaPlayer documentation page, but it was not helpful. I have tried various sinks, sources and it did not work.

The code runs without throwing any errors, but all I get is empty gray window.

import sys
from PySide6.QtCore import QUrl
from PySide6.QtWidgets import QApplication, QGraphicsView, QGraphicsScene
from PySide6.QtMultimedia import QMediaPlayer, QVideoFrame
from PySide6.QtMultimediaWidgets import QGraphicsVideoItem

app = QApplication(sys.argv)

class VideoPlayer(QGraphicsView):
    def __init__(self):
        super().__init__()
        self.scene = QGraphicsScene(0, 0, 640, 480)
        self.setScene(self.scene)

        self.player = QMediaPlayer()
        self.videoItem = QGraphicsVideoItem()
        self.scene.addItem(self.videoItem)

        self.player.setVideoOutput(self.videoItem)

        self.player.setSource(QUrl("gst-pipeline: videotestsrc ! xvimagesink name=\"qtvideosink\""))
        self.player.play()

    def closeEvent(self, event):
        self.player.stop()
        super().closeEvent(event)

if __name__ == '__main__':
    player = VideoPlayer()
    player.show()
    sys.exit(app.exec())

I believe that this is Gstreamer pipeline issue, because, if I launch my a local video, the display does work:

self.player.setSource(QUrl.fromLocalFile("1.mp4"))

I am not exactly sure what else could I do...

Pl45m4

@JokZiz1

I don't have much knowledge about GStreamer but check this topic here

JokZiz1

@Pl45m4
Qt6 does not have such a thing as videoItem.videoSurface(), therefore, the thread that you have pointed to does not help me.

Downgrading version is out of question for me, I need several components that are available only in Qt6 (even though I have tried your solution in PySide2 and it appears to be working there, but it is of no use for me)

Pl45m4

@JokZiz1

The function was renamed or replaced in Qt6 by

https://doc.qt.io/qt-6/qgraphicsvideoitem.html#videoSink-prop

so try this instead. Might work if your GStreamer is setup correctly.

JokZiz1

@Pl45m4

I have tried the solution you have pointed to and I get the same issue. I just get a gray window.

self.player.setSource(QUrl.fromLocalFile("1.mp4")) works well
self.player.setSource(QUrl("gst-pipeline: videotestsrc ! autovideosink")) does not work

Downgrading to PySide2 does solve the issue, so I am not sure what is the issue, that does sort of imply that GStreamer works well, unless PySide6 and PySide2 requires different gstreamer plugins or something...

As I said however, using PySide2 is not a solution for me.

JoeCFD

@JokZiz1 I guess you can overlay or attach the videowidget on top of or to QGraphicsView. It will work. I did both.

SGaist

@JokZiz1 Hi,

Custom GStreamer pipeline have been dropped in Qt 6. That said, the PySide6 version you have should be using the ffmpeg backend so it should be able to play mp4 files.

JoeCFD

@SGaist right, I forgot that again. But he said: It works if I use QVideoWidget.

SGaist

@JoeCFD @JokZiz1 wrote that it is working for local files.

JokZiz1

@SGaist
I have tested local files to confirm that I have been using widgets appropriately and also in order to try to identify source of the issue.

I dont need local files, my ultimate intention is to use gstreamer pipeline, this was for testing only...

So are you telling me that there is no way for me to use gstreamer pipeline or are there any potential workarounds???

Theoretically I could use OpenCV to do all this, so all of this is not too much of an issue, but I still prefer using gstreamer if possible.... Is there no way?

SGaist

@JokZiz1 AFAIK You have to use the GStreamer API directly and use the qml6sink to show the result on a Qt application

JoeCFD

@JokZiz1 check here.
https://forum.qt.io/topic/154489/simple-app-to-show-webcam-output-via-gstreamer-pipeline/6?_=1708034483864

you can create a qml widget as sink and use gstreamer raw code to run a pipeline if you know gstreamer well.