[PyQt5] Allow QMediaPlayer to read from QBuffer()

jsulm

@Elus said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

Is this a fair interpretation?

Yes.
Did you debug your app actually? Was the file open successfully?

Elus

Hi,

Yes, I re-ran the program and I did not receive any errors. However, I still receive a black screen.

Also, as a sanity check, replacing the call to set_buffer() with a call to set_playlist() function does successfully play the files, so they are not corrupt.

I am a Qt newbie, so if there is a formal way to debug further, I am happy to try.

Thank you.

Edit: Apparently, I'm a new user so I cannot respond more frequently than every 10 minutes since the Qt forum seems to be limiting the rate at which I can post replies. Sorry about that!

Edit2: Odd, seems like the posting limitation went away?

jsulm

@Elus You should connect slots to QMediaPlayer::mediaStatusChanged() and QMediaPlayer::error() signals to see what happens.

Elus

@jsulm

Hi,

Good idea. When I do that and I use the set_playlist() function instead of set_buffer(), I get the following output:

Status: 2
Status: 6
Status: 7
Status: 1

According to this reference: https://doc.qt.io/archives/qt-5.5/qmediaplayer.html#MediaStatus-enum

Status 2 corresponds to LoadingMedia
Status 6 represents BufferedMedia
Status 7 represents EndOfMedia
Status 1 represents NoMedia

When I use set_buffer(), I do not receive any output to stdout. Here's my implementation of connecting the signals, just to make sure I've done things properly:

import sys
import os

from PyQt5.QtWidgets import QMainWindow
from PyQt5.QtCore import QUrl, QFile, QIODevice,  QBuffer
from PyQt5.QtMultimedia import QMediaContent, QMediaPlaylist, QMediaPlayer
from PyQt5.QtMultimediaWidgets import QVideoWidget
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QWidget


class SimplePlayer(QMainWindow):
    """
    Extremely simple video player using QMediaPlayer
    Consists of vertical layout, widget, and a QLabel
    """

    def __init__(self, master=None):
        QMainWindow.__init__(self, master)

        # Define file variables
        self.playlist_files = ['video_file_1.mp4', 'video_file_2.mp4']

        # Define the QT-specific variables we're going to use
        self.vertical_box_layout = QVBoxLayout()
        self.central_widget = QWidget(self)
        self.video_frame = QVideoWidget()

        # Define the media player related information
        self.playlist = QMediaPlaylist()
        self.video_player = QMediaPlayer(flags=QMediaPlayer.VideoSurface)
        self.buffer = QBuffer()

        # Connect error & media status signals to functions that print those signals to stdout
        self.video_player.error.connect(self.print_media_player_error)
        self.video_player.mediaStatusChanged.connect(self.print_media_player_status)

        # Create the user interface, set up the player, and play the 2 videos
        self.create_user_interface()
        self.video_player_setup()

    def print_media_player_error(self, value):
        """Prints any errors media player encounters"""
        print(f"Error: {value}")

    def print_media_player_status(self, value):
        """Prints any status changes to media player"""
        print(f"Status: {value}")

    def video_player_setup(self):
        """Sets media list for the player and then sets output to the video frame"""
        self.video_player.setVideoOutput(self.video_frame)

        self.set_buffer()
        # self.set_playlist()
        self.video_player.play()

    def set_playlist(self):
        """Opens a single video file, puts it into a playlist which is read by the QMediaPlayer"""
        self.playlist.addMedia(QMediaContent(QUrl.fromLocalFile(os.path.abspath(self.playlist_files[0]))))
        self.playlist.setCurrentIndex(0)
        self.video_player.setPlaylist(self.playlist)

    def set_buffer(self):
        """Opens a single video file and writes it to a buffer to be read by QMediaPlayer"""
        media_file_name = os.path.abspath(self.playlist_files[0])
        media_file = QFile(media_file_name)
        media_file.open(QIODevice.ReadOnly)

        self.byte_array = media_file.readAll()
        self.buffer.setData(self.byte_array)
        self.buffer.open(QIODevice.ReadOnly)

        self.video_player.setMedia(QMediaContent(), self.buffer)

    def create_user_interface(self):
        """Create a 1280x720 UI consisting of a vertical layout, central widget, and QLabel"""
        self.setCentralWidget(self.central_widget)
        self.vertical_box_layout.addWidget(self.video_frame)
        self.central_widget.setLayout(self.vertical_box_layout)

        self.resize(1280, 720)


if __name__ == '__main__':
    app = QApplication([])
    player = SimplePlayer()
    player.show()
    sys.exit(app.exec_())

Elus

I also went ahead and did a sanity check on the buffer variable by checking its size before and after I write the byte_array to the buffer. I get the following output to stdout:

The size of buffer before adding the byte_array is: 0
The size of buffer after adding the byte_array is: 3759778

This is the code:

import sys
import os

from PyQt5.QtWidgets import QMainWindow
from PyQt5.QtCore import QUrl, QFile, QIODevice,  QBuffer
from PyQt5.QtMultimedia import QMediaContent, QMediaPlaylist, QMediaPlayer
from PyQt5.QtMultimediaWidgets import QVideoWidget
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QWidget


class SimplePlayer(QMainWindow):
    """
    Extremely simple video player using QMediaPlayer
    Consists of vertical layout, widget, and a QLabel
    """

    def __init__(self, master=None):
        QMainWindow.__init__(self, master)

        # Define file variables
        self.playlist_files = ['video_file_1.mp4', 'video_file_2.mp4']

        # Define the QT-specific variables we're going to use
        self.vertical_box_layout = QVBoxLayout()
        self.central_widget = QWidget(self)
        self.video_frame = QVideoWidget()

        # Define the media player related information
        self.playlist = QMediaPlaylist()
        self.video_player = QMediaPlayer(flags=QMediaPlayer.VideoSurface)
        self.buffer = QBuffer()

        # Connect error & media status signalsto functions that print those signals to stdout
        self.video_player.error.connect(self.print_media_player_error)
        self.video_player.mediaStatusChanged.connect(self.print_media_player_status)

        # Create the user interface, set up the player, and play the 2 videos
        self.create_user_interface()
        self.video_player_setup()

    def print_media_player_error(self, value):
        """Prints any errors media player encounters"""
        print(f"Error: {value}")

    def print_media_player_status(self, value):
        """Prints any status changes to media player"""
        print(f"Status: {value}")

    def video_player_setup(self):
        """Sets media list for the player and then sets output to the video frame"""
        self.video_player.setVideoOutput(self.video_frame)

        self.set_buffer()
        # self.set_playlist()
        self.video_player.play()

    def set_playlist(self):
        """Opens a single video file, puts it into a playlist which is read by the QMediaPlayer"""
        self.playlist.addMedia(QMediaContent(QUrl.fromLocalFile(os.path.abspath(self.playlist_files[0]))))
        self.playlist.setCurrentIndex(0)
        self.video_player.setPlaylist(self.playlist)

    def set_buffer(self):
        """Opens a single video file and writes it to a buffer to be read by QMediaPlayer"""
        media_file_name = os.path.abspath(self.playlist_files[0])
        media_file = QFile(media_file_name)
        media_file.open(QIODevice.ReadOnly)
        print(f"The size of buffer before adding the byte_array is: {self.buffer.size()}")
        self.byte_array = media_file.readAll()
        self.buffer.setData(self.byte_array)
        self.buffer.open(QIODevice.ReadOnly)
        print(f"The size of buffer after adding the byte_array is: {self.buffer.size()}")
        self.video_player.setMedia(QMediaContent(), self.buffer)

    def create_user_interface(self):
        """Create a 1280x720 UI consisting of a vertical layout, central widget, and QLabel"""
        self.setCentralWidget(self.central_widget)
        self.vertical_box_layout.addWidget(self.video_frame)
        self.central_widget.setLayout(self.vertical_box_layout)

        self.resize(1280, 720)


if __name__ == '__main__':
    app = QApplication([])
    player = SimplePlayer()
    player.show()
    sys.exit(app.exec_())

Elus

@jsulm

Is this the cause of the problem? https://bugreports.qt.io/browse/QTBUG-69101

Elus

Bumping... any solutions / suggestions?

Bonnie

@Elus Hi, don't pass a QMediaContent() as in the doc:

Setting the media to a null QMediaContent will cause the player to discard all information relating to the current media source and to cease all I/O operations related to that media.

Try pass a QUrl() to it.
(I'm not sure if you can do this in python, or you can pass a QMediaContent(QUrl()))

Elus

@Bonnie said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

Setting the media to a null QMediaContent will cause the player to discard all information relating to the current media source and to cease all I/O operations related to that media.

I tried:
media_file = QUrl(media_file_name)
But that resulted in exception:
setData(self, Union[QByteArray, bytes, bytearray]): argument 1 has unexpected type 'QUrl'

I also tried:
media_file = QMediaContent(QUrl(media_file_name))
But that resulted in exception:
setData(self, Union[QByteArray, bytes, bytearray]): argument 1 has unexpected type 'QMediaContent'

I can't pass QMediaContent to buffer directly.

Bonnie

@Elus No, no, I mean inset_buffer

self.video_player.setMedia(QUrl(), self.buffer)

Elus

@VaL-Doroshchuk Would you know anything about this?

@Bonnie
One moment let me try

Elus

@Bonnie said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

self.video_player.setMedia(QUrl(), self.buffer)

I tried this and received error:
TypeError: setMedia(self, QMediaContent, stream: QIODevice = None): argument 1 has unexpected type 'QUrl'

Bonnie

@Elus Yes, I said I'm not sure if you can do this in python...
Then can this work? I don't know about python...

self.video_player.setMedia(QMediaContent(QUrl()), self.buffer)

Elus

@Bonnie

Unfortunately, the screen is still blank if I do that. Do you think this is related to my issue? https://bugreports.qt.io/browse/QTBUG-69101

Bonnie

@Elus Not sure, I tested in Windows and C++.

eyllanesc

@Elus I don't think joining 2 videos is as simple as concatenating raw data since it has a header that indicates the format, image size, metadata, etc.

If you still want to display a video using raw data then you can use the following example based on my SO answer:

import os
import sys

from PyQt5 import QtCore, QtWidgets, QtMultimedia, QtMultimediaWidgets

CURRENT_DIR = os.path.dirname(os.path.realpath(__file__))


class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)

        self.player = QtMultimedia.QMediaPlayer(
            flags=QtMultimedia.QMediaPlayer.VideoSurface
        )
        self.buff = QtCore.QBuffer()

        self.video_widget = QtMultimediaWidgets.QVideoWidget()
        self.setCentralWidget(self.video_widget)

        self.player.setVideoOutput(self.video_widget)

        self.resize(640, 480)

    def load_from_file(self, filename):
        f = QtCore.QFile(filename)
        if f.open(QtCore.QIODevice.ReadOnly):
            ba = f.readAll()
            self.load_from_data(ba)

    def load_from_data(self, data):
        ba = QtCore.QByteArray(data)
        self.buff.setData(ba)
        self.buff.open(QtCore.QIODevice.ReadOnly)
        self.player.setMedia(QtMultimedia.QMediaContent(), self.buff)
        self.player.play()


def main():
    app = QtWidgets.QApplication.instance()
    if app is None:
        app = QtWidgets.QApplication(sys.argv)

    w = MainWindow()
    w.show()

    filename = os.path.join(CURRENT_DIR, "video.mp4")
    w.load_from_file(filename)

    sys.exit(app.exec_())


if __name__ == "__main__":
    main()

├── main.py
└── video.mp4

My example has been tested on Linux with PyQt5 5.14.2 and python 3.8.3

Elus

@Bonnie Thanks for giving it a try. I think your experiment, along with @eyllanesc 's code, confirms my suspicions.

@eyllanesc said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

import os
import sys

from PyQt5 import QtCore, QtWidgets, QtMultimedia, QtMultimediaWidgets

CURRENT_DIR = os.path.dirname(os.path.realpath(file))

class MainWindow(QtWidgets.QMainWindow):
def init(self, parent=None):
super().init(parent)

    self.player = QtMultimedia.QMediaPlayer(
        flags=QtMultimedia.QMediaPlayer.VideoSurface
    )
    self.buff = QtCore.QBuffer()

    self.video_widget = QtMultimediaWidgets.QVideoWidget()
    self.setCentralWidget(self.video_widget)

    self.player.setVideoOutput(self.video_widget)

    self.resize(640, 480)

def load_from_file(self, filename):
    f = QtCore.QFile(filename)
    if f.open(QtCore.QIODevice.ReadOnly):
        ba = f.readAll()
        self.load_from_data(ba)

def load_from_data(self, data):
    ba = QtCore.QByteArray(data)
    self.buff.setData(ba)
    self.buff.open(QtCore.QIODevice.ReadOnly)
    self.player.setMedia(QtMultimedia.QMediaContent(), self.buff)
    self.player.play()

def main():
app = QtWidgets.QApplication.instance()
if app is None:
app = QtWidgets.QApplication(sys.argv)

w = MainWindow()
w.show()

filename = os.path.join(CURRENT_DIR, "video.mp4")
w.load_from_file(filename)

sys.exit(app.exec_())

if name == "main":
main()

Thanks a bunch. Your code, which you've tested, confirms my suspicions that https://bugreports.qt.io/browse/QTBUG-69101 is the root cause of my issues and that the underlying problem is Mac OS. Running your code against several different videos produces the same result for me every time: a black screen.

Regarding your comment about the header requirement, I think what you're saying is true of certain video formats. For example, certain compression algorithms are able to reduce the file size by looking at the changes between frames. Therefore, if you simply concatenate one set of bytes to another, as I am trying to do, you'd be missing a critical piece of information that can help you decode the next frame.

However, I do not know if all video formats rely on this kind of paradigm. I believe certain formats, like .ts, allow videos to be concatenated, one to another. I have done this successfully in my own application and playback appears to be fine.

I am not sure where to go from here.

Do I Dockerize my application to avoid the issues associated with Mac OS?
Do I move away from QMediaPlayer in favor of VLC? (I made a thread on their forums listing a different problem I was having https://forum.videolan.org/viewtopic.php?f=32&t=153667, maybe someone here can chime in about how to resolve it).
Do I use gstreamer, which I am entirely unfamiliar with?

I have to weigh my options. I appreciate everyone's help so far.