[PyQt5] Allow QMediaPlayer to read from QBuffer()

[Elus]

Hi there,

I'm trying to create a video player. My goal is to play 2 or more consecutive videos seamlessly, without a gap where one video ends and the next one starts.

QMediaPlaylist, which I'm currently using, has a small 'gap' in playback, which is disruptive to viewing and listening.

Potential solution: one should in theory be able to read the videos, write them to a buffer, and then allow QMediaPlayer to read from that buffer as if it's one big video file.

Does anyone have any insight as to why the code below doesn't work?

I am on Mac OS 10.14 running Python 3.7.6 with PyQT5==5.14.2.

import sys
import os

from PyQt5.QtWidgets import QMainWindow
from PyQt5.QtCore import QUrl, QFile, QIODevice,  QBuffer
from PyQt5.QtMultimedia import QMediaContent, QMediaPlaylist, QMediaPlayer
from PyQt5.QtMultimediaWidgets import QVideoWidget
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QWidget


class SimplePlayer(QMainWindow):
    """
    Extremely simple video player using QMediaPlayer
    Consists of vertical layout, widget, and a QLabel
    """

    def __init__(self, master=None):
        QMainWindow.__init__(self, master)

        # Define file variables
        self.playlist_files = ['video_file_1.mp4', 'video_file_2.mp4']

        # Define the ui-specific variables we're going to use
        self.vertical_box_layout = QVBoxLayout()
        self.central_widget = QWidget(self)
        self.video_frame = QVideoWidget()

        # Define the media player related information
        self.playlist = QMediaPlaylist()
        self.video_player = QMediaPlayer(flags=QMediaPlayer.VideoSurface)

        # Create the user interface, set up the player, and play the 2 videos
        self.create_user_interface()
        self.video_player_setup()

    def video_player_setup(self):
        """Sets media list for the player and then sets output to the video frame"""
        self.video_player.setVideoOutput(self.video_frame)

        self.set_buffer()
        # self.set_playlist()
        self.video_player.play()

    def set_playlist(self):
        """Opens a single video file, puts it into a playlist which is read by the QMediaPlayer"""
        self.playlist.addMedia(QMediaContent(QUrl.fromLocalFile(os.path.abspath(self.playlist_files[0]))))
        self.playlist.setCurrentIndex(0)
        self.video_player.setPlaylist(self.playlist)

    def set_buffer(self):
        """Opens a single video file and writes it to a buffer to be read by QMediaPlayer"""
        media_file_name = os.path.abspath(self.playlist_files[0])
        media_file = QFile(media_file_name)
        media_file.open(QIODevice.ReadOnly)

        byte_array = media_file.readAll()
        buffer = QBuffer(byte_array)
        buffer.setData(byte_array)

        buffer.open(QIODevice.ReadOnly)

        self.video_player.setMedia(QMediaContent(), buffer)

    def create_user_interface(self):
        """Create a 1280x720 UI consisting of a vertical layout, central widget, and QLabel"""
        self.setCentralWidget(self.central_widget)
        self.vertical_box_layout.addWidget(self.video_frame)
        self.central_widget.setLayout(self.vertical_box_layout)

        self.resize(1280, 720)


if __name__ == '__main__':
    app = QApplication([])
    player = SimplePlayer()
    player.show()
    sys.exit(app.exec_())

If anyone has any helpful advice on making this work or how I can accomplish this goal, I would much appreciate it.

Thank you.

[jsulm]

@Elus said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

buffer = QBuffer(byte_array)
buffer.setData(byte_array)

Why do you set the array two times?
But the actual problem is that byte_array only exists inside set_buffer(). See https://doc.qt.io/qt-5/qbuffer.html#QBuffer-1 "The caller is responsible for ensuring that byteArray remains valid until the QBuffer is destroyed".

[Elus]

@jsulm said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

buffer.setData(byte_array)

Hi,

Thanks so much for the advice!

If I understand what you're saying, when we exit the set_buffer() function, the byte_array goes out of scope and becomes invalid. Is this a fair interpretation?

If so, is the solution to make byte_array and buffer class variables that persist at runtime?

I attempted to do so below, but that does not seem to work either.

import sys
import os

from PyQt5.QtWidgets import QMainWindow
from PyQt5.QtCore import QUrl, QFile, QIODevice,  QBuffer
from PyQt5.QtMultimedia import QMediaContent, QMediaPlaylist, QMediaPlayer
from PyQt5.QtMultimediaWidgets import QVideoWidget
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QWidget


class SimplePlayer(QMainWindow):
    """
    Extremely simple video player using QMediaPlayer
    Consists of vertical layout, widget, and a QLabel
    """

    def __init__(self, master=None):
        QMainWindow.__init__(self, master)

        # Define file variables
        self.playlist_files = ['video_file_1.mp4', 'video_file_2.mp4']

        # Define the QT-specific variables we're going to use
        self.vertical_box_layout = QVBoxLayout()
        self.central_widget = QWidget(self)
        self.video_frame = QVideoWidget()

        # Define the media player related information
        self.playlist = QMediaPlaylist()
        self.video_player = QMediaPlayer(flags=QMediaPlayer.VideoSurface)
        self.buffer = QBuffer()

        # Create the user interface, set up the player, and play the 2 videos
        self.create_user_interface()
        self.video_player_setup()

    def video_player_setup(self):
        """Sets media list for the player and then sets output to the video frame"""
        self.video_player.setVideoOutput(self.video_frame)

        self.set_buffer()
        # self.set_playlist()
        self.video_player.play()

    def set_playlist(self):
        """Opens a single video file, puts it into a playlist which is read by the QMediaPlayer"""
        self.playlist.addMedia(QMediaContent(QUrl.fromLocalFile(os.path.abspath(self.playlist_files[0]))))
        self.playlist.setCurrentIndex(0)
        self.video_player.setPlaylist(self.playlist)

    def set_buffer(self):
        """Opens a single video file and writes it to a buffer to be read by QMediaPlayer"""
        media_file_name = os.path.abspath(self.playlist_files[0])
        media_file = QFile(media_file_name)
        media_file.open(QIODevice.ReadOnly)

        self.byte_array = media_file.readAll()
        self.buffer.setData(self.byte_array)
        self.buffer.open(QIODevice.ReadOnly)

        self.video_player.setMedia(QMediaContent(), self.buffer)

    def create_user_interface(self):
        """Create a 1280x720 UI consisting of a vertical layout, central widget, and QLabel"""
        self.setCentralWidget(self.central_widget)
        self.vertical_box_layout.addWidget(self.video_frame)
        self.central_widget.setLayout(self.vertical_box_layout)

        self.resize(1280, 720)


if __name__ == '__main__':
    app = QApplication([])
    player = SimplePlayer()
    player.show()
    sys.exit(app.exec_())

Thank you.

[jsulm]

@Elus said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

Is this a fair interpretation?

Yes.
Did you debug your app actually? Was the file open successfully?

[Elus]

@jsulm

Hi,

Yes, I re-ran the program and I did not receive any errors. However, I still receive a black screen.

Also, as a sanity check, replacing the call to set_buffer() with a call to set_playlist() function does successfully play the files, so they are not corrupt.

I am a Qt newbie, so if there is a formal way to debug further, I am happy to try.

Thank you.

Edit: Apparently, I'm a new user so I cannot respond more frequently than every 10 minutes since the Qt forum seems to be limiting the rate at which I can post replies. Sorry about that!

Edit2: Odd, seems like the posting limitation went away?

[jsulm]

@Elus You should connect slots to QMediaPlayer::mediaStatusChanged() and QMediaPlayer::error() signals to see what happens.

[Elus]

@jsulm

Hi,

Good idea. When I do that and I use the set_playlist() function instead of set_buffer(), I get the following output:

Status: 2
Status: 6
Status: 7
Status: 1

According to this reference: https://doc.qt.io/archives/qt-5.5/qmediaplayer.html#MediaStatus-enum

• Status 2 corresponds to LoadingMedia
• Status 6 represents BufferedMedia
• Status 7 represents EndOfMedia
• Status 1 represents NoMedia

When I use set_buffer(), I do not receive any output to stdout. Here's my implementation of connecting the signals, just to make sure I've done things properly:

import sys
import os

from PyQt5.QtWidgets import QMainWindow
from PyQt5.QtCore import QUrl, QFile, QIODevice,  QBuffer
from PyQt5.QtMultimedia import QMediaContent, QMediaPlaylist, QMediaPlayer
from PyQt5.QtMultimediaWidgets import QVideoWidget
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QWidget


class SimplePlayer(QMainWindow):
    """
    Extremely simple video player using QMediaPlayer
    Consists of vertical layout, widget, and a QLabel
    """

    def __init__(self, master=None):
        QMainWindow.__init__(self, master)

        # Define file variables
        self.playlist_files = ['video_file_1.mp4', 'video_file_2.mp4']

        # Define the QT-specific variables we're going to use
        self.vertical_box_layout = QVBoxLayout()
        self.central_widget = QWidget(self)
        self.video_frame = QVideoWidget()

        # Define the media player related information
        self.playlist = QMediaPlaylist()
        self.video_player = QMediaPlayer(flags=QMediaPlayer.VideoSurface)
        self.buffer = QBuffer()

        # Connect error & media status signals to functions that print those signals to stdout
        self.video_player.error.connect(self.print_media_player_error)
        self.video_player.mediaStatusChanged.connect(self.print_media_player_status)

        # Create the user interface, set up the player, and play the 2 videos
        self.create_user_interface()
        self.video_player_setup()

    def print_media_player_error(self, value):
        """Prints any errors media player encounters"""
        print(f"Error: {value}")

    def print_media_player_status(self, value):
        """Prints any status changes to media player"""
        print(f"Status: {value}")

    def video_player_setup(self):
        """Sets media list for the player and then sets output to the video frame"""
        self.video_player.setVideoOutput(self.video_frame)

        self.set_buffer()
        # self.set_playlist()
        self.video_player.play()

    def set_playlist(self):
        """Opens a single video file, puts it into a playlist which is read by the QMediaPlayer"""
        self.playlist.addMedia(QMediaContent(QUrl.fromLocalFile(os.path.abspath(self.playlist_files[0]))))
        self.playlist.setCurrentIndex(0)
        self.video_player.setPlaylist(self.playlist)

    def set_buffer(self):
        """Opens a single video file and writes it to a buffer to be read by QMediaPlayer"""
        media_file_name = os.path.abspath(self.playlist_files[0])
        media_file = QFile(media_file_name)
        media_file.open(QIODevice.ReadOnly)

        self.byte_array = media_file.readAll()
        self.buffer.setData(self.byte_array)
        self.buffer.open(QIODevice.ReadOnly)

        self.video_player.setMedia(QMediaContent(), self.buffer)

    def create_user_interface(self):
        """Create a 1280x720 UI consisting of a vertical layout, central widget, and QLabel"""
        self.setCentralWidget(self.central_widget)
        self.vertical_box_layout.addWidget(self.video_frame)
        self.central_widget.setLayout(self.vertical_box_layout)

        self.resize(1280, 720)


if __name__ == '__main__':
    app = QApplication([])
    player = SimplePlayer()
    player.show()
    sys.exit(app.exec_())

[Elus]

I also went ahead and did a sanity check on the buffer variable by checking its size before and after I write the byte_array to the buffer. I get the following output to stdout:

The size of buffer before adding the byte_array is: 0
The size of buffer after adding the byte_array is: 3759778

This is the code:

import sys
import os

from PyQt5.QtWidgets import QMainWindow
from PyQt5.QtCore import QUrl, QFile, QIODevice,  QBuffer
from PyQt5.QtMultimedia import QMediaContent, QMediaPlaylist, QMediaPlayer
from PyQt5.QtMultimediaWidgets import QVideoWidget
from PyQt5.QtWidgets import QApplication, QVBoxLayout, QWidget


class SimplePlayer(QMainWindow):
    """
    Extremely simple video player using QMediaPlayer
    Consists of vertical layout, widget, and a QLabel
    """

    def __init__(self, master=None):
        QMainWindow.__init__(self, master)

        # Define file variables
        self.playlist_files = ['video_file_1.mp4', 'video_file_2.mp4']

        # Define the QT-specific variables we're going to use
        self.vertical_box_layout = QVBoxLayout()
        self.central_widget = QWidget(self)
        self.video_frame = QVideoWidget()

        # Define the media player related information
        self.playlist = QMediaPlaylist()
        self.video_player = QMediaPlayer(flags=QMediaPlayer.VideoSurface)
        self.buffer = QBuffer()

        # Connect error & media status signalsto functions that print those signals to stdout
        self.video_player.error.connect(self.print_media_player_error)
        self.video_player.mediaStatusChanged.connect(self.print_media_player_status)

        # Create the user interface, set up the player, and play the 2 videos
        self.create_user_interface()
        self.video_player_setup()

    def print_media_player_error(self, value):
        """Prints any errors media player encounters"""
        print(f"Error: {value}")

    def print_media_player_status(self, value):
        """Prints any status changes to media player"""
        print(f"Status: {value}")

    def video_player_setup(self):
        """Sets media list for the player and then sets output to the video frame"""
        self.video_player.setVideoOutput(self.video_frame)

        self.set_buffer()
        # self.set_playlist()
        self.video_player.play()

    def set_playlist(self):
        """Opens a single video file, puts it into a playlist which is read by the QMediaPlayer"""
        self.playlist.addMedia(QMediaContent(QUrl.fromLocalFile(os.path.abspath(self.playlist_files[0]))))
        self.playlist.setCurrentIndex(0)
        self.video_player.setPlaylist(self.playlist)

    def set_buffer(self):
        """Opens a single video file and writes it to a buffer to be read by QMediaPlayer"""
        media_file_name = os.path.abspath(self.playlist_files[0])
        media_file = QFile(media_file_name)
        media_file.open(QIODevice.ReadOnly)
        print(f"The size of buffer before adding the byte_array is: {self.buffer.size()}")
        self.byte_array = media_file.readAll()
        self.buffer.setData(self.byte_array)
        self.buffer.open(QIODevice.ReadOnly)
        print(f"The size of buffer after adding the byte_array is: {self.buffer.size()}")
        self.video_player.setMedia(QMediaContent(), self.buffer)

    def create_user_interface(self):
        """Create a 1280x720 UI consisting of a vertical layout, central widget, and QLabel"""
        self.setCentralWidget(self.central_widget)
        self.vertical_box_layout.addWidget(self.video_frame)
        self.central_widget.setLayout(self.vertical_box_layout)

        self.resize(1280, 720)


if __name__ == '__main__':
    app = QApplication([])
    player = SimplePlayer()
    player.show()
    sys.exit(app.exec_())

[Elus]

@jsulm

Is this the cause of the problem? https://bugreports.qt.io/browse/QTBUG-69101

[Elus]

Bumping... any solutions / suggestions?

[Bonnie]

@Elus Hi, don't pass a QMediaContent() as in the doc:

Setting the media to a null QMediaContent will cause the player to discard all information relating to the current media source and to cease all I/O operations related to that media.

Try pass a QUrl() to it.
(I'm not sure if you can do this in python, or you can pass a QMediaContent(QUrl()))

[Elus]

@Bonnie said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

Setting the media to a null QMediaContent will cause the player to discard all information relating to the current media source and to cease all I/O operations related to that media.

I tried:
media_file = QUrl(media_file_name)
But that resulted in exception:
setData(self, Union[QByteArray, bytes, bytearray]): argument 1 has unexpected type 'QUrl'

I also tried:
media_file = QMediaContent(QUrl(media_file_name))
But that resulted in exception:
setData(self, Union[QByteArray, bytes, bytearray]): argument 1 has unexpected type 'QMediaContent'

I can't pass QMediaContent to buffer directly.

[Bonnie]

@Elus No, no, I mean inset_buffer

self.video_player.setMedia(QUrl(), self.buffer)

[Elus]

@VaL-Doroshchuk Would you know anything about this?

@Bonnie
One moment let me try

[Elus]

@Bonnie said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

self.video_player.setMedia(QUrl(), self.buffer)

I tried this and received error:
TypeError: setMedia(self, QMediaContent, stream: QIODevice = None): argument 1 has unexpected type 'QUrl'

[Bonnie]

@Elus Yes, I said I'm not sure if you can do this in python...
Then can this work? I don't know about python...

self.video_player.setMedia(QMediaContent(QUrl()), self.buffer)

[Elus]

@Bonnie

Unfortunately, the screen is still blank if I do that. Do you think this is related to my issue? https://bugreports.qt.io/browse/QTBUG-69101

[Bonnie]

@Elus Not sure, I tested in Windows and C++.

[eyllanesc]

@Elus I don't think joining 2 videos is as simple as concatenating raw data since it has a header that indicates the format, image size, metadata, etc.

If you still want to display a video using raw data then you can use the following example based on my SO answer:

import os
import sys

from PyQt5 import QtCore, QtWidgets, QtMultimedia, QtMultimediaWidgets

CURRENT_DIR = os.path.dirname(os.path.realpath(__file__))


class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)

        self.player = QtMultimedia.QMediaPlayer(
            flags=QtMultimedia.QMediaPlayer.VideoSurface
        )
        self.buff = QtCore.QBuffer()

        self.video_widget = QtMultimediaWidgets.QVideoWidget()
        self.setCentralWidget(self.video_widget)

        self.player.setVideoOutput(self.video_widget)

        self.resize(640, 480)

    def load_from_file(self, filename):
        f = QtCore.QFile(filename)
        if f.open(QtCore.QIODevice.ReadOnly):
            ba = f.readAll()
            self.load_from_data(ba)

    def load_from_data(self, data):
        ba = QtCore.QByteArray(data)
        self.buff.setData(ba)
        self.buff.open(QtCore.QIODevice.ReadOnly)
        self.player.setMedia(QtMultimedia.QMediaContent(), self.buff)
        self.player.play()


def main():
    app = QtWidgets.QApplication.instance()
    if app is None:
        app = QtWidgets.QApplication(sys.argv)

    w = MainWindow()
    w.show()

    filename = os.path.join(CURRENT_DIR, "video.mp4")
    w.load_from_file(filename)

    sys.exit(app.exec_())


if __name__ == "__main__":
    main()

├── main.py
└── video.mp4

My example has been tested on Linux with PyQt5 5.14.2 and python 3.8.3

[Elus]

@Bonnie Thanks for giving it a try. I think your experiment, along with @eyllanesc 's code, confirms my suspicions.

@eyllanesc said in [PyQt5] Allow QMediaPlayer to read from QBuffer():

import os
import sys

from PyQt5 import QtCore, QtWidgets, QtMultimedia, QtMultimediaWidgets

CURRENT_DIR = os.path.dirname(os.path.realpath(file))

class MainWindow(QtWidgets.QMainWindow):
def init(self, parent=None):
super().init(parent)

    self.player = QtMultimedia.QMediaPlayer(
        flags=QtMultimedia.QMediaPlayer.VideoSurface
    )
    self.buff = QtCore.QBuffer()

    self.video_widget = QtMultimediaWidgets.QVideoWidget()
    self.setCentralWidget(self.video_widget)

    self.player.setVideoOutput(self.video_widget)

    self.resize(640, 480)

def load_from_file(self, filename):
    f = QtCore.QFile(filename)
    if f.open(QtCore.QIODevice.ReadOnly):
        ba = f.readAll()
        self.load_from_data(ba)

def load_from_data(self, data):
    ba = QtCore.QByteArray(data)
    self.buff.setData(ba)
    self.buff.open(QtCore.QIODevice.ReadOnly)
    self.player.setMedia(QtMultimedia.QMediaContent(), self.buff)
    self.player.play()

def main():
app = QtWidgets.QApplication.instance()
if app is None:
app = QtWidgets.QApplication(sys.argv)

w = MainWindow()
w.show()

filename = os.path.join(CURRENT_DIR, "video.mp4")
w.load_from_file(filename)

sys.exit(app.exec_())

if name == "main":
main()

Thanks a bunch. Your code, which you've tested, confirms my suspicions that https://bugreports.qt.io/browse/QTBUG-69101 is the root cause of my issues and that the underlying problem is Mac OS. Running your code against several different videos produces the same result for me every time: a black screen.

Regarding your comment about the header requirement, I think what you're saying is true of certain video formats. For example, certain compression algorithms are able to reduce the file size by looking at the changes between frames. Therefore, if you simply concatenate one set of bytes to another, as I am trying to do, you'd be missing a critical piece of information that can help you decode the next frame.

However, I do not know if all video formats rely on this kind of paradigm. I believe certain formats, like .ts, allow videos to be concatenated, one to another. I have done this successfully in my own application and playback appears to be fine.

I am not sure where to go from here.

Do I Dockerize my application to avoid the issues associated with Mac OS?
Do I move away from QMediaPlayer in favor of VLC? (I made a thread on their forums listing a different problem I was having https://forum.videolan.org/viewtopic.php?f=32&t=153667, maybe someone here can chime in about how to resolve it).
Do I use gstreamer, which I am entirely unfamiliar with?

I have to weigh my options. I appreciate everyone's help so far.