Problems with QMediaPlayer video frames

  • I'm using a QMediaPlayer to playback a video, and I need to process the video frames. Ideally I would like to process these frames on the GPU so I was hoping I could get access to the OpenGL textures associated with each frame.

    I am having two problems:
    First, the QVideoFrames generated by the player are not giving me access to the texture, and calling handleType() returns NoHandle.

    Second, when mapping the bits on a video frame, just to access the data, the format appears to be BGR24 instead of RGB24, even though my video surface only supports RGB24.

    I wrote a class inheriting from QAbstractVideoSurface to act as a video output. I have a QMediaPlayer that is loading a video and have connected it to my video surface using setVideoOutput().

    Here is the code:

    /* TestVideoSurface.h */
    #include <QAbstractVideoSurface>
    class TestVideoSurface : public QAbstractVideoSurface
        bool present(const QVideoFrame &frame) override;
        QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType type = QAbstractVideoBuffer::NoHandle) const override;
        bool isFormatSupported(const QVideoSurfaceFormat &format) const override { return true; }
    /* TestVideoSurface.cpp */
    #include "TestVideoSurface.h"
    bool TestVideoSurface::present(const QVideoFrame &frame)
        QAbstractVideoBuffer::HandleType handleType = frame.handleType();
        switch (handleType)
        case QAbstractVideoBuffer::NoHandle:
            // this case is always hit
            // CPU approach
            QVideoFrame frameCopy = frame;
        case QAbstractVideoBuffer::GLTextureHandle:
            // this case is never hit
            // GPU approach
        return true;
    QList<QVideoFrame::PixelFormat> TestVideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType type) const
        QList<QVideoFrame::PixelFormat> list;
        list << QVideoFrame::PixelFormat::Format_RGB24;
        return list;

    And to start playback:

    qtPlayer.setMedia(QUrl::fromUserInput("data/test.mp4", QCoreApplication::applicationDirPath()));;

  • Lifetime Qt Champion


    The management of the format and type of the texture depends on the platform multimedia backend.

    What OS are you running ? With which version of Qt ?

Log in to reply

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.