Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

HDR videos playback with multimedia/MediaPlayer



  • Does multimedia's MediaPlayer support HDR playback ? The default playback outputs lower quality or color depth.

    Simple example of MediaPlayer and VideoOutput(on the left) and QuickTime(on the right) plays the same video file:

    703b4ae7-d6d2-47a1-b370-ee13656843fe-image.png
    385aca7b-7a9f-4bc8-9191-04283c55b39e-image.png

    ffprobe output of the video file:

    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Whale_3840x2160_60fps_hdr.mp4':
      Metadata:
        major_brand     : mp42
        minor_version   : 16785407
        compatible_brands: mp42iso4
        creation_time   : 2017-02-27T03:31:16.000000Z
      Duration: 00:01:31.63, start: 0.000000, bitrate: 74630 kb/s
      Stream #0:0(und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 74462 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 59.94 tbc (default)
        Metadata:
          creation_time   : 2017-02-27T03:31:16.000000Z
          handler_name    : Video Media Handler
          vendor_id       : [0][0][0][0]
          encoder         : HEVC Coding
      Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
        Metadata:
          creation_time   : 2017-02-27T03:31:16.000000Z
          handler_name    : Sound Media Handler
          vendor_id       : [0][0][0][0]
    

  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Since you are mentioning QuickTime, I guess you're on macOS.

    Qt uses the official backend of the platform so it should be able to play them.

    Do you have any settings in QuickTime ?

    Which version of Qt are you using ?



  • @SGaist Thank you for response.

    Do you have any settings in QuickTime ?

    No, and I can't find any settings or "preference" that I could change in QuickTime.

    Which version of Qt are you using ?

    Qt6.2.1
    Here's my main.qml, and the video can be downloaded from https://www.demolandia.net/downloads.html?id=5465624, if they are helpful.

    import QtCore
    import QtMultimedia
    
    import QtQuick
    import QtQuick.Window
    import QtQuick.Dialogs
    
    Window {
        width: 640
        height: 480
        visible: true
        title: qsTr("QML Video Example")
    
        Rectangle {
            anchors.fill: parent
    
            MediaPlayer {
                id: player
                videoOutput: videoOutput
                audioOutput: AudioOutput{}
                loops: MediaPlayer.Infinite
            }
    
            VideoOutput {
                id: videoOutput
                anchors.fill: parent
            }
    
            FileDialog {
                id: fileDialog
                fileMode: FileDialog.OpenFile
                currentFile: player.source
                currentFolder: StandardPaths.standardLocations(StandardPaths.MoviesLocation)[0]
                onSelectedFileChanged: {
                    player.setSource(fileDialog.selectedFile)
                    player.play()
                }
            }
    
            MouseArea {
                anchors.fill: parent
                onClicked: {
                    if (player.playbackState === MediaPlayer.PlayingState) player.pause()
                    else if (player.playbackState === MediaPlayer.PausedState) player.play()
                }
    
                onDoubleClicked: {
                    if (player.playbackState === MediaPlayer.PlayingState) player.pause()
                    fileDialog.open()
                }
            }
    
        }
    }
    


  • I made a VideoSinkPassThrough QML object and found that the color space of decoded frames is YCbCr_Undefined while the ffprobe tool reports the BT.2020 color space, which is releated to HDR but Qt have no support:
    0eeb6ca7-e06a-4523-94c3-ef2512b92e04-image.png

    I also have tried changing the color space to each enumeration value of QVideoFrameFormat::YCbCrColorSpace, none of them could output correct colors.

    void VideoSinkPassThrough::handleSourceFrameChanged() {
        QVideoFrame frame = m_source->videoFrame();
        if (!frame.isValid()) return;
    
    
        if (true) {
            QVideoFrameFormat newFormat = frame.surfaceFormat();
            newFormat.setYCbCrColorSpace(
                        QVideoFrameFormat::YCbCrColorSpace::YCbCr_JPEG);
            QVideoFrame newFrame(newFormat);
            frame.map(QVideoFrame::MapMode::ReadOnly);
            newFrame.map(QVideoFrame::MapMode::WriteOnly);
            for (int i = 0; i < frame.planeCount(); ++i) {
                memcpy(newFrame.bits(i), frame.bits(i), frame.mappedBytes(i));
            }
            frame.unmap();
            newFrame.unmap();
            frame = newFrame;
        }
    
        ++frameCount;
        startTime = frame.startTime();
        endTime = frame.endTime();
        frameFormat = frame.surfaceFormat();
        if (m_target) m_target->setVideoFrame(frame);
    }
    
    void VideoSinkPassThrough::state() {
        qDebug() << "total frame count: " << frameCount;
        qDebug() << "last frmae:";
        qDebug() << "  time: " << startTime << " : " << endTime;
        qDebug() << "  plane count:" << frameFormat.planeCount();
        qDebug() << "  frame rate:" << frameFormat.frameRate();
        qDebug() << "  frame size:" << frameFormat.frameSize();
        qDebug() << "  pixel format:" << frameFormat.pixelFormat();
        qDebug() << "  color space:" << frameFormat.yCbCrColorSpace();
    }
    

  • Lifetime Qt Champion

    @namniav said in HDR videos playback with multimedia/MediaPlayer:

    BT.2020

    Good catch.

    I haven't thought about the color space.

    That said, your technique won't work, you can't change color spaces by just copying the data around.

    It would be better to add support for it directly or implement the conversion from BT.2020 to one of the supported space.



  • @SGaist said in HDR videos playback with multimedia/MediaPlayer:

    It would be better to add support for it directly or implement the conversion from BT.2020 to one of the supported space.

    That is not enough, extra meta data is needed for hdr playback. Decoded frames from Qt lose those meta data. Diving deeper to make things work is beyond my ability, all I can do now is to give up hdr support and wait for official support. Thanks!

    HDR format : SMPTE ST 2086, HDR10 compatible
    Color range : Limited
    Color primaries : BT.2020
    Transfer characteristics : PQ
    Matrix coefficients : BT.2020 non-constant
    Mastering display color primaries : R: x=1.000000 y=1.000000, G: x=1.000000 y=1.000000, B: x=1.000000 y=1.000000, White point: x=1.000000 y=1.000000
    Mastering display luminance : min: 0.1000 cd/m2, max: 0.5000 cd/m2


  • Lifetime Qt Champion

    One thing you can do is check the bug report system to see if there's already something there. If not, please open a feature request with regard to HDR support and the BT.2020 color space.



  • @SGaist Ok, I am trying. Thank you.


Log in to reply