HDR videos playback with multimedia/MediaPlayer

namniav

Does multimedia's MediaPlayer support HDR playback ? The default playback outputs lower quality or color depth.

Simple example of MediaPlayer and VideoOutput(on the left) and QuickTime(on the right) plays the same video file:

ffprobe output of the video file:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Whale_3840x2160_60fps_hdr.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 16785407
    compatible_brands: mp42iso4
    creation_time   : 2017-02-27T03:31:16.000000Z
  Duration: 00:01:31.63, start: 0.000000, bitrate: 74630 kb/s
  Stream #0:0(und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 74462 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2017-02-27T03:31:16.000000Z
      handler_name    : Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : HEVC Coding
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2017-02-27T03:31:16.000000Z
      handler_name    : Sound Media Handler
      vendor_id       : [0][0][0][0]

SGaist

Hi and welcome to devnet,

Since you are mentioning QuickTime, I guess you're on macOS.

Qt uses the official backend of the platform so it should be able to play them.

Do you have any settings in QuickTime ?

Which version of Qt are you using ?

namniav

@SGaist Thank you for response.

Do you have any settings in QuickTime ?

No, and I can't find any settings or "preference" that I could change in QuickTime.

Which version of Qt are you using ?

Qt6.2.1
Here's my main.qml, and the video can be downloaded from https://www.demolandia.net/downloads.html?id=5465624, if they are helpful.

import QtCore
import QtMultimedia

import QtQuick
import QtQuick.Window
import QtQuick.Dialogs

Window {
    width: 640
    height: 480
    visible: true
    title: qsTr("QML Video Example")

    Rectangle {
        anchors.fill: parent

        MediaPlayer {
            id: player
            videoOutput: videoOutput
            audioOutput: AudioOutput{}
            loops: MediaPlayer.Infinite
        }

        VideoOutput {
            id: videoOutput
            anchors.fill: parent
        }

        FileDialog {
            id: fileDialog
            fileMode: FileDialog.OpenFile
            currentFile: player.source
            currentFolder: StandardPaths.standardLocations(StandardPaths.MoviesLocation)[0]
            onSelectedFileChanged: {
                player.setSource(fileDialog.selectedFile)
                player.play()
            }
        }

        MouseArea {
            anchors.fill: parent
            onClicked: {
                if (player.playbackState === MediaPlayer.PlayingState) player.pause()
                else if (player.playbackState === MediaPlayer.PausedState) player.play()
            }

            onDoubleClicked: {
                if (player.playbackState === MediaPlayer.PlayingState) player.pause()
                fileDialog.open()
            }
        }

    }
}

namniav

I made a VideoSinkPassThrough QML object and found that the color space of decoded frames is YCbCr_Undefined while the ffprobe tool reports the BT.2020 color space, which is releated to HDR but Qt have no support:

I also have tried changing the color space to each enumeration value of QVideoFrameFormat::YCbCrColorSpace, none of them could output correct colors.

void VideoSinkPassThrough::handleSourceFrameChanged() {
    QVideoFrame frame = m_source->videoFrame();
    if (!frame.isValid()) return;


    if (true) {
        QVideoFrameFormat newFormat = frame.surfaceFormat();
        newFormat.setYCbCrColorSpace(
                    QVideoFrameFormat::YCbCrColorSpace::YCbCr_JPEG);
        QVideoFrame newFrame(newFormat);
        frame.map(QVideoFrame::MapMode::ReadOnly);
        newFrame.map(QVideoFrame::MapMode::WriteOnly);
        for (int i = 0; i < frame.planeCount(); ++i) {
            memcpy(newFrame.bits(i), frame.bits(i), frame.mappedBytes(i));
        }
        frame.unmap();
        newFrame.unmap();
        frame = newFrame;
    }

    ++frameCount;
    startTime = frame.startTime();
    endTime = frame.endTime();
    frameFormat = frame.surfaceFormat();
    if (m_target) m_target->setVideoFrame(frame);
}

void VideoSinkPassThrough::state() {
    qDebug() << "total frame count: " << frameCount;
    qDebug() << "last frmae:";
    qDebug() << "  time: " << startTime << " : " << endTime;
    qDebug() << "  plane count:" << frameFormat.planeCount();
    qDebug() << "  frame rate:" << frameFormat.frameRate();
    qDebug() << "  frame size:" << frameFormat.frameSize();
    qDebug() << "  pixel format:" << frameFormat.pixelFormat();
    qDebug() << "  color space:" << frameFormat.yCbCrColorSpace();
}

SGaist

@namniav said in HDR videos playback with multimedia/MediaPlayer:

BT.2020

Good catch.

I haven't thought about the color space.

That said, your technique won't work, you can't change color spaces by just copying the data around.

It would be better to add support for it directly or implement the conversion from BT.2020 to one of the supported space.

namniav

@SGaist said in HDR videos playback with multimedia/MediaPlayer:

It would be better to add support for it directly or implement the conversion from BT.2020 to one of the supported space.

That is not enough, extra meta data is needed for hdr playback. Decoded frames from Qt lose those meta data. Diving deeper to make things work is beyond my ability, all I can do now is to give up hdr support and wait for official support. Thanks!

HDR format : SMPTE ST 2086, HDR10 compatible
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : R: x=1.000000 y=1.000000, G: x=1.000000 y=1.000000, B: x=1.000000 y=1.000000, White point: x=1.000000 y=1.000000
Mastering display luminance : min: 0.1000 cd/m2, max: 0.5000 cd/m2

SGaist

One thing you can do is check the bug report system to see if there's already something there. If not, please open a feature request with regard to HDR support and the BT.2020 color space.

namniav

@SGaist Ok, I am trying. Thank you.

namniav

This issue has been solved(or improved) since Qt 6.4.0-beta4: QTBUG-98508. Thanks!