Where and in which widget should I put decoded video frames?

lucaszanella

In which widget should I display my ffmpeg decoded frames?

I've found here: https://doc.qt.io/qt-5/videooverview.html that I can use a subclass of QAbstractVideoSurface to process my frames. And I used a QMediaPlayer and passed my QAbstractVideoSurface to it. However, QMediaPlayer is not a displayable element, so which element should output the frames? (In the right way, not using QImages or something like that). Also, my QMediaPlayer never calls the method bool present of QAbstractVideoSurface, so I don't know how it works.

This is my code:

QList<QVideoFrame::PixelFormat> VideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const
{
    Q_UNUSED(handleType);

    // Return the formats you will support
    return QList<QVideoFrame::PixelFormat>() << QVideoFrame::Format_YUV420P;
}

bool VideoSurface::present(const QVideoFrame &frame)
{
    //Q_UNUSED(frame);
    std:: cout << "VideoSurface processing 1 frame " << std::endl; 

    QVideoFrame frametodraw(frame);

    if(!frametodraw.map(QAbstractVideoBuffer::ReadOnly))
    {
        setError(ResourceError);
        return false;
    } 
    // Handle the frame and do your processing
    const size_t bufferSize = 398304;
    uint8_t frameBuffer[bufferSize];
    this->mediaStream->receiveFrame(frameBuffer, bufferSize);
    //Frame is now in frameBuffer, we must put into frametodraw, I guess
    // ------------What should I do here?-------------
    frametodraw.unmap();
    return true;
}

Look at this->mediaStream.decodeFrame(frameBuffer, bufferSize). This line decodes a new h264 frame into frameBuffer in the YUV420P format. Now I need to pass it to the QFrame and I must find a way to display this QFrame on screen

SGaist

Hi and welcome to devnet,

Before digging into this, rather than doing things manually, did you consider using something like VLC-Qt that offers Qt support on top of VLC ?

lucaszanella

@SGaist thank you, but I really need to make it low level, and it's almost ready, as ffmpeg already works, it's just a matter of drawing the pixels in the screen in an efficient way

SGaist

IIRC, you should provide a custom class that provides a QVideoRendererControl interface like explained here.

lucaszanella

@SGaist the problem is that QVideoRendererControl is also a not displayable element. I don't know which element of QT (like, which Widget) is going to display the video. Also, this class should implement QVideoRendererControl but what more? For example, I know I can set my AbstractVideoSurface to it, but when will it call my video surface?

SGaist

QVideoWidget will show your video. Implement a QMediaObject subclass that will provide the control interface. It will also be responsible for the decoding and the rest. Like the any of the QtMultimedia backend is.