QtMultimedia and QAbstractVideoBuffer

aureshinite

Hi everyone...

I am currently trying to write a low level video player, which reads raw YUV frames and displays them.
Its video engine will interact with many other components, principally by signaling the current frame.
Actually, my player is working fine individually, but i believe i need a nice structure to encapsulate
everything.

I found out QtMultimedia has what I need with QVideoFrame, but I don't really
know how QAbstractVideoBuffer is supposed to be used.
Can we create a valid QVideoFrame without a QAbstractVideoBuffer? (refering to the ::map () member function )

Thanks

Mauro

Hi!
I'm having the same problem: trying to figure out how to use the QtMultimedia module for playing video files.
Maybe we can share our efforts and get some progress together.

I'm using libvlc in order to decode video files: each time a new frame is ready, libvlc stores the RGB32 bytes representation in a preallocated buffer in memory, I'll call a pointer to this buffer "backendBuffer".

I thought that I could just subclass QAbstractVideoBuffer so as to have a wrapper for the backendBuffer and return it with the map funcion:

This is the header:

@
class VideoBuffer : public QAbstractVideoBuffer
{
public:
VideoBuffer(HandleType type = NoHandle);
virtual ~VideoBuffer();

uchar* map(MapMode mode, int* numBytes, int* bytesPerLine);
MapMode mapMode() const;
void unmap();

void setBackendBuffer(uchar* buffer, int frameWidth, int frameHeight);
int frameWidth() { return _frameWidth; };
int frameHeight() { return _frameHeight; };

private:
uchar* _backendBuffer;
int _frameWidth;
int _frameHeight;
};
@

and this is the definition:

@
VideoBuffer::VideoBuffer(HandleType type)
: QAbstractVideoBuffer(type)
{
}

VideoBuffer::~VideoBuffer()
{
}

uchar* VideoBuffer::map(MapMode mode, int* numBytes, int* bytesPerLine)
{
// I just ignore the MapMode

if (numBytes != NULL)
*numBytes = _frameWidth * _frameHeight * 4;
if (bytesPerLine != NULL)
*bytesPerLine = _frameWidth * 4;

return _backendBuffer; // should I return a copy of the backendBuffer?
}

QAbstractVideoBuffer::MapMode VideoBuffer::mapMode() const
{
return ReadOnly;
}

void VideoBuffer::unmap()
{
// I assume there's nothing to do if I don't return a copy of the bakendBuffer
}

void VideoBuffer::setBackendBuffer(uchar* buffer, int frameWidth, int frameHeight)
{
_backendBuffer = buffer;
_frameWidth = frameWidth;
_frameHeight = frameHeight;
}
@

The idea is to call VideoBuffer::setBackendBuffer(...) passing in the preallocated memory buffer which is filled in by libvlc, for instance:

@
_videoBuffer = new VideoBuffer;
_videoBuffer->setBackendBuffer(_vlcBufferDataPtr, _frameWidth, _frameHeight);
@

Now, each time a new frame is ready to be displayed (its RGB32 representation has been written in the backendBuffer), libvlc calls the following method with a pointer to the VideoBuffer that wraps the backendBuffer.
I create a QVideoFrame and then call QAbstractVideoSurface::present(...).

QAbstractVideoSurface code is exactly the one you can find in the Video Widget Example.
In the following code, videoSurface is a pointer to a QAbstractVideoSurface object.

@
void Controller::bufferReady(VideoBuffer* videoBuffer)
{
mutex.lock();

QSize frameSize( videoBuffer->frameWidth(), videoBuffer->frameHeight() );
QVideoFrame videoFrame( videoBuffer, frameSize, QVideoFrame::Format_RGB32 );

if (!videoFrame.isValid())
    return; // false

QVideoSurfaceFormat currentSurfaceFormat = videoSurface->surfaceFormat();

if (videoFrame.pixelFormat() != currentSurfaceFormat.pixelFormat()
        || videoFrame.size() != currentSurfaceFormat.frameSize()) {
    QVideoSurfaceFormat format(videoFrame.size(), videoFrame.pixelFormat());

    if (!videoSurface->start(format))
        return; // false;
}

if (!videoSurface->present(videoFrame))
{
 videoSurface->stop();
    return; // false;
} else {
    return; // true;
}

mutex.unlock();
}
@

However, this code crashes...
Any idea why?
Thanks for any help!