Qt6 Multimedia QVideoFrame: native handle?
-
Hi,
I am using Widgets in Qt6. I want to create a small video player example with custom shader effects.
The player is basically working. However, I have to map the QVideoFrame in order to upload its content (bits()) via glTexImage2D(). Due to the necessity of mapping the frame I assume the frame data is already where I need it: on the GPU.
Older Qt versions provided a handle() function. In Qt6 this is removed, although the description still mentions it here: https://doc.qt.io/qt-6/qvideoframe.html#details
The time required for the map() function sometimes exceeds 16ms, resulting in less than 60fps for the video playback.Is there a way to get this texture handle in Qt6 or do I have to copy it once back and forth for every frame?
Thanks in advance! -
Hi,
I am using Widgets in Qt6. I want to create a small video player example with custom shader effects.
The player is basically working. However, I have to map the QVideoFrame in order to upload its content (bits()) via glTexImage2D(). Due to the necessity of mapping the frame I assume the frame data is already where I need it: on the GPU.
Older Qt versions provided a handle() function. In Qt6 this is removed, although the description still mentions it here: https://doc.qt.io/qt-6/qvideoframe.html#details
The time required for the map() function sometimes exceeds 16ms, resulting in less than 60fps for the video playback.Is there a way to get this texture handle in Qt6 or do I have to copy it once back and forth for every frame?
Thanks in advance!@Philerenz The related APIs have been changed and moved to private headers.
You can access the private headers by adding (qmake)QT += multimedia-private
in the .pro file or (cmake)include_directories(${Qt${QT_VERSION_MAJOR}Multimedia_PRIVATE_INCLUDE_DIRS})
in CMakeLists.txt
And then in your source file adding#include <private/qabstractvideobuffer_p.h>
You can get an
QAbstractVideoBuffer
instance byQVideoFrame::videoBuffer()
In Qt5,
QVideoFrame::handle()
just returnsQAbstractVideoBuffer::handle()
of its buffer, in Qt6 there is nohandle()
function inQAbstractVideoBuffer
but other functions likerhi()
,textureHandle()
ortexture()
are added, not sure if you can use those though. -
@Bonnie
Thank you very much for your reply.It feels like I am one step closer, unfortunately
textureHandle()
returns 0. The other two returnQRhi
andQRhiTexture
. I am not sure yet how to actually use these but I keep looking.I guess if
textureHandle()
already returns 0, chances are high I will not find anything in theQRhiTexture
either. Does that mean there is no texture handle? Why is the call tomap()
necessary then?As far as my understanding goes,
QMediaPlayer
decodes the file and calls the specifiedQVideoSink
object for every new frame. If themap()
function is necessary to get a valid buffer viabits()
, the frame data is not in RAM. I suppose a hardware decoder is used behind the scenes. This should result in a texture. Maybe that is not how theQMediaPlayer
works, but I wonder how at least 60fps are achievable otherwise. -
@Bonnie
Thank you very much for your reply.It feels like I am one step closer, unfortunately
textureHandle()
returns 0. The other two returnQRhi
andQRhiTexture
. I am not sure yet how to actually use these but I keep looking.I guess if
textureHandle()
already returns 0, chances are high I will not find anything in theQRhiTexture
either. Does that mean there is no texture handle? Why is the call tomap()
necessary then?As far as my understanding goes,
QMediaPlayer
decodes the file and calls the specifiedQVideoSink
object for every new frame. If themap()
function is necessary to get a valid buffer viabits()
, the frame data is not in RAM. I suppose a hardware decoder is used behind the scenes. This should result in a texture. Maybe that is not how theQMediaPlayer
works, but I wonder how at least 60fps are achievable otherwise.I'm struggling with this as well, I did find an internal function that seems to do exactly what I want:
But since it's private, and the private headers for Qt6 Multimedia don't seem to be available on Ubuntu 22.04, I can't really test it (without getting a custom qt-version).
Waiting for a public API :'(
-
@Philerenz The related APIs have been changed and moved to private headers.
You can access the private headers by adding (qmake)QT += multimedia-private
in the .pro file or (cmake)include_directories(${Qt${QT_VERSION_MAJOR}Multimedia_PRIVATE_INCLUDE_DIRS})
in CMakeLists.txt
And then in your source file adding#include <private/qabstractvideobuffer_p.h>
You can get an
QAbstractVideoBuffer
instance byQVideoFrame::videoBuffer()
In Qt5,
QVideoFrame::handle()
just returnsQAbstractVideoBuffer::handle()
of its buffer, in Qt6 there is nohandle()
function inQAbstractVideoBuffer
but other functions likerhi()
,textureHandle()
ortexture()
are added, not sure if you can use those though.@Bonnie The sort of transitional state of Qt Multimedia is pretty entertaining when you try to read the documentation. It mentions things like "The return value of the handle() function" in the text but then doesn't document a handle() as existing in the Public Functions section.
https://doc.qt.io/qt-6/qvideoframe.html
It's all a bit dusty, and no longer really works the way the docs say, or how it was apparently intended,