Qt6 Multimedia QVideoFrame: native handle?
-
Hi,
I am using Widgets in Qt6. I want to create a small video player example with custom shader effects.
The player is basically working. However, I have to map the QVideoFrame in order to upload its content (bits()) via glTexImage2D(). Due to the necessity of mapping the frame I assume the frame data is already where I need it: on the GPU.
Older Qt versions provided a handle() function. In Qt6 this is removed, although the description still mentions it here: https://doc.qt.io/qt-6/qvideoframe.html#details
The time required for the map() function sometimes exceeds 16ms, resulting in less than 60fps for the video playback.Is there a way to get this texture handle in Qt6 or do I have to copy it once back and forth for every frame?
Thanks in advance! -
Hi,
I am using Widgets in Qt6. I want to create a small video player example with custom shader effects.
The player is basically working. However, I have to map the QVideoFrame in order to upload its content (bits()) via glTexImage2D(). Due to the necessity of mapping the frame I assume the frame data is already where I need it: on the GPU.
Older Qt versions provided a handle() function. In Qt6 this is removed, although the description still mentions it here: https://doc.qt.io/qt-6/qvideoframe.html#details
The time required for the map() function sometimes exceeds 16ms, resulting in less than 60fps for the video playback.Is there a way to get this texture handle in Qt6 or do I have to copy it once back and forth for every frame?
Thanks in advance!@Philerenz The related APIs have been changed and moved to private headers.
You can access the private headers by adding (qmake)QT += multimedia-privatein the .pro file or (cmake)include_directories(${Qt${QT_VERSION_MAJOR}Multimedia_PRIVATE_INCLUDE_DIRS})in CMakeLists.txt
And then in your source file adding#include <private/qabstractvideobuffer_p.h>You can get an
QAbstractVideoBufferinstance byQVideoFrame::videoBuffer()In Qt5,
QVideoFrame::handle()just returnsQAbstractVideoBuffer::handle()of its buffer, in Qt6 there is nohandle()function inQAbstractVideoBufferbut other functions likerhi(),textureHandle()ortexture()are added, not sure if you can use those though. -
@Bonnie
Thank you very much for your reply.It feels like I am one step closer, unfortunately
textureHandle()returns 0. The other two returnQRhiandQRhiTexture. I am not sure yet how to actually use these but I keep looking.I guess if
textureHandle()already returns 0, chances are high I will not find anything in theQRhiTextureeither. Does that mean there is no texture handle? Why is the call tomap()necessary then?As far as my understanding goes,
QMediaPlayerdecodes the file and calls the specifiedQVideoSinkobject for every new frame. If themap()function is necessary to get a valid buffer viabits(), the frame data is not in RAM. I suppose a hardware decoder is used behind the scenes. This should result in a texture. Maybe that is not how theQMediaPlayerworks, but I wonder how at least 60fps are achievable otherwise. -
@Bonnie
Thank you very much for your reply.It feels like I am one step closer, unfortunately
textureHandle()returns 0. The other two returnQRhiandQRhiTexture. I am not sure yet how to actually use these but I keep looking.I guess if
textureHandle()already returns 0, chances are high I will not find anything in theQRhiTextureeither. Does that mean there is no texture handle? Why is the call tomap()necessary then?As far as my understanding goes,
QMediaPlayerdecodes the file and calls the specifiedQVideoSinkobject for every new frame. If themap()function is necessary to get a valid buffer viabits(), the frame data is not in RAM. I suppose a hardware decoder is used behind the scenes. This should result in a texture. Maybe that is not how theQMediaPlayerworks, but I wonder how at least 60fps are achievable otherwise.I'm struggling with this as well, I did find an internal function that seems to do exactly what I want:
But since it's private, and the private headers for Qt6 Multimedia don't seem to be available on Ubuntu 22.04, I can't really test it (without getting a custom qt-version).
Waiting for a public API :'(
-
@Philerenz The related APIs have been changed and moved to private headers.
You can access the private headers by adding (qmake)QT += multimedia-privatein the .pro file or (cmake)include_directories(${Qt${QT_VERSION_MAJOR}Multimedia_PRIVATE_INCLUDE_DIRS})in CMakeLists.txt
And then in your source file adding#include <private/qabstractvideobuffer_p.h>You can get an
QAbstractVideoBufferinstance byQVideoFrame::videoBuffer()In Qt5,
QVideoFrame::handle()just returnsQAbstractVideoBuffer::handle()of its buffer, in Qt6 there is nohandle()function inQAbstractVideoBufferbut other functions likerhi(),textureHandle()ortexture()are added, not sure if you can use those though.@Bonnie The sort of transitional state of Qt Multimedia is pretty entertaining when you try to read the documentation. It mentions things like "The return value of the handle() function" in the text but then doesn't document a handle() as existing in the Public Functions section.
https://doc.qt.io/qt-6/qvideoframe.html
It's all a bit dusty, and no longer really works the way the docs say, or how it was apparently intended,