Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Qt6 Multimedia QVideoFrame: native handle?
Forum Updated to NodeBB v4.3 + New Features

Qt6 Multimedia QVideoFrame: native handle?

Scheduled Pinned Locked Moved Unsolved General and Desktop
5 Posts 4 Posters 1.0k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P Offline
    P Offline
    Philerenz
    wrote on last edited by
    #1

    Hi,

    I am using Widgets in Qt6. I want to create a small video player example with custom shader effects.
    The player is basically working. However, I have to map the QVideoFrame in order to upload its content (bits()) via glTexImage2D(). Due to the necessity of mapping the frame I assume the frame data is already where I need it: on the GPU.
    Older Qt versions provided a handle() function. In Qt6 this is removed, although the description still mentions it here: https://doc.qt.io/qt-6/qvideoframe.html#details
    The time required for the map() function sometimes exceeds 16ms, resulting in less than 60fps for the video playback.

    Is there a way to get this texture handle in Qt6 or do I have to copy it once back and forth for every frame?
    Thanks in advance!

    B 1 Reply Last reply
    0
    • P Philerenz

      Hi,

      I am using Widgets in Qt6. I want to create a small video player example with custom shader effects.
      The player is basically working. However, I have to map the QVideoFrame in order to upload its content (bits()) via glTexImage2D(). Due to the necessity of mapping the frame I assume the frame data is already where I need it: on the GPU.
      Older Qt versions provided a handle() function. In Qt6 this is removed, although the description still mentions it here: https://doc.qt.io/qt-6/qvideoframe.html#details
      The time required for the map() function sometimes exceeds 16ms, resulting in less than 60fps for the video playback.

      Is there a way to get this texture handle in Qt6 or do I have to copy it once back and forth for every frame?
      Thanks in advance!

      B Offline
      B Offline
      Bonnie
      wrote on last edited by Bonnie
      #2

      @Philerenz The related APIs have been changed and moved to private headers.
      You can access the private headers by adding (qmake) QT += multimedia-private in the .pro file or (cmake) include_directories(${Qt${QT_VERSION_MAJOR}Multimedia_PRIVATE_INCLUDE_DIRS}) in CMakeLists.txt
      And then in your source file adding

      #include <private/qabstractvideobuffer_p.h>
      

      You can get an QAbstractVideoBuffer instance by QVideoFrame::videoBuffer()

      In Qt5, QVideoFrame::handle() just returns QAbstractVideoBuffer::handle() of its buffer, in Qt6 there is no handle() function in QAbstractVideoBuffer but other functions like rhi(), textureHandle() or texture() are added, not sure if you can use those though.

      W 1 Reply Last reply
      1
      • P Offline
        P Offline
        Philerenz
        wrote on last edited by
        #3

        @Bonnie
        Thank you very much for your reply.

        It feels like I am one step closer, unfortunately textureHandle() returns 0. The other two return QRhi and QRhiTexture. I am not sure yet how to actually use these but I keep looking.

        I guess if textureHandle() already returns 0, chances are high I will not find anything in the QRhiTexture either. Does that mean there is no texture handle? Why is the call to map() necessary then?

        As far as my understanding goes, QMediaPlayer decodes the file and calls the specified QVideoSink object for every new frame. If the map() function is necessary to get a valid buffer via bits(), the frame data is not in RAM. I suppose a hardware decoder is used behind the scenes. This should result in a texture. Maybe that is not how the QMediaPlayer works, but I wonder how at least 60fps are achievable otherwise.

        M 1 Reply Last reply
        0
        • P Philerenz

          @Bonnie
          Thank you very much for your reply.

          It feels like I am one step closer, unfortunately textureHandle() returns 0. The other two return QRhi and QRhiTexture. I am not sure yet how to actually use these but I keep looking.

          I guess if textureHandle() already returns 0, chances are high I will not find anything in the QRhiTexture either. Does that mean there is no texture handle? Why is the call to map() necessary then?

          As far as my understanding goes, QMediaPlayer decodes the file and calls the specified QVideoSink object for every new frame. If the map() function is necessary to get a valid buffer via bits(), the frame data is not in RAM. I suppose a hardware decoder is used behind the scenes. This should result in a texture. Maybe that is not how the QMediaPlayer works, but I wonder how at least 60fps are achievable otherwise.

          M Offline
          M Offline
          mercotui
          wrote on last edited by mercotui
          #4

          I'm struggling with this as well, I did find an internal function that seems to do exactly what I want:

          https://github.com/qt/qtmultimedia/blob/0e3439d35917c61da016a94a4904ab7668486d0b/src/multimedia/video/qvideotexturehelper.cpp#L639

          But since it's private, and the private headers for Qt6 Multimedia don't seem to be available on Ubuntu 22.04, I can't really test it (without getting a custom qt-version).

          Waiting for a public API :'(

          1 Reply Last reply
          0
          • B Bonnie

            @Philerenz The related APIs have been changed and moved to private headers.
            You can access the private headers by adding (qmake) QT += multimedia-private in the .pro file or (cmake) include_directories(${Qt${QT_VERSION_MAJOR}Multimedia_PRIVATE_INCLUDE_DIRS}) in CMakeLists.txt
            And then in your source file adding

            #include <private/qabstractvideobuffer_p.h>
            

            You can get an QAbstractVideoBuffer instance by QVideoFrame::videoBuffer()

            In Qt5, QVideoFrame::handle() just returns QAbstractVideoBuffer::handle() of its buffer, in Qt6 there is no handle() function in QAbstractVideoBuffer but other functions like rhi(), textureHandle() or texture() are added, not sure if you can use those though.

            W Offline
            W Offline
            wrosecrans
            wrote on last edited by
            #5

            @Bonnie The sort of transitional state of Qt Multimedia is pretty entertaining when you try to read the documentation. It mentions things like "The return value of the handle() function" in the text but then doesn't document a handle() as existing in the Public Functions section.

            https://doc.qt.io/qt-6/qvideoframe.html

            It's all a bit dusty, and no longer really works the way the docs say, or how it was apparently intended,

            1 Reply Last reply
            0

            • Login

            • Login or register to search.
            • First post
              Last post
            0
            • Categories
            • Recent
            • Tags
            • Popular
            • Users
            • Groups
            • Search
            • Get Qt Extensions
            • Unsolved