Painting NV12-Formatted Images to QLabel



  • Hello,

    I'm working on an application that will feature a gstreamer video in the main Qt window. In my case Qt 5 is running in an embedded environment which requires me to use a gstreamer decoding plugin that outputs NV12-formatted frames only. Seeing as converting from NV12 to RGB is placing a large strain on my CPU, I'd like to display the frames in NV12. I've noticed that QVideoFrame works with NV12, but receiving a buffer in memory (from gstreamer) does not jive well with QVideoFrame's constructors. Is there a way to take an NV12 formatted image residing in memory and paint it to the screen with low latency?


  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Are you using gstreamer directly ?

    If so, you should consider the QtGStreamer module from the gstreamer folks.



  • I am using gstreamer directly (using the appsink plugin to get the frames from my NV12-only decoder). Would QtGstreamer be able to allow me to display NV12-formatted buffers on the display?


  • Lifetime Qt Champion

    AFAIR yes. They have several sinks between widget, OpenGL and QtQuick.



  • Okay, but what if I don't have those sinks that are mentioned in the QtGstreamer API? ("qtvideosink", "qtglvideosink" and "qwidgetvideosink" (or "qt5videosink", "qt5glvideosink" and "qwidget5videosink" in Qt5))


  • Lifetime Qt Champion

    Ho did you build Qt and the support for OpenSSL ?



  • ???

    I think this is the wrong post, my application has no need for SSL.


  • Lifetime Qt Champion

    Indeed, that answer was supposed to be for another thread. I wonder how it ended up here.

    Can you elaborate on the reasons you don't have these backends with QtGStreamer ?



  • Honestly QtGstreamer is not what I'm looking for. I'm looking to see if there's any Qt5 support for taking an NV12 frame, and painting it on the screen similar to a QImage. I know that there's QVideoFrame, but the images are coming in live, not from a file.


  • Lifetime Qt Champion

    Out of curiosity, what's the problem with QtGStreamer ?

    What kind of camera are you using?



  • The problem with QtGstreamer is simply that we already have a working standard Gstreamer plugin/program for our custom camera. The other problem is that it's a nonstandard processor that offloads all of its image decoding onto a co-processor, but that coprocessor sends back NV12-formatted decoded frames.


  • Lifetime Qt Champion

    Can you describe how your application is currently working ? And what is your problem with loading the NV12 data in QVideoFrame ?



  • I have a camera that I receive data from over a special co-processor, through a manufacturer specific gstreamer plugin. The decoded frame appears in a uchar array in NV12 format, and I would like to be able to paint that on a screen using QVideoFrame. Going through the documentation it appears that the best way to get a uchar* like this into QVideoFrame is through QAbstractVideoBuffer, but since this video is not in the framebuffer to begin with, QAbstractVideoBuffer doesn't want to latch onto it.

    I was wondering if there's a way to get this uchar array onto the screen.


  • Lifetime Qt Champion

    How are you using that gstreamer plugin ?



  • Just using good ol' gst_parse_launch. The key here is that the plugin does not support anything other than NV12, and colorspace changes on the processor using the gstreamer convert plugin to get to RGB are too processor intensive to be worth it. But the gstreamer plugin isn't negotiable. It's manufacturer and hardware specific.

    So that's why I'm wondering if I can get NV12-formatted frames to be painted on the screen.


  • Lifetime Qt Champion

    I'm not trying to avoid that plugin, just to understand how your setup is working.

    If using OpenGL is an option, QtMultimedia through Qt Quick should be able to show these frames. The backend is using gstreamer so depending on the gstreamer line you have to use you may have to tweak Qt's backend to do the same thing.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.