Best way to add a new pixel format (to work with QVideoFrameFormat and/or QVideoFrame)?

paulpv

This is a follow up to my question https://forum.qt.io/post/727612

My app (https://github.com/NightVsKnight/NDI-Monitor/tree/main/ndi-monitor-qt6) is receiving P216 pixel formatted NDI video frames.
QVideoFrameFormat does not list P216 as a supported format.

https://doc.qt.io/qt-6/qvideoframeformat.html#PixelFormat-enum

I naively used P016 thinking it might work.
It almost works.

The NDI Processing.NDI.structs.h and documentation describe P216 as:

	// YCbCr color space using 4:2:2 in 16bpp.
	// In memory this is a semi-planar format. This is identical to a 16bpp version of the NV16 format.
	// The first buffer is a 16bpp luminance buffer.
	// Immediately after this is an interleaved buffer of 16bpp Cb, Cr pairs.

So, NV16 is an alias for P216 (or vice versa).

I have also seen P216 referred to as YUV422P16 (as opposed to 8bit YUV422P), but I don't know if that is accurate.

I am just going to call it P216, which it or its aliases are described here:

Following the same pattern as NV12/NV21, there are NV16/NV61 (4:2:2 sub-sampling) and NV24/NV42 (4:4:4 sampling) formats. They are mostly used in some funky cheap camera circuitry and not supported in VLC (as of VLC version 2.0).

The first link diagrams the P016 as:

And the P216 as:

That H/2 difference makes P016 not suitable.

So, unless there is a simple and efficient trick to convert P216 4:2:2 to P016 4:2:0, I am wondering:
What the best way is to add a new pixel format support to QVideoFrame and/or QVideoFrameFormat?

Can I subclass QVideoFrameFormat? I don't really think that [alone] will help much.
I am looking at the git blames for when other pixel formats were added or removed from the code, and it seems like this requires adding shader code too:
Example: https://github.com/qt/qtmultimedia/blob/85cf52390c5d8183820367f84f31b2a1182b224c/src/multimedia/shaders/p010.frag
(was removed and consolidated in to NV12 in https://github.com/qt/qtmultimedia/commit/b72e11072ef484fe2f1a54267cae81a50ff24b57#diff-45acde77e2c2b14ab3470d603471f3a6e45b78254577d90de51d89df07b25237 with the comment "P010/P016 have the same layout as NV12, just 16 instead of 8 bits per pixel")
It seems to me that if all of the code is there to support P016/NV12, which I naively think is just a bitshift of P216/NV16, it would not be very hard to add P216/NV16 support.

Should I open a ticket to have this added? What is the realistic tiimeline of that happening?

Should I just learn how to code it myself (I suspect basically implement NV16) and open a PR?

Also: What is QPixelFormat (https://doc.qt.io/qt-6/qpixelformat.html) used for? It might be nice to be able to define a custom pixel format in QPixelFormat and then be able to pass that to QVideoFrameFormat and/or QVideoFrame.

Thanks!

Pv

paulpv

As I mentioned in my other thread, I am able to unblock myself with a poor man's P216->P016 conversion of scanning every other UV line:

It still leaves a few small artifacts laying around, but it is passable while I continue this conversation of getting P216 added to Qt.

KH-219Design

"looking at the git blames for when other pixel formats were added or removed"

Yeah... that is how I would go about it, too. After reading just the subject of this thread, I came here to suggest that, but now I see that you are already doing that.

I guess one other idea would be to try the "Development" mailing list: https://lists.qt-project.org/