Video playback with ffmpeg backend does not work on the raspberry pi 3

[Jojojoris]

I found something interesting.

In another thread it was suggested to debug mesa. I got this error message:

GL_INVALID_VALUE in glTexImage2D(internalFormat=GL_R8)

So now i am trying to build qt with just gles2 support. And if that does not work, I might try to hardcode r8 formats as unsupported in qtbase.

[Jojojoris]

I noticed that src/multimedia/video/qvideotexturehelper.cpp defines R8 in some places.

Could it be that the unsupported texture format comes form here? And if yes, how to fix it.

[SGaist]

Hi,

Your spelunking looks good. You might want to take a look at the RHI part of Qt. Can you extract from ffmpeg which format is used ?

[Jojojoris]

@SGaist Do you mean like the output format of genoeg?

Because in that case it's probably NV12 from the top of my head.

[SGaist]

I would recommend to open a ticket on the bug report system.

AFAIR, NV12 is a supported format OpenGL wise and it looks like there is some conversion being done that seems out of place. I might be wrong though as I don't fully know the internals of the ffmpeg backend.

[Jojojoris]

@SGaist
So this journey got interesting.

So GL_R8 is not valid in gles2
And this is defined in:
src/multimedia/video/qvideotexturehelper.cpp

    // Format_NV12
    { 2, 1,
      [](int stride, int height) { return stride * ((height * 3 / 2 + 1) & ~1); },
     { QRhiTexture::R8, QRhiTexture::RG8, QRhiTexture::UnknownFormat },
     { { 1, 1 }, { 2, 2 }, { 1, 1 } }
    },

Changing R8 to RED_OR_ALPHA8 gets rid of that specific error message. Which now was replaces by a message that RG8 is also not supported.

By this time I tried to mark NV12 as invalid format in the ffmeg code. Now a YUV output was chosen. With this I did the same R8 to RED_OR_ALPHA8 modification.
Now I also had to change the shader src/multimedia/shaders/yuv_triplanar.frag to use the alpha component of the textures instead of the red component.

And now my raspberry pi can play video just fine.

Though these changes are unfortunately only a workaround for my specific situation.

[Jojojoris]

@Jojojoris
Added a bugreport
https://bugreports.qt.io/browse/QTBUG-114104

[SGaist]

@Jojojoris very nice ! Thanks for the detailed feedback !

[Andreas Hartmetz]

Hello @Jojojoris, I ran into the same problem on an i.MX6 DualLite with the etnaviv driver.
I could adjust qvideotexturehelper.cpp and the corresponding shaders without much trouble, but haven't figured out yet how (and where) to mark the NV12 format as invalid. Could you help me (and any others with the same issue) out there?
Thank you for your work so far, and I wish the Qt Company would consider the still-common i.MX6 as a platform worth supporting with some priority.

[Andreas Hartmetz]

Here's a complete patch for hacks according to @Jojojoris's investigation and changes to make video playback work with un-extended GLES2. I have only tested it with the GStreamer backend where it does work in my situation: VP8 video resulting in NV12 format, i.MX6 DualLite SoC, Linux 5.15.74, etnaviv driver from Mesa 22.1.7.

Since it isn't possible to paste files here, I'm pasting it as a code block. The file name would be: 0001-HACK-disable-texture-and-pixel-formats-that-don-t-wo.patch

From 7a194147152534efd213346771ce3af135e3df02 Mon Sep 17 00:00:00 2001
From: Andreas Hartmetz <andreas@ixgreen.de>
Date: Thu, 1 Aug 2024 14:10:27 +0200
Subject: [PATCH] HACK: disable texture and pixel formats that don't work with
 GLES2

Un-extended GLES2 doesn't support R8 and RG8 texture formats. For
R8, RHI's RED_OR_ALPHA8 is a suitable workaround resulting in
A[lpha]8, but for RG8, there is no simple workaround. To avoid the
need for RG8, crudely mark the NV12 and NV21 pixel formats whose
OpenGL pixel conversion shaders require it as unsupported in the
commonly used GStreamer and FFMPEG backends.
Symptoms without this hack, on i.MX6 DualLite / etnaviv: OpenGL
error messages about unsupported texture format R8 and the video
output is just solid green.

Mostly based on a post of user Jojojoris on forum.qt.io.
---
 src/multimedia/shaders/yuv_triplanar.frag     |  6 +++---
 src/multimedia/shaders/yuv_triplanar_p10.frag |  6 +++---
 src/multimedia/shaders/yvu_triplanar.frag     |  6 +++---
 src/multimedia/video/qvideotexturehelper.cpp  | 20 +++++++++----------
 .../multimedia/ffmpeg/qffmpegvideobuffer.cpp  |  9 +++++++++
 .../common/qgstvideorenderersink.cpp          |  6 ++++++
 6 files changed, 34 insertions(+), 19 deletions(-)

diff --git a/src/multimedia/shaders/yuv_triplanar.frag b/src/multimedia/shaders/yuv_triplanar.frag
index 5b83b05b5..0d3ab7a2c 100644
--- a/src/multimedia/shaders/yuv_triplanar.frag
+++ b/src/multimedia/shaders/yuv_triplanar.frag
@@ -6,20 +6,20 @@
 
 layout(location = 0) in vec2 texCoord;
 layout(location = 0) out vec4 fragColor;
 
 layout(binding = 1) uniform sampler2D plane1Texture;
 layout(binding = 2) uniform sampler2D plane2Texture;
 layout(binding = 3) uniform sampler2D plane3Texture;
 
 void main()
 {
-    float Y = texture(plane1Texture, texCoord).r;
-    float U = texture(plane2Texture, texCoord).r;
-    float V = texture(plane3Texture, texCoord).r;
+    float Y = texture(plane1Texture, texCoord).a;
+    float U = texture(plane2Texture, texCoord).a;
+    float V = texture(plane3Texture, texCoord).a;
     vec4 color = vec4(Y, U, V, 1.);
     fragColor = ubuf.colorMatrix * color * ubuf.opacity;
 
 #ifdef QMM_OUTPUTSURFACE_LINEAR
     fragColor = convertSRGBToLinear(fragColor);
 #endif
 }
diff --git a/src/multimedia/shaders/yuv_triplanar_p10.frag b/src/multimedia/shaders/yuv_triplanar_p10.frag
index bac34c72c..fbbed3506 100644
--- a/src/multimedia/shaders/yuv_triplanar_p10.frag
+++ b/src/multimedia/shaders/yuv_triplanar_p10.frag
@@ -6,20 +6,20 @@
 
 layout(location = 0) in vec2 texCoord;
 layout(location = 0) out vec4 fragColor;
 
 layout(binding = 1) uniform sampler2D plane1Texture;
 layout(binding = 2) uniform sampler2D plane2Texture;
 layout(binding = 3) uniform sampler2D plane3Texture;
 
 void main()
 {
-    float Y = texture(plane1Texture, texCoord).r * 64;
-    float U = texture(plane2Texture, texCoord).r * 64;
-    float V = texture(plane3Texture, texCoord).r * 64;
+    float Y = texture(plane1Texture, texCoord).a * 64;
+    float U = texture(plane2Texture, texCoord).a * 64;
+    float V = texture(plane3Texture, texCoord).a * 64;
     vec4 color = vec4(Y, U, V, 1.);
     fragColor = ubuf.colorMatrix * color * ubuf.opacity;
 
 #ifdef QMM_OUTPUTSURFACE_LINEAR
     fragColor = convertSRGBToLinear(fragColor);
 #endif
 }
diff --git a/src/multimedia/shaders/yvu_triplanar.frag b/src/multimedia/shaders/yvu_triplanar.frag
index 37fc3a18d..cfe406f2a 100644
--- a/src/multimedia/shaders/yvu_triplanar.frag
+++ b/src/multimedia/shaders/yvu_triplanar.frag
@@ -6,20 +6,20 @@
 
 layout(location = 0) in vec2 texCoord;
 layout(location = 0) out vec4 fragColor;
 
 layout(binding = 1) uniform sampler2D plane1Texture;
 layout(binding = 2) uniform sampler2D plane2Texture;
 layout(binding = 3) uniform sampler2D plane3Texture;
 
 void main()
 {
-    float Y = texture(plane1Texture, texCoord).r;
-    float V = texture(plane2Texture, texCoord).r;
-    float U = texture(plane3Texture, texCoord).r;
+    float Y = texture(plane1Texture, texCoord).a;
+    float V = texture(plane2Texture, texCoord).a;
+    float U = texture(plane3Texture, texCoord).a;
     vec4 color = vec4(Y, U, V, 1.);
     fragColor = ubuf.colorMatrix * color * ubuf.opacity;
 
 #ifdef QMM_OUTPUTSURFACE_LINEAR
     fragColor = convertSRGBToLinear(fragColor);
 #endif
 }
diff --git a/src/multimedia/video/qvideotexturehelper.cpp b/src/multimedia/video/qvideotexturehelper.cpp
index 3675e157f..0627cf556 100644
--- a/src/multimedia/video/qvideotexturehelper.cpp
+++ b/src/multimedia/video/qvideotexturehelper.cpp
@@ -88,97 +88,97 @@ static const TextureDescription descriptions[QVideoFrameFormat::NPixelFormats] =
     },
     // Format_AYUV_Premultiplied
     { 1, 4,
         [](int stride, int height) { return stride*height; },
         { QRhiTexture::RGBA8, QRhiTexture::UnknownFormat, QRhiTexture::UnknownFormat },
         { { 1, 1 }, { 1, 1 }, { 1, 1 } }
     },
     // Format_YUV420P
     { 3, 1,
       [](int stride, int height) { return stride * ((height * 3 / 2 + 1) & ~1); },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::R8 },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8 },
      { { 1, 1 }, { 2, 2 }, { 2, 2 } }
     },
      // Format_YUV422P
     { 3, 1,
       [](int stride, int height) { return stride * ((height * 3 / 2 + 1) & ~1); },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::R8 },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8 },
      { { 1, 1 }, { 2, 1 }, { 2, 1 } }
     },
      // Format_YV12
     { 3, 1,
       [](int stride, int height) { return stride * ((height * 3 / 2 + 1) & ~1); },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::R8 },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8 },
      { { 1, 1 }, { 2, 2 }, { 2, 2 } }
     },
     // Format_UYVY
     { 1, 2,
       [](int stride, int height) { return stride*height; },
      { QRhiTexture::RGBA8, QRhiTexture::UnknownFormat, QRhiTexture::UnknownFormat },
      { { 2, 1 }, { 1, 1 }, { 1, 1 } }
     },
     // Format_YUYV
     { 1, 2,
       [](int stride, int height) { return stride*height; },
      { QRhiTexture::RGBA8, QRhiTexture::UnknownFormat, QRhiTexture::UnknownFormat },
      { { 2, 1 }, { 1, 1 }, { 1, 1 } }
     },
     // Format_NV12
     { 2, 1,
       [](int stride, int height) { return stride * ((height * 3 / 2 + 1) & ~1); },
-     { QRhiTexture::R8, QRhiTexture::RG8, QRhiTexture::UnknownFormat },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RG8, QRhiTexture::UnknownFormat },
      { { 1, 1 }, { 2, 2 }, { 1, 1 } }
     },
     // Format_NV21
     { 2, 1,
       [](int stride, int height) { return stride * ((height * 3 / 2 + 1) & ~1); },
-     { QRhiTexture::R8, QRhiTexture::RG8, QRhiTexture::UnknownFormat },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RG8, QRhiTexture::UnknownFormat },
      { { 1, 1 }, { 2, 2 }, { 1, 1 } }
     },
     // Format_IMC1
     { 3, 1,
       [](int stride, int height) {
           // IMC1 requires that U and V components are aligned on a multiple of 16 lines
           int h = (height + 15) & ~15;
           h += 2*(((h/2) + 15) & ~15);
           return stride * h;
       },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::R8 },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8 },
      { { 1, 1 }, { 2, 2 }, { 2, 2 } }
     },
     // Format_IMC2
     { 2, 1,
       [](int stride, int height) { return 2*stride*height; },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::UnknownFormat },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::UnknownFormat },
      { { 1, 1 }, { 1, 2 }, { 1, 1 } }
     },
     // Format_IMC3
     { 3, 1,
       [](int stride, int height) {
           // IMC3 requires that U and V components are aligned on a multiple of 16 lines
           int h = (height + 15) & ~15;
           h += 2*(((h/2) + 15) & ~15);
           return stride * h;
       },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::R8 },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8 },
      { { 1, 1 }, { 2, 2 }, { 2, 2 } }
     },
     // Format_IMC4
     { 2, 1,
       [](int stride, int height) { return 2*stride*height; },
-     { QRhiTexture::R8, QRhiTexture::R8, QRhiTexture::UnknownFormat },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::RED_OR_ALPHA8, QRhiTexture::UnknownFormat },
      { { 1, 1 }, { 1, 2 }, { 1, 1 } }
     },
     // Format_Y8
     { 1, 1,
       [](int stride, int height) { return stride*height; },
-     { QRhiTexture::R8, QRhiTexture::UnknownFormat, QRhiTexture::UnknownFormat },
+     { QRhiTexture::RED_OR_ALPHA8, QRhiTexture::UnknownFormat, QRhiTexture::UnknownFormat },
      { { 1, 1 }, { 1, 1 }, { 1, 1 } }
     },
     // Format_Y16
     { 1, 2,
       [](int stride, int height) { return stride*height; },
      { QRhiTexture::R16, QRhiTexture::UnknownFormat, QRhiTexture::UnknownFormat },
      { { 1, 1 }, { 1, 1 }, { 1, 1 } }
     },
     // Format_P010
     { 2, 2,
diff --git a/src/plugins/multimedia/ffmpeg/qffmpegvideobuffer.cpp b/src/plugins/multimedia/ffmpeg/qffmpegvideobuffer.cpp
index 7b72a3cfe..d70e9cb9c 100644
--- a/src/plugins/multimedia/ffmpeg/qffmpegvideobuffer.cpp
+++ b/src/plugins/multimedia/ffmpeg/qffmpegvideobuffer.cpp
@@ -264,24 +264,33 @@ QVideoFrameFormat::PixelFormat QFFmpegVideoBuffer::toQtPixelFormat(AVPixelFormat
     case AV_PIX_FMT_YUV422P:
         return QVideoFrameFormat::Format_YUV422P;
     case AV_PIX_FMT_YUV420P:
         return QVideoFrameFormat::Format_YUV420P;
     case AV_PIX_FMT_YUV420P10:
         return QVideoFrameFormat::Format_YUV420P10;
     case AV_PIX_FMT_UYVY422:
         return QVideoFrameFormat::Format_UYVY;
     case AV_PIX_FMT_YUYV422:
         return QVideoFrameFormat::Format_YUYV;
+// The following two are disabled because the pixel format conversion shader for these requires the RG8 pixel
+// format, which is not supported in OpenGL ES2 without GL_EXT_texture_rg, which etnaviv does not currently
+// (July 2024) support. It will probably come with OpenGL ES3 support in etnaviv, which seems to be the focus
+// of its development. ES3 supports RG8 textures in the base specification.
+// (UNTESTED. It is possible that this crude approach breaks something or everything video-related. It might
+// be necessary to recognize these formats in situations other than "which Qt format should we choose for the
+// output buffer".)
+#if 0
     case AV_PIX_FMT_NV12:
         return QVideoFrameFormat::Format_NV12;
     case AV_PIX_FMT_NV21:
         return QVideoFrameFormat::Format_NV21;
+#endif
     case AV_PIX_FMT_GRAY8:
         return QVideoFrameFormat::Format_Y8;
     case AV_PIX_FMT_GRAY16:
         return QVideoFrameFormat::Format_Y16;
 
     case AV_PIX_FMT_P010:
         return QVideoFrameFormat::Format_P010;
     case AV_PIX_FMT_P016:
         return QVideoFrameFormat::Format_P016;
     case AV_PIX_FMT_MEDIACODEC:
diff --git a/src/plugins/multimedia/gstreamer/common/qgstvideorenderersink.cpp b/src/plugins/multimedia/gstreamer/common/qgstvideorenderersink.cpp
index d67319fdd..e6a572210 100644
--- a/src/plugins/multimedia/gstreamer/common/qgstvideorenderersink.cpp
+++ b/src/plugins/multimedia/gstreamer/common/qgstvideorenderersink.cpp
@@ -55,22 +55,28 @@ void QGstVideoRenderer::createSurfaceCaps()
 
     auto caps = QGstCaps::create();
 
     // All the formats that both we and gstreamer support
     auto formats = QList<QVideoFrameFormat::PixelFormat>()
                    << QVideoFrameFormat::Format_YUV420P
                    << QVideoFrameFormat::Format_YUV422P
                    << QVideoFrameFormat::Format_YV12
                    << QVideoFrameFormat::Format_UYVY
                    << QVideoFrameFormat::Format_YUYV
+// The following two are disabled because the pixel format conversion shader for these requires the RG8
+// pixel format, which is not supported in OpenGL ES2 without GL_EXT_texture_rg, which etnaviv does not
+// currently (July 2024) support. It will probably come with OpenGL ES3 support in etnaviv, which seems to
+// be the focus of its development. ES3 supports RG8 textures in the base specification.
+#if 0
                    << QVideoFrameFormat::Format_NV12
                    << QVideoFrameFormat::Format_NV21
+#endif
                    << QVideoFrameFormat::Format_AYUV
                    << QVideoFrameFormat::Format_P010
                    << QVideoFrameFormat::Format_XRGB8888
                    << QVideoFrameFormat::Format_XBGR8888
                    << QVideoFrameFormat::Format_RGBX8888
                    << QVideoFrameFormat::Format_BGRX8888
                    << QVideoFrameFormat::Format_ARGB8888
                    << QVideoFrameFormat::Format_ABGR8888
                    << QVideoFrameFormat::Format_RGBA8888
                    << QVideoFrameFormat::Format_BGRA8888
-- 
2.43.0