Qt6 QVideoSink Not Getting Frames from V4L2 Webcam

FestivalPanda

I'm currently writing a C++ Qt App that takes a webcam feed and applies some processing to each frame before rending the processed frame to the app. The problem I'm getting is that when I start my Qt App, whether the VideoSink receives any frames to process is wildly inconsistent -- either it works completely fine, or I don't receive a single frame for the lifetime of the app. No middle ground. It's been making it really hard to get work done and for some reason only seems to be happening on my machine.

MainWindow::MainWindow(QWidget* parent)
    : QWidget(parent) {
    // Layout
    QHBoxLayout* mainLayout = new QHBoxLayout(this);

    // Video Display Widget
    m_videoStream = new QLabel(this);
    QImage blackImage(640, 480, QImage::Format_RGB888);
    blackImage.fill(QColor(Qt::black));
    m_videoStream->setPixmap(QPixmap::fromImage(blackImage));
    mainLayout->addWidget(m_videoStream);

    // Camera Setup
    m_camera = new QCamera(this);
    for (const QCameraDevice& device : QMediaDevices::videoInputs()) {
        if (device.description() == "Depstech webcam (V4L2)")
            m_camera->setCameraDevice(device);
    }
    m_imageCapture = new QImageCapture(this);

    // Create video sink to intercept video frames
    m_videoSink = new QVideoSink(this);

    // Media Session Setup
    m_captureSession.setCamera(m_camera);
    m_captureSession.setImageCapture(m_imageCapture);
    m_captureSession.setVideoOutput(m_videoSink);
    m_camera->start();

    // Make connections
    connect(m_videoSink, &QVideoSink::videoFrameChanged, this, &MainWindow::processFrame);
}

Things I noticed

Other webcam based apps on my computer such as "Cheese" do not run into this issue.

Regardless of whether I called m_camera->start();, the camera feed still appeared just as often as when I exclude it entirely from my code. Even if the camera is NOT active, I'll still be receiving frames and rendering them.

When the frames are not being received, my webcam's light doesn't even turn on.

Things I've Tried

Switching cameras back and forth does not seem to help.

Connecting another camera (HuddleCamHD [V4L2]) instead, did not help.

My brain went straight to "race-condition", but adding a few seconds delay when initializing the capture session or when connecting the VideoSink::videoFrameChanged signal, did not help.

Debug Prints

Debug Call	Result
`QT_VERSION_STR`	6.2.4
`m_camera->cameraDevice().description()`	Depstech webcam (V4L2)
`m_camera->errorString()`	`""`
`m_camera->isAvailable()`	`true`
`m_camera->isActive()`	`true`
`qt.multimedia.imageCapture`: cameraActiveChanged	`false false`
`qt.multimedia.videooutput`: sinkChanged	`videoSinkBin`
`qt.multimedia.imageCapture`: cameraActiveChanged	`false true`
`qt.multimedia.imageCapture`: isReady	`true`

On working runs, the debug print will proceed to print qt.multimedia.gstvideorenderer messages. These messages are NOT printed if the webcam isn't turning on.


qt.multimedia.gstvideorenderer: set_caps: "video/x-raw, format=(string)xBGR, width=(int)3840, height=(int)2160, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)0:1:0:0, framerate=(fraction)30/1"
qt.multimedia.gstvideorenderer: QGstVideoRenderer::start "video/x-raw, format=(string)xBGR, width=(int)3840, height=(int)2160, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)0:1:0:0, framerate=(fraction)30/1"
qt.multimedia.gstvideorenderer: QGstVideoRenderer::render
qt.multimedia.gstvideorenderer: QGstVideoRenderer::handleEvent(renderBuffer) true QGstreamerVideoSink(0x62f0569cda00)
qt.multimedia.gstvideorenderer:     sending video frame

Environment Details

I'm running PopOS (Ubuntu 22.04) on a Lenovo Thinkpad

JoeCFD

@FestivalPanda said in Qt6 QVideoSink Not Getting Frames from V4L2 Webcam:

6.2.4

QMultimedia module was basically reworked from Qt6.
Qt 6.2.4 might be too new. Better to try the latest Qt6.

Also better to use gstreamer + qml sink directly. The example is here:
https://github.com/GStreamer/gst-plugins-good/blob/master/tests/examples/qt/qmlsink/main.cpp