QVideoSink::videoFrameChanged not fired when dragging application window

BrianL

I have an Qt6 application that simply opens a camera with QCamera in QMediaCaptureSession, use QVideoSink::videoFrameChanged to trigger a paintEvent to draw the current frame.

Everything is working correctly, but then I notice if I drag the application window, it starts dropping frames. This does not happen if I resize the window. Is this a known issue or there are ways to get around this issue?

Below are a stripped version of my code:
.h file

class VideoManager : public QWidget
{
    Q_OBJECT

public:
    VideoManager();

protected:
    void paintEvent(QPaintEvent* event) override;

private slots:
    void OnFrameChanged(QVideoFrame const& frame);

private:
    void Start();
    void Stop();

private:
    QMediaCaptureSession m_capture;
    QCamera m_camera;
    QVideoSink m_sink;
    QImage m_frame;
};

.cpp file

VideoManager::VideoManager()
{
    m_capture.setCamera(&m_camera);
    m_capture.setVideoSink(&m_sink);
    connect(&m_sink, &QVideoSink::videoFrameChanged, this, &VideoManager::OnFrameChanged);
}

void VideoManager::Start()
{
    for (const QCameraDevice &device : QMediaDevices::videoInputs())
    {
            for (auto const& format : device.videoFormats())
            {
                    m_camera.setCameraDevice(device);
                    m_camera.setCameraFormat(format);
                    m_camera.start();

                    this->show();
                    return;
                }
        }
}

void VideoManager::paintEvent(QPaintEvent *event)
{
    QPainter painter(this);
    painter.drawImage(this->rect(), m_frame);
}

void VideoManager::OnFrameChanged(const QVideoFrame &frame)
{
    // I used this to check when I drag the window, this debug stops, which means the signal wasn't fired
    qDebug() << frame.startTime(); 

    m_frame = frame.toImage();
    this->update();
}

Nils Petter Skalerud

This sounds like a bug to me. We'd need more information about your environment, platform and Qt version in order to investigate. Could you please file a bug report at the following bug tracker? https://bugreports.qt.io/

Please include information such include information such as:

Platform (WIndows, Linux, Android...)
Qt version
Qt Multimedia media backend being used (FFmpeg, gstreamer, native)
If it's a mobile or embedded device, please also include the model description.

BrianL

@Nils-Petter-Skalerud It seems like I am not authorized to create a bug, it asked me to log in but I was already logged in.

But here are the requested information:

Platform: Windows
Qt Version: 6.10.0
Qt Multimedia media backend: I'm unsure about this one, is there somewhere I can check this?

jsulm

@BrianL said in QVideoSink::videoFrameChanged not fired when dragging application window:

Qt Multimedia media backend: I'm unsure about this one, is there somewhere I can check this?

You can set QT_DEBUG_PLUGINS env variable before starting your app and see which plug-ins are loaded.

BrianL

@jsulm

qt.core.plugin.factoryloader: checking directory path "C:/Qt6/6.10.0/mingw_64/plugins/multimedia" ...
qt.core.plugin.factoryloader: looking at "ffmpegmediaplugin.dll"
qt.core.plugin.loader: Found metadata in lib C:/Qt6/6.10.0/mingw_64/plugins/multimedia/ffmpegmediaplugin.dll, metadata=
{
    "IID": "org.qt-project.Qt.QPlatformMediaPlugin",
    "MetaData": {
        "Keys": [
            "ffmpeg"
        ]
    },
    "archlevel": 2,
    "className": "QFFmpegMediaPlugin",
    "debug": false,
    "version": 395776
}

BrianL

Update: it only starts lagging if the camera frame is big enough then it will lag when I drag the window, if I make the window smaller it doesn't lag

jsulm

@BrianL said in QVideoSink::videoFrameChanged not fired when dragging application window:

C:/Qt6/6.10.0/mingw_64/plugins/multimedia/ffmpegmediaplugin.dll

You're using ffmpeg backend