QVideoProbe not emitting videoFrameProbed signal

ImpossibleMushroom

I am attempting to use a QVideoProbe to get frames from a QMediaPlayer which then plays to a QVideoWidget for the user to see. I have tried example code from the documentation and my own code, neither appears to emit the videoFrameProbed event.

My Code:

Setting up the signals/slots and objects

	// Set up video probe (for processing frames)
	myVideoProbe = new QVideoProbe;
	connect(myVideoProbe, &QVideoProbe::videoFrameProbed, this, &MotionDetection::processVideoFrame);
	if (!myVideoProbe->setSource(myVideoPlayer)) {
		qDebug() << "Failed to set VideoProbe source.";
	}

	// Set up QVideoWidget
	myVideoWidget = new QVideoWidget();
	myVideoWidget->show();
	

	// Connect video player and video widget
	myVideoPlayer->setMedia(QUrl::fromLocalFile(currentVideoToProcess));
	myVideoPlayer->setVideoOutput(myVideoWidget);
	connect(myVideoPlayer, &QMediaPlayer::mediaStatusChanged, this, &MotionDetection::repeatVideo);

	// Play video
	myVideoPlayer->play();

The processVideoFrame slot

void MotionDetection::processVideoFrame(QVideoFrame frame)
{
	qDebug() << " Processing video frame";
}

edit
My MainWindow class

class MotionDetection : public QMainWindow
{
	Q_OBJECT

public:
	MotionDetection(QWidget *parent = Q_NULLPTR);
public slots:
	void chooseVideoFile();
	void repeatVideo(QMediaPlayer::MediaStatus newMediaStatus);
	void processVideoFrame(QVideoFrame frame);
private:
	QMediaPlayer* myVideoPlayer = nullptr;
	QVideoWidget* myVideoWidget = nullptr;
	QVideoProbe* myVideoProbe = nullptr;
	QVideoFrame* lastProcessedFrame = nullptr;

	QString currentVideoToProcess;
	Ui::MotionDetectionClass ui;
};

mrjj

Hi and welcome to the forums
What platform/Os and Qt version?
Does the video the play.
Can it generally play videos?

ImpossibleMushroom

Thanks, I am using Qt 5.12 on Windows 10. The video does play in the QVideoWidget I am using K-Lite Codec for MP4 Playback. I am compiling with Microsoft Visual Studios 2017 using the Qt Plugin/Extension.

mrjj

@ImpossibleMushroom
Ok. super.
and you check myVideoProbe->setSource so it should be supported by the backbone.
Im not sure what could be wrong.

if no suggestions show up. ( give it some hours, people are in different time zones)
then maybe
https://stackoverflow.com/questions/37724602/how-to-save-a-frame-using-qmediaplayer/37726742
can be used.

ImpossibleMushroom

@mrjj Okay, thank you! I have already looked into the possible option of using a QAbstractVideoSurface for this scenario due to this issue. I have been stuck on it all day and I as well have no clue what's wrong, I've tried with different video files, without setting a video output and many other small one line changes I can think of. I do hope for a solution involving the QVideoProbe still as using QAbstractVideoSurface would be unfavorable (but still doable) in my scenario.

mrjj

Hi
I have seen other posts where probe didn't fire.
I looked around
https://code.woboq.org/qt5/qtmultimedia/src/multimedia/video/qvideoprobe.cpp.html

but i did not any checks or conditions to explain why it would not.

You seem to check in the right places so i will assume code is right and it simply do not trigger.

ImpossibleMushroom

I have decided to use OpenCV instead for my image processing, it is more appropriate for in-depth image processing anyway, I believe. If anyone does find a solution to this problem later down the road please feel free to post it! I'm sure it'll help someone.

SGaist

Hi and welcome to devnet,

Not all backends support all feature. Here you have a matrix of the features per backend for Qt 5.11. I believe that it's still accurate for 5.12.

mnbv

I'm also having the same exact issue; I'm wondering if anybody ever found an explanation?

For me it's Qt 5.15.2, Windows 64-bit, MinGW 64-bit.

I am using the DirectShow back-end; which says it supports probing in the feature matrix. Also I am pretty sure the probe is being registered in the graph because at first I had been deleting the QVideoProbe incorrectly (not related to this issue) and the errors I received were in fact from the DShow back-end; so it at least made it into DShow.

SGaist

Hi,

5.15.4 being a commercial only release, you should ask the Qt Company directly.

Note that there has been a big overhaul of the Qt Multimedia module for Qt 6.2, you might want to check it as well.

mnbv

@SGaist 5.15.2! I meant 2!

jsulm

@mnbv said in QVideoProbe not emitting videoFrameProbed signal:

5.15.2! I meant 2!

This is Qt version which was used to build QtCreator! It is not necessarily the Qt version you're using!
Please check the Kit you are using to see what Qt version you're using.

mnbv

@jsulm It's 5.15.2.

Anyways, I never figured it out but needed to get something working so I kludged one out of a QAbstractVideoSurface, and it's been working well so far. It's all in one file.

VideoProbeSurface.h (link is to Gist)

#ifndef VIDEOPROBESURFACE_H
#define VIDEOPROBESURFACE_H

#include <QAbstractVideoSurface>
#include <QVideoSurfaceFormat>

/* A bit of a hack, but... 
 *
 * Example:
 *
 * QMediaPlayer *player = ...;
 *
 * QVideoWidget *widget = ...;
 *
 * VideoProbeSurface *probe = new VideoProbeSurface(...);
 * probe->setFormatSource(widget->videoSurface());
 * connect(probe, &VideoProbeSurface::videoFrameProbed, ...);
 *
 * player->setVideoOutput({ widget->videoSurface(), probe });
 */

class VideoProbeSurface : public QAbstractVideoSurface {
    Q_OBJECT
public:
    VideoProbeSurface (QObject *parent = nullptr)
        : QAbstractVideoSurface(parent)
        , formatSource_(nullptr)
    {
    }
    void setFormatSource (QAbstractVideoSurface *source) {
        formatSource_ = source;
    }
    QList<QVideoFrame::PixelFormat> supportedPixelFormats (QAbstractVideoBuffer::HandleType type) const override {
        return formatSource_ ? formatSource_->supportedPixelFormats(type)
                             : QList<QVideoFrame::PixelFormat>();
    }
    QVideoSurfaceFormat nearestFormat (const QVideoSurfaceFormat &format) const override {
        return formatSource_ ? formatSource_->nearestFormat(format)
                             : QAbstractVideoSurface::nearestFormat(format);
    }
    bool present (const QVideoFrame &frame) override {
        emit videoFrameProbed(frame);
        return true;
    }
signals:
    void videoFrameProbed (const QVideoFrame &frame);
private:
    QAbstractVideoSurface *formatSource_;
};

#endif // VIDEOPROBESURFACE_H

I went for the quickest-to-write implementation possible so it just forwards supported pixel formats from another surface (my intent was to both probe and play back to a QVideoWidget) and you get whatever format you get. I'm just grabbing subimages into QImages anyways though so it handles whatever format. But you could modify this to force any formats you want.

Example usage:

 QMediaPlayer *player = ...;
 QVideoWidget *widget = ...;

 // forward surface formats provided by the video widget:
 VideoProbeSurface *probe = new VideoProbeSurface(...);
 probe->setFormatSource(widget->videoSurface());

 // same signal signature as QVideoProbe's signal:
 connect(probe, &VideoProbeSurface::videoFrameProbed, ...);

 // the key move is to render to both the widget (for viewing) and probe (for processing).
 // fortunately, QMediaPlayer takes a list:
 player->setVideoOutput({ widget->videoSurface(), probe });

The only really sketchy thing I had to do was const_cast the QVideoFrame on the receiver side (for read-only access), since QVideoFrame::map() isn't const:

    if (const_cast<QVideoFrame&>(frame).map(QAbstractVideoBuffer::ReadOnly)) {
        ...;
        const_cast<QVideoFrame&>(frame).unmap();
    }

But the real QVideoProbe would make you do the same thing so, I don't know what's up with that -- it's a strange API. I ran some tests with sw, native hw, and copy-back hw renderers and decoders and map/unmap in read mode seem to be functioning OK, so, whatever.

Performance-wise, the video will bog down if you spend too much time in the callback, so design accordingly. However, I didn't test QueuedConnection, so I don't know if that'd still have the issue (although the fact that the signal parameter is a reference would make me wary of trying it, as well as conceivable issues with the GPU releasing the memory before the slot ends up being called). I don't know how QVideoProbe behaves in this regard. I do know that, at least on my machine, I can pack and queue HD-resolution QImages to a thread pool for processing without slowing down the video.

You probably also want to implement some sort of auto-unmapper utility object for exception safe unmap(), etc. But again, that's not unique to this, same thing you'd have to do with QVideoProbe.

So hopefully that helps somebody else.

Would still really like to know why the real thing doesn't work, though...