Drawing data over a video

LucaCiucci

Hi, I'm new on this forum.
In my project, I want to to display some preprocessed data over a video. Basically I want to do some markers tracking and then draw them over a video with some other informations. To do that I use OpenCV to analyze the video and then I use the QMediaPlayer and QGraphicsVideoItem to draw the video in a graphics scene.

My problem is how to synchronize the video with the data. In the beginning, I was using QMediaPlayer::position() but it has two problems:

the video is not perfectly sychronized with the time given by QMediaPlayer::position()
depending on the video codec, sometimes the QGraphicsVideoItem cannot keep up with the player speed and lags behind by several seconds (windows)

I found simple solution by connecting QVideoSink::videoFrameChanged and using the time information stored in the video frame, for example:

QVideoSink *const videoSink = m_videoItem->videoSink();
connect(videoSink, &QVideoSink::videoFrameChanged, this, &VideoScene::onVideoFrameChanged);

and then:

void VideoScene::onVideoFrameChanged(const QVideoFrame& frame)
{
    qint64 frame_pos_us = frame.startTime();
    ...
}

This gave me frame perfect synchronization but, unfortunately, QVideoFrame::startTime and QVideoFrame::endTime always return zero on MacOS (sometimes even on Windows but it was not reproducible).

So I want to ask if anyone has some more reliable solutions...

Some other notes:

I've found this other discussion but there was not a simple solution
I'm using OpenCV to do the video analysis, but the problem is independent of OpenCV, the problem is all in the Qt multimedia system
I'm considering throwing away the Qt multimedia system because it has some other problems, but, for now, I don't see any simple alternative

I hope I was clear, thanks in advance.

SGaist

Hi,

There's currently a backend in the works based on ffmpeg that should give you a more complete cross-platform support.

You might want to check it.