Get frames from a video
-
So, this seems like a really simple question, but I can't figure out how to just open up a video file and get at the frames using QtMultimedia. I am currently using a library based on ffmpeg called ffms for the task, but I would like to have video functionality in my app without needing 3rd party libraries, and using the native backends, so QtMultimedia seems perfect in theory.
I don't want to make a video player app. I already have one of those installed, works fine. For some reason, that seems to be exactly what all the video stuff in QtMultimedia is designed for. Having the API handle timing and drawing on screen for me is completely useless, I really just want to get the frames and take it from there. What's the best way to go about this?
-
maybe "this":http://qt-project.org/doc/qt-5.0/qtmultimedia/videooverview.html#working-with-low-level-video-frames will help you to get started with.
And also see "QVideoFrame":http://qt-project.org/doc/qt-5.0/qtmultimedia/qvideoframe.html#details -
Well, it's certainly a starting point, but it doesn't seem to cover everything. I tried making a subclass of QAbstractVideoSurface which pauses a QMediaPlayer in the surface::present(), but this seems to explode gstreamer on Linux, and makes my test app hang. The QMediaPlayer documentation doesn't really give any details about when calling pause would be permitted/dangerous, so I'm not sure what the behavior would be on other platforms with different backends. My next experiment was based on emitting a signal in present(), but that did the same thing.
My most recent attempt sets a QTimer::singleShot(0, this, SLOT(haltVideo())); which pauses the QMediaPlayer. This is the closest I have gotten, but every time I play() it again to get the next frame, my surface present()'s a frame and then the next frame, so I am getting two frames presented with each step of one frame forward in time. It's not clear if I am actually decoding frames doubly, or if it is just presenting the previous frame from a cache and then decoding the new frame. It's also not clear that I can use QMediaPlayer to get faster than real time access to the video frames. My hunt for some sample code showing the "sanctioned" way to just get a frame from a video continues... Grr.
-
Well first you have to know that Qt offers nothing to retrieve video frames in a flexing way.
The best thing you can do is to use QVideoProbe to do post frame analysis of a QMediaObject (for instance played with a QMediaPlayer or QCamera). But you won't be able to analyse a video faster than how it play (changing the playback rate can increase the process though).
You can also use QAbstractVideoSurface to display the frames playing with a QMediaPlayer or QCamera.
With this you also have to know that all multimedia platform depends on a multimedia backend plugin that is related to the platform you develop and depends on audio/video codecs. For instance QAudioDecoder is available under Linux but not for windows xp. So be careful if you are developping a cross-platform application. You can have more details here:
http://qt-project.org/wiki/Qt_Multimedia_Backends
http://qt-project.org/doc/qt-5.0/qtmultimedia/multimediabackend.htmlAccording to the readmap video encoding/decoding and cross-platform multimedia backend will come later.
http://qt-project.org/wiki/Qt_MultimediaFor now the best solution I see is to use ffmpeg. Here is a project that looks interesting.
https://code.google.com/p/qtffmpegwrapper/source/browse/#svn/trunk/QTFFmpegWrapper
If you are working with remoted video then libvlc can also be a solution as it offers a lot of networking features.
