How to modify Camera Example to be able to process the video frames?

morkia

I am trying to modify the Camera Example so I can access each frame and do some processing (face detection, draw a rectangle around the face) before displaying it. I read that I have to subclass the QAbstractVideoSurface class and pass it as viewfinder but the viewfinder used in the example is from QCameraViewFinder class which is a subclass of QVideoWidget. I can't simply replace it as the ui compiler will complain about a couple of methods not being available. what needs to be exactly done? do I have to subclass both of the classes QAbstractVideoSurface and QVideoWidget? there's no documentation or example available on this. Any help and suggestion is greatly appreciated.

SGaist

Hi,

What do you want to use for face detection ?
If OpenCV then you should do the capture and processing with it and then display the result with Qt.

morkia

Actually the problem is how to get the frames from the camera. As you can see in that example a widget is promoted to QCameraViewFinder and it's used in designer. I tried to replace it with a class say FaceDetector which I subclassed from QAbstractVideoSurface but the ui compiler could not place it in a stacked layout complaining it's not a QWidget. I tried modifying the QCameraViewFinder which is subclassed from QVideoWidget but there's no way to get the frames from a QVideoWidget as I understand it's only for displaying the frames. So I want to know what should I replace that promoted widget with so I can achieve my objective. do I have to sublass both QVideoWidget and QAbstractVideoSurface together? I have already successfully done this that is detection of the face using OpenCV and Dlib in another application but now I want to make a GUI application with that ability. The closest example I could find was the Camera Example supplied with Qt. So all I need to do is get the frames, do my processing and display it. I read in a couple of examples online and here in this forum that I need to subclass QAbstractVideoSurface and implement the present() method. But I don't know how to replace the viewFinder of the said example by my class. I hope I've been able to explain clearly. Thank you for reading my question.

Pablo J. Rogina

@morkia I think the way to go is subclassing QAbstractVideoFilter, check the details.
One thing though, all the examples I saw so far use QML as the fornt end (i.e. to camera viewfinder)

Kent-Dorfman

Just a dumb WAG, but are you sure that you are reimplementing all the pure virtuals in the QAbstract classes you are using? Shouldn't even compile if not.

morkia

@Pablo-J-Rogina is there a way to do it in pure C++? I also can see more examples in QML than C++ but this shouldn't be impossible. Also the examples supplied by Qt are all in QML but one which I have mentioned in my earlier post.

Pablo J. Rogina

@morkia said in How to modify Camera Example to be able to process the video frames?:

is there a way to do it in pure C++?

I really don't know. It's a big question for me why the QAbstractVideoFilter is implemented in C++ but all the examples I know come from QML side.

You may want to take a look at QAbstractVideoFilter source code , where you'll find references to QVideoFrame and QAbstractVideoSurface as well.

wrosecrans

@morkia One of the gotchas you may run into is that QAbstractVideoSurface can potentially represent either a buffer in GPU memory, or a buffer in CPU memory, depending on the platform and circumstances. So there may not actually be an image you can copy out of it and into your face detection code.