Android: QAbstractVideoSurface not triggered by QMediaPlayer to grab frames from video

johnc

I am getting a similar issue.

Putting in the frame.handle() call in my present() method kick started it, but nothing is painting.

I think I must be hitting the raw frame issue as outlined in one of the bug reports.

In the paint function it fails on the frame.map() function. It seems that on Android it is unable to copy the raw frame data over to accessible memory.

Does anyone know how to get around this issue? We need raw access so we can paint to a specific control, and also run a Qr reader over the frames.

The code runs fine on iOS so the error is definitely just Android specific.

MartynW

Supposedly you will be able to get the frame using QVideoProbe in 5.2.1 to be released beginning of Feb time (?) or the proper fix in 5.3 end of April time. Meantime it doesn't look possible (see http://qt-project.org/forums/viewthread/36973/) . Unless someone knows differently!

MartynW

Yes, QVideoProbe works fine in 5.2.1 (don't forget to call frame.map()) and returns image frames in NV21 format, altho the orientation does not change and is at 90 degrees when in portrait mode. It gives consecutive frames even when you don't call frame.handle() in the VideoSurface (which is best as the callbacks don't return anything useful).

and.fo

Grabbing frames is still not working for me on Android.

Android version is 4.4.3. Qt version 5.2.1 as Qt 5.3. crashes for another reason.

Either with a custom QAbstractVideoSurface or the QVideoProbe there is no callback triggered to "present" or "videoFrameProbed".

In case of QVideoProbe the following function returns false on Android and Windows7.

@
probe.setSource(&m_player)}
@

How did you manage it? Is the frame grabbing working on Android?

MartynW

Yes, it works fine for Windows an Android. I vaguely remember setSource returning false at one time, I think I just got the camera name wrong. I show a snippet of what I have below in the hope it may help.

BTW, I think the 'proper' fix never made it into 5.3.

@ QVideoProbe *m_pProbe;

// Search for system camera devices: We pick the first if any specified one is not found
foreach (const QByteArray &deviceName, QCamera::availableDevices())
{
    QString description = m_pCamera->deviceDescription(deviceName);
    if (cameraDevice.isEmpty())
        cameraDevice = deviceName;

    if (description.compare(strCameraName, Qt::CaseInsensitive) == 0)
        cameraDevice = deviceName;

    qDebug() << "Camera: " << description; 
}

// Example device on Windows: "@device:pnp:\\?\usb#vid_0c45&pid_6409&mi_00#7&373d688d&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"
// Example device on Nexus 7 2012: "front", description "Front-facing camera"

if (cameraDevice.isEmpty())
    m_pCamera = new QCamera;
else
    m_pCamera = new QCamera(cameraDevice);

m_pConsumerVideoSurface = new ConsumerVideoSurface(this);
m_pCamera->setViewfinder(m_pConsumerVideoSurface);
connect(m_pConsumerVideoSurface, SIGNAL(signalOnFrame(QImage)), this , SLOT(slotOnFrame(QImage)), Qt::DirectConnection);
if (bIsVideoProbeRequired)
{
    m_pProbe = new QVideoProbe(this);
    m_pConsumerVideoSurface->SuppressVideo(true);
}


m_pProbe = new QVideoProbe(this);
if (m_pProbe != NULL)
{
    m_pProbe->setSource(m_pCamera); // Returns true, hopefully.
    connect(m_pProbe, SIGNAL(videoFrameProbed(QVideoFrame)), this, SLOT(slotOnProbeFrame(QVideoFrame)), Qt::DirectConnection);
}
m_pCamera->start();

@

and.fo

Thanks a lot. I will have a try :-)

But in may application I'm not grabbing frames from the camera but from the mediaplayer. I'm working on frames from a video playback file.

Maybe this is not working on Android? Do you have any experience with that?

MartynW

Sorry, no, I was only interested in the camera, but I thought QVideoProbe was meant to be working generally so I would expect you to be ok.

and.fo

Thanks again!

I found this post, mentioning QVideoProbe is working only for the camera on Android, not for the MediaPlayer.

http://lists.qt-project.org/pipermail/interest/2014-February/011125.html

MartynW

It seems to be a bit of a work in progress with changes proposed then delayed - see "https://bugreports.qt-project.org/browse/QTBUG-35416":https://bugreports.qt-project.org/browse/QTBUG-35416.

johnc

Is there any further update to this or solution?

The bug reports all seem to say to use QVideoProbe instead, but I can't get that to work on Android ( 4.1.2 ) and QT 5.4.
If I leave out setViewFinder I get QCamera::InvalidRequestError

Putting setViewFinder in removes all errors, the camera changes state from Loading, to Loaded, to Active.
But then after this no frames are captured.

I have seen a number of examples like this one http://www.programering.com/a/MTN3IjMwATQ.html
But all seem to have the same result, no frames.