QML Camera and Video Probe (probe not working)

[Trayon]

I'm trying to use a QML camera to get the raw data from a webcam and send it to a server for image recognition. I'm using a video probe for this purpose, but the probe doesn't seem to emit the signal I need to process the frame.

(I've posted what I'm trying to do before, but my solutions weren't cross-platform, and failed pretty hard on Android)

Here's my code:

FrameHandler.cpp:

#include "framehandler.h"

FrameHandler::FrameHandler(QQmlApplicationEngine *engine)
{

    QCamera *camera;
    QObject *qmlCamera = engine->rootObjects().at(0)->findChild<QObject*>("qmlCam");
    QVideoProbe probe;

    camera = qvariant_cast<QCamera*>(qmlCamera->property("mediaObject"));

    //QObject::connect(&probe,SIGNAL(videoFrameProbed(QVideoFrame)),this,SLOT(handleFrame(QVideoFrame)));
    connect(&probe,&QVideoProbe::videoFrameProbed,this,&FrameHandler::handleFrame);
    probe.setSource(camera);
}

void FrameHandler::handleFrame(QVideoFrame frame)
{
    qDebug() << "In handleFrame";
}

main.cpp:

#include <QGuiApplication>
#include "framehandler.h"

int main(int argc, char *argv[])
{
    QGuiApplication app(argc, argv);
#if defined(Q_OS_WIN)
    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
#endif

    QQmlApplicationEngine engine;
    engine.load(QUrl(QStringLiteral("qrc:/main.qml")));

    FrameHandler frameHandler(&engine);


    return app.exec();
}

main.qml:

import QtQuick 2.9
import QtQuick.Controls 1.2
import QtMultimedia 5.3
import "qrc:/"

ApplicationWindow {
  visible: true
  width: 640
  height: 480
  title: qsTr("QML Camera Demo")

  Camera {
    id: camera
    objectName: "qmlCam"

    imageCapture {
      onImageCaptured: {
        photoPreview.source = preview
        photoPreview.visible = true;
        previewTimer.running = true;
      }
    }
  }

  VideoOutput {
    id: viewfinder
    source: camera
    anchors.fill: parent
  }

  Image {
    id: photoPreview
    anchors.fill: viewfinder
  }

  Timer {
    id: previewTimer
    interval: 2000
    onTriggered: photoPreview.visible = false;
  }

  CaptureButton {
    anchors.right: parent.right
    anchors.verticalCenter: parent.verticalCenter
    diameter: 50
  }
}

[SGaist]

Hi,

What version of Qt are you using ?
On what platform ?
With which compiler ?

[Trayon]

5.9.3
Linux
GCC-64

[SGaist]

QVideoProbe is not available on all platforms. See this matrix.

Do you want to do the face recognition on one specific image ?

[Trayon]

The charts don't seem to correspond to operating system directly (except maybe Android). I didn't see any QVideoProbe class there. Is it a subclass of one of these?

Also, it should work according to this source:

https://stackoverflow.com/questions/28041741/qt-qml-camera-to-c-qimage-on-android/33238150

The solution that Waldez Junior proposed apparently works on Android. I also tried it on Windows with no luck. Also, I may sound naive asking this, but why don't the QT developers put tests that check the OS and tell the programmer that his work is in vain?

[Trayon]

Also, yes. One specific image.

[Trayon]

Whoops, made a noob mistake and didn't declare the objects in the header file, so they went out of scope and got deleted. Fixed that, but now the frames getting fed in aren't filled with any data (frame.mappedBytes = 0). I also tried inheriting from an AbstractVideoSurface to see if I could get the data from the 'present' function, but it's not even getting inside (it's now declared in main).

Here's how I have my main.cpp:

#include <QGuiApplication>
#include "framehandler.h"
#include "cameraframegrabber.h"

int main(int argc, char *argv[])
{
    QGuiApplication app(argc, argv);
#if defined(Q_OS_WIN)
    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
#endif

    QQmlApplicationEngine engine;
    engine.load(QUrl(QStringLiteral("qrc:/main.qml")));

    QObject *qmlCamera = engine.rootObjects().at(0)->findChild<QObject*>("qmlCam");

    QCamera *camera = qvariant_cast<QCamera*>(qmlCamera->property("mediaObject"));

    CameraFrameGrabber *cameraFrameGrabber = new CameraFrameGrabber();
    camera->setViewfinder(cameraFrameGrabber);

    //camera->start(); // I tried this too, but it didn't help.

    return app.exec();
}

[SGaist]

IIRC, on Android you get a texture.

[Trayon]

This is on Linux. Is it the same there?

[Trayon]

Has anyone tried to use the media probe or abstract media surface inheritance to get the information? Has anyone succeeded beofore?