from QCamera or QVideoWidget get video bytes

zabitqt

Hi, How can I get ARRAY OF BYTES (frame) from QCamera or QVideoWidget? I successfull capture video from camera and audio from microphone and now I want send they to socket tcp channel for creating a simple video audio call software.

SGaist

Hi,

Depending on your platform, you could use QVideoProbe.

That said, dumping the raw content of your camera on the network is not a good idea, you are going to kill your bandwidth. There are libraries around that are designed for that kind of activities, you should check them.

zabitqt

Ok, with QVideoProbe I have access to QVideoFrame. QVideoFrame bits() method return ( char * ). I want array of bytes, How Can I do? casting char * in QByteArray() ?

jsulm

@zabitqt said in from QCamera or QVideoWidget get video bytes:

I want array of bytes

char* is basically an array of bytes in C/C++.
If you want QByteArray then use https://doc.qt.io/qt-6/qbytearray.html#QByteArray-1

ChrisW67

Since some of the bytes in the video frame can be zero you will need to pass the size of the char* buffer to the QByteArray constructor @jsulm linked. See QVideoFrame::mappedBytes().

zabitqt

@jsulm ok thank you so much. Mean that char * is format for video frame, not bytes.

zabitqt

QVideoProbe* probe = new QVideoProbe;
qDebug() << "lo stato di probe";
qDebug() << probe->setSource(recorder);
QObject::connect(probe, SIGNAL(videoFrameProbed(QVideoFrame)), this, SLOT(processFrame(QVideoFrame)));

The function probe->setSource(recorder); return always false. Why?

jsulm

@zabitqt said in from QCamera or QVideoWidget get video bytes:

qDebug() << probe->setSource(recorder);

What exactly is recorder here and was it initialised?

zabitqt

@jsulm

QCamera* mCamera = new QCamera();
QMediaRecorder* recorder = new QMediaRecorder(mCamera);
QVideoEncoderSettings settings = recorder->videoSettings();
settings.setResolution(640, 480);
settings.setQuality(QMultimedia::VeryHighQuality);
settings.setFrameRate(30.0);
settings.setCodec("video/mp4");
recorder->setVideoSettings(settings);
recorder->setContainerFormat("mp4");
mCamera->setCaptureMode(QCamera::CaptureVideo);
recorder->record();
mCamera->start();

Could be that If the media recorder instance does not support monitoring video, this function will return false?

jsulm

@zabitqt said in from QCamera or QVideoWidget get video bytes:

Could be that If the media recorder instance does not support monitoring video, this function will return false?

According to documentation: "If the media recorder instance does not support monitoring video, this function will return false"

zabitqt

Ok but Is there an alternative? It's impossible I can't get bytes from camera and send via QTcpSocket.

Help me please

Bonnie

Take a look at https://wiki.qt.io/Qt_5.13_Multimedia_Backends, QVideoProbe needs the backend to support "Video probe" feature.
As listed in the tables, recording of Windows and Unix doesn't support it, but player and camera (except wmf) do support.
So I think it may be possible to monitor the camera object directly, but you'll need to handle the media encoding yourself.

zabitqt

@Bonnie hi, I also ask to stackoverflow

https://stackoverflow.com/questions/74058405/capture-qvideoframe-and-display-it-in-qvideowidget

Can you help me

Bonnie

I don't know much about QAbstractVideoSurface.
Did you try using QVideoProbe to monitor the QCamera object as I said?
If that works then there's no need to create your custom video surface to get frames.

zabitqt

@Bonnie said in from QCamera or QVideoWidget get video bytes:

QVideoProbe

I test QVideoProbe but unfortunately return false
qDebug() << "return" << prob->setSource(recorder);

QCamera* mCamera = new QCamera();
QMediaRecorder* recorder = new QMediaRecorder(mCamera);
QVideoEncoderSettings settings = recorder->videoSettings();
QVideoProbe* prob = new QVideoProbe();

settings.setResolution(640, 480);
settings.setQuality(QMultimedia::VeryHighQuality);
settings.setFrameRate(30.0);
settings.setCodec("video/mp4");
recorder->setVideoSettings(settings);
recorder->setContainerFormat("mp4");
mCamera->setCaptureMode(QCamera::CaptureVideo);

Bonnie

@zabitqt Haven't you read my above reply? I said to monitor the camera object, not the recorder, because that's unsupported on Windows and Unix...

zabitqt

Ok now return true!

but it display
QVideoFrame(QSize(640, 480), Format_YUYV, NoHandle, ReadOnly, [no timestamp])
image base64: ""

I don't understand. I can pay you for consulting If you resolve me problem

Bonnie

@zabitqt I don't do consulting jobs haha. What do you not understand? How to get bytes from QVideoFrame?

zabitqt

void QtVideoWidgetsIssueTrack::processFrame(const QVideoFrame& frame) {
    qDebug() << "processFrame";
    if (frame.isValid()) {
        QVideoFrame cloneFrame(frame);

        cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
        qDebug() << cloneFrame;

        const QImage image(cloneFrame.bits(),
            cloneFrame.width(),
            cloneFrame.height(),
            QVideoFrame::imageFormatFromPixelFormat(cloneFrame.pixelFormat()));

        QByteArray ba;
        QBuffer bu(&ba);
        bu.open(QBuffer::ReadWrite);
        //bu.open(QIODevice::WriteOnly);
        image.save(&bu, "PNG");
        //bu.close();
        //QString imgBase64 = ba.toBase64();
        QString imgBase64 = QString::fromLatin1(ba.toBase64().data());
        qDebug() << "image base64: " << imgBase64;

        cloneFrame.unmap();
    }
}

Bonnie

@zabitqt Oh, you can't convert it to QImage like this, as your debug info shows, the frame data you get from the camera is in YUV format, which QImage doesn't support.
If you want to get a QImage, just call frame.image(), you don't even need to clone, map and unmap...