Cannot map QVideoFrame in a QVideoFilterRunnable on Linux with Qt 5.11.1

Stevenm

Hi,

I am trying to use QVideoFilter on an image coming from my webcam on Linux.

Here is my filter's implementation :

QVideoFrame FaceFilter::Runnable::run(QVideoFrame *input, const QVideoSurfaceFormat &,
                                      QVideoFilterRunnable::RunFlags )
{
    if( input->isValid() && input->map(QAbstractVideoBuffer::ReadWrite))
    {

        auto ptr = input->bits();
 
       //... Do some Read/Write operations on ptr ...

        input->unmap();
    }
    return *input;
}

This filter implementation works very well on Windows. However, on Linux (Ubuntu 18.04), I am getting this error each time I try to map the frame in Read/Write or Write only mode :

(QtCam:30044): GStreamer-CRITICAL **: 18:27:11.763: write map requested on non-writable buffer

On Linux, I have this problem with Qt 5.9.5, installed with apt-get, and with Qt 5.11.1 I have compiled by myself.

Here are some informations about gstreamer plugins versions I have installed :

$apt list gstream*

Listing... Done
gstreamer-qapt/bionic,now 3.0.4-0ubuntu1 amd64 [installed]
gstreamer0.10-qapt/bionic,bionic 3.0.4-0ubuntu1 all
gstreamer1.0-alsa/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-clutter-3.0/bionic,now 3.0.26-1 amd64 [installed,automatic]
gstreamer1.0-crystalhd/bionic 1:0.0~git20110715.fdd2f19-12 amd64
gstreamer1.0-doc/bionic-updates,bionic-updates 1.14.1-1~ubuntu18.04.1 all
gstreamer1.0-espeak/bionic 0.4.0-3 amd64
gstreamer1.0-fluendo-mp3/bionic 0.10.32.debian-1 amd64
gstreamer1.0-gl/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed,automatic]
gstreamer1.0-gtk3/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-libav/bionic-updates,now 1.14.1-1~ubuntu18.04.1 amd64 [installed,automatic]
gstreamer1.0-libav-dbg/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-nice/bionic 0.1.14-1 amd64
gstreamer1.0-omx-bellagio-config/bionic 1.12.4-1 amd64
gstreamer1.0-omx-generic/bionic 1.12.4-1 amd64
gstreamer1.0-omx-generic-config/bionic 1.12.4-1 amd64
gstreamer1.0-opencv/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-packagekit/bionic-updates 1.1.9-1ubuntu2.18.04.3 amd64
gstreamer1.0-plugins-bad/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed]
gstreamer1.0-plugins-bad-dbg/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-plugins-bad-doc/bionic-updates,bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 all
gstreamer1.0-plugins-base/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed]
gstreamer1.0-plugins-base-apps/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed]
gstreamer1.0-plugins-base-dbg/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-plugins-base-doc/bionic-updates,bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 all
gstreamer1.0-plugins-good/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed]
gstreamer1.0-plugins-good-dbg/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-plugins-good-doc/bionic-updates,bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 all
gstreamer1.0-plugins-ugly/bionic-updates,now 1.14.1-1~ubuntu18.04.1 amd64 [installed]
gstreamer1.0-plugins-ugly-dbg/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-plugins-ugly-doc/bionic-updates,bionic-updates 1.14.1-1~ubuntu18.04.1 all
gstreamer1.0-pocketsphinx/bionic 0.8.0+real5prealpha-1ubuntu2 amd64
gstreamer1.0-pulseaudio/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed]
gstreamer1.0-python3-dbg-plugin-loader/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-python3-plugin-loader/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-qt5/bionic-updates 1.14.1-1ubuntu1~ubuntu18.04.1 amd64
gstreamer1.0-rtsp/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-rtsp-dbg/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-tools/bionic-updates,now 1.14.1-1~ubuntu18.04.1 amd64 [installed,automatic]
gstreamer1.0-vaapi/bionic-updates 1.14.1-1~ubuntu18.04.1 amd64
gstreamer1.0-vaapi-doc/bionic-updates,bionic-updates 1.14.1-1~ubuntu18.04.1 all
gstreamer1.0-x/bionic-updates,now 1.14.1-1ubuntu1~ubuntu18.04.1 amd64 [installed]

Here is a small project I've made that shows the problem :

main.cpp : https://pastebin.com/DjeSNZXt
facefilter.cpp : https://pastebin.com/KWMn1xLi
facefilter.h : https://pastebin.com/3SwJEVAh
main.qml : https://pastebin.com/UhCrdk3h
QtCamera.pro : https://pastebin.com/yUgZGVik
ressources.qrc : https://pastebin.com/DHi0eyhB

Thanks in advance for your help !
Steven

SGaist

Hi and welcome to devnet,

Not a direct answer but did you try to operate on a copy of the input video frame ?

Stevenm

Hi ! Thanks for your answer !

Mapping the frame doesnt fail in ReadOnly mode and I can create a copy of the frame.

Actually, it's not a really good copy because the format of the frame is QVideoFrame::Format_YUYV. I just tried to put the Y component into the R component of an RGB Qimage and I can recognise my face :)
-> So yes, accessing the data works !

Steven

SGaist

In that case, why not convert it to RGB and then do the writing you want on that version ?

Stevenm

Yes, I think that's what I'll eventually do ... But I was really hoping to use this mechanism to do some detections, modify the image and display an "augmented" image into the qml videooutput. It seemed perfect for that.

Any clue about the reason why the problem occurs on Linux and not on Windows ?
Is gstreamer used in the windows compiled version ?

SGaist

No, gstreamer is the Linux backend. On Windows, IIRC it's DirectX for the camera access.