After convertion QVideoFrame to QImage, the QImage is zoomed and streched

[Melle_87]

Hi all,
I am using Raspberry pi 4 with Raspbian Buster and qt15.12.7.
Now I am using a usb webcam. I use Camera and VideoOutput with a filter.
In the filter I am converting the QVideoFrame to a QImage.
For checking I save the QImage.
The image has 640x480 as expected, but it shows only a cutout (top left corner) of the image.
The image looks like it is zoomed and streched.
The preview of the camera stream in GUI looks good.

Any ideas what could be the problem here?

[jsulm]

@Melle_87 How do you show the image?
Please show the code.

[Melle_87]

This post is deleted!

[Melle_87]

@jsulm:

Rectangle {
        id: videoOutput
        z:99
        color: Style.buttonBackground
        VideoOutput {
            id: vid
            source: camera
            filters: [ camera.filt ]
            visible: true
            anchors.fill: parent
        }
    }

    Camera {
        id: camera
        deviceId: "/dev/video0"
        viewfinder.resolution: "640x480"
        viewfinder.maximumFrameRate: 10
        viewfinder.minimumFrameRate: 10
        exposure.exposureMode: Camera.ExposureManual
        exposure.manualAperture: 16
        exposure.manualShutterSpeed: 1/100
        exposure.manualIso: -1
    }

    MyVideoFilter {
        id: myFilter
    }

This is the live stream in the GUI:

And this is the saved image after convertion:

I have to mention that everything has worked fine on the Raspberry pi 3 with Raspbian Stretch.

[GrecKo]

The important part of your code is the QVideoFrame to QImage conversion.

[Melle_87]

@GrecKo:

QByteArray data;
QSize size;
QVideoFrame::PixelFormat pixelFormat;
int bytesPerLine;

void copyData(QVideoFrame & frame)
{
    frame.map(QAbstractVideoBuffer::ReadOnly);

    if(data.size() != frame.mappedBytes()) {
        data.resize(frame.mappedBytes());
    }

    memcpy(data.data(), frame.bits(), frame.mappedBytes());
    size = frame.size();
    pixelFormat = frame.pixelFormat();
    bytesPerLine = frame.bytesPerLine();

    frame.unmap();
}

QVideoFrame MyVideoFilterRunnable::run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags)
{  
    Q_UNUSED(surfaceFormat);
    Q_UNUSED(flags);

    if(!input || !input->isValid())
        return *input;

    m_filter->m_frame.copyData(*input);
    m_filter->m_processThread = QtConcurrent::run(this, &MyVideoFilterRunnable::processVideoFrame, m_filter->m_frame);

    return * input;
}

void MyVideoFilterRunnable::processVideoFrame(MyVideoFrame &videoFrame)
{
    QImage img((const uchar*)(videoFrame.data.data()), videoFrame.size.width(), videoFrame.size.height(), videoFrame.bytesPerLine, QImage::Format_Indexed8);
    img.save("/path/img.png");
    ...
}

[Melle_87]

@GrecKo: Do you have any ideas what could be the problem here?

[GrecKo]

Since you are copying raw data from the QVideoFrame to the QImage, I think there is format issue.

I guess QImage::Format_Indexed8 is the wrong format. Have you check the pixel format of the QVideoFrame?

[Melle_87]

@GrecKo: Pixel format of the QVideoFrame is QVideoFrame::Format_YUYV (21). But this is not available when I am creating my QImage. Which pixel format do I have to use instead of QImage::Format_Indexed8?

[GrecKo]

Some ideas here : https://stackoverflow.com/questions/27829830/convert-qvideoframe-to-qimage

[SGaist]

Hi,

To add to @GrecKo, YUYV is not directly mappable to an indexed image format.

What exactly are you trying to achieve ?

[fcarney]

@SGaist This talks about converting YUV to RGB. Is this the same as UYUV?

[SGaist]

YUV is a color space as is RGB, YUYV is one of the many possible ways to store YUV data like you have RGB, BGR, etc.

[Melle_87]

@SGaist: I want to detect barcodes. Therefore I use ZBar and QZXing.
For both I need a data of a QImage. That is the reason why I have to convert the QVideoFrame to QImage.

[Melle_87]

@SGaist: But what could be the explanation that everything was fine on Raspberry Pi 3 with Raspbian Stretch and now I have problems on Raspberry Pi 4 with Raspbian Buster?

[GrecKo]

@Melle_87 said in After convertion QVideoFrame to QImage, the QImage is zoomed and streched:

@SGaist: But what could be the explanation that everything was fine on Raspberry Pi 3 with Raspbian Stretch and now I have problems on Raspberry Pi 4 with Raspbian Buster?

My guess is that the Pi 3 was providing frames in a different format.

[GrecKo]

@Melle_87 said in After convertion QVideoFrame to QImage, the QImage is zoomed and streched:

@SGaist: I want to detect barcodes. Therefore I use ZBar and QZXing.
For both I need a data of a QImage. That is the reason why I have to convert the QVideoFrame to QImage.

QZXing is supposed to handle YUYV fine in its video filters : https://github.com/ftylitak/qzxing/blob/master/src/QZXingFilter.cpp#L325

[Melle_87]

@GrecKo: Thank for the link. I tried to register QZXingFilter in main.cpp:

qmlRegisterType<QZXingFilter>("MY.QZXingFilter", 1, 0, "MyQZXingFilter");

Then I importet it in my qml file, instatiated it and assigned to my VideoOutput:

import MY.QZXingFilter 1.0

MyQZXingFilter {
    id: myBarcodeFilter
}

My_Camera {
    id: camera
}
   
VideoOutput {
    id: vid
    source: camera
    filters: [ myBarcodeFilter ]   
}

But now I get some errors.

/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `int qmlRegisterType<QZXingFilter>(char const*, int, int, char const*)':
/home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtQml/qqml.h:311: undefined reference to `QZXingFilter::staticMetaObject'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QQmlPrivate::QQmlElement<QZXingFilter>::QQmlElement()':
/home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtQml/qqmlprivate.h:103: undefined reference to `QZXingFilter::QZXingFilter(QObject*)'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QtPrivate::MetaObjectForType<QZXingFilter*, void>::value()':
/home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtCore/qmetatype.h:1476: undefined reference to `QZXingFilter::staticMetaObject'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QMetaTypeIdQObject<QZXingFilter*, 8>::qt_metatype_id()':
/home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtCore/qmetatype.h:1841: undefined reference to `QZXingFilter::staticMetaObject'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x8): undefined reference to `QZXingFilter::metaObject() const'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0xc): undefined reference to `QZXingFilter::qt_metacast(char const*)'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x10): undefined reference to `QZXingFilter::qt_metacall(QMetaObject::Call, int, void**)'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x38): undefined reference to `QZXingFilter::createFilterRunnable()'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTIN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTIN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x8): undefined reference to `typeinfo for QZXingFilter'
/opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QQmlPrivate::QQmlElement<QZXingFilter>::~QQmlElement()':
/home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtQml/qqmlprivate.h:108: undefined reference to `QZXingFilter::~QZXingFilter()'

Do you know what I am doing wrong?

[jsulm]

@Melle_87 said in After convertion QVideoFrame to QImage, the QImage is zoomed and streched:

MyQZXingFilter

Is it derived from QObject?

[Melle_87]

@jsulm: It is derived from QAbstractVideoFilter.