Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

After convertion QVideoFrame to QImage, the QImage is zoomed and streched



  • Hi all,
    I am using Raspberry pi 4 with Raspbian Buster and qt15.12.7.
    Now I am using a usb webcam. I use Camera and VideoOutput with a filter.
    In the filter I am converting the QVideoFrame to a QImage.
    For checking I save the QImage.
    The image has 640x480 as expected, but it shows only a cutout (top left corner) of the image.
    The image looks like it is zoomed and streched.
    The preview of the camera stream in GUI looks good.

    Any ideas what could be the problem here?


  • Lifetime Qt Champion

    @Melle_87 How do you show the image?
    Please show the code.



  • This post is deleted!


  • @jsulm:

    Rectangle {
            id: videoOutput
            z:99
            color: Style.buttonBackground
            VideoOutput {
                id: vid
                source: camera
                filters: [ camera.filt ]
                visible: true
                anchors.fill: parent
            }
        }
    
        Camera {
            id: camera
            deviceId: "/dev/video0"
            viewfinder.resolution: "640x480"
            viewfinder.maximumFrameRate: 10
            viewfinder.minimumFrameRate: 10
            exposure.exposureMode: Camera.ExposureManual
            exposure.manualAperture: 16
            exposure.manualShutterSpeed: 1/100
            exposure.manualIso: -1
        }
    
        MyVideoFilter {
            id: myFilter
        }
    

    This is the live stream in the GUI:
    LiveStreamInGui.png

    And this is the saved image after convertion:
    SaveImageAfterConvertion.png

    I have to mention that everything has worked fine on the Raspberry pi 3 with Raspbian Stretch.


  • Qt Champions 2018

    The important part of your code is the QVideoFrame to QImage conversion.



  • @GrecKo:

    QByteArray data;
    QSize size;
    QVideoFrame::PixelFormat pixelFormat;
    int bytesPerLine;
    
    void copyData(QVideoFrame & frame)
    {
        frame.map(QAbstractVideoBuffer::ReadOnly);
    
        if(data.size() != frame.mappedBytes()) {
            data.resize(frame.mappedBytes());
        }
    
        memcpy(data.data(), frame.bits(), frame.mappedBytes());
        size = frame.size();
        pixelFormat = frame.pixelFormat();
        bytesPerLine = frame.bytesPerLine();
    
        frame.unmap();
    }
    
    QVideoFrame MyVideoFilterRunnable::run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags)
    {  
        Q_UNUSED(surfaceFormat);
        Q_UNUSED(flags);
    
        if(!input || !input->isValid())
            return *input;
    
        m_filter->m_frame.copyData(*input);
        m_filter->m_processThread = QtConcurrent::run(this, &MyVideoFilterRunnable::processVideoFrame, m_filter->m_frame);
    
        return * input;
    }
    
    void MyVideoFilterRunnable::processVideoFrame(MyVideoFrame &videoFrame)
    {
        QImage img((const uchar*)(videoFrame.data.data()), videoFrame.size.width(), videoFrame.size.height(), videoFrame.bytesPerLine, QImage::Format_Indexed8);
        img.save("/path/img.png");
        ...
    }
    


  • @GrecKo: Do you have any ideas what could be the problem here?


  • Qt Champions 2018

    Since you are copying raw data from the QVideoFrame to the QImage, I think there is format issue.

    I guess QImage::Format_Indexed8 is the wrong format. Have you check the pixel format of the QVideoFrame?



  • @GrecKo: Pixel format of the QVideoFrame is QVideoFrame::Format_YUYV (21). But this is not available when I am creating my QImage. Which pixel format do I have to use instead of QImage::Format_Indexed8?


  • Qt Champions 2018


  • Lifetime Qt Champion

    Hi,

    To add to @GrecKo, YUYV is not directly mappable to an indexed image format.

    What exactly are you trying to achieve ?



  • @SGaist This talks about converting YUV to RGB. Is this the same as UYUV?


  • Lifetime Qt Champion

    YUV is a color space as is RGB, YUYV is one of the many possible ways to store YUV data like you have RGB, BGR, etc.



  • @SGaist: I want to detect barcodes. Therefore I use ZBar and QZXing.
    For both I need a data of a QImage. That is the reason why I have to convert the QVideoFrame to QImage.



  • @SGaist: But what could be the explanation that everything was fine on Raspberry Pi 3 with Raspbian Stretch and now I have problems on Raspberry Pi 4 with Raspbian Buster?


  • Qt Champions 2018

    @Melle_87 said in After convertion QVideoFrame to QImage, the QImage is zoomed and streched:

    @SGaist: But what could be the explanation that everything was fine on Raspberry Pi 3 with Raspbian Stretch and now I have problems on Raspberry Pi 4 with Raspbian Buster?

    My guess is that the Pi 3 was providing frames in a different format.


  • Qt Champions 2018

    @Melle_87 said in After convertion QVideoFrame to QImage, the QImage is zoomed and streched:

    @SGaist: I want to detect barcodes. Therefore I use ZBar and QZXing.
    For both I need a data of a QImage. That is the reason why I have to convert the QVideoFrame to QImage.

    QZXing is supposed to handle YUYV fine in its video filters : https://github.com/ftylitak/qzxing/blob/master/src/QZXingFilter.cpp#L325



  • @GrecKo: Thank for the link. I tried to register QZXingFilter in main.cpp:

    qmlRegisterType<QZXingFilter>("MY.QZXingFilter", 1, 0, "MyQZXingFilter");
    

    Then I importet it in my qml file, instatiated it and assigned to my VideoOutput:

    import MY.QZXingFilter 1.0
    
    MyQZXingFilter {
        id: myBarcodeFilter
    }
    
    My_Camera {
        id: camera
    }
       
    VideoOutput {
        id: vid
        source: camera
        filters: [ myBarcodeFilter ]   
    }
    

    But now I get some errors.

    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `int qmlRegisterType<QZXingFilter>(char const*, int, int, char const*)':
    /home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtQml/qqml.h:311: undefined reference to `QZXingFilter::staticMetaObject'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QQmlPrivate::QQmlElement<QZXingFilter>::QQmlElement()':
    /home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtQml/qqmlprivate.h:103: undefined reference to `QZXingFilter::QZXingFilter(QObject*)'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QtPrivate::MetaObjectForType<QZXingFilter*, void>::value()':
    /home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtCore/qmetatype.h:1476: undefined reference to `QZXingFilter::staticMetaObject'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QMetaTypeIdQObject<QZXingFilter*, 8>::qt_metatype_id()':
    /home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtCore/qmetatype.h:1841: undefined reference to `QZXingFilter::staticMetaObject'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x8): undefined reference to `QZXingFilter::metaObject() const'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0xc): undefined reference to `QZXingFilter::qt_metacast(char const*)'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x10): undefined reference to `QZXingFilter::qt_metacall(QMetaObject::Call, int, void**)'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTVN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x38): undefined reference to `QZXingFilter::createFilterRunnable()'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o:(.data.rel.ro._ZTIN11QQmlPrivate11QQmlElementI12QZXingFilterEE[_ZTIN11QQmlPrivate11QQmlElementI12QZXingFilterEE]+0x8): undefined reference to `typeinfo for QZXingFilter'
    /opt/raspi-tools-8.3.0/lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: main.o: in function `QQmlPrivate::QQmlElement<QZXingFilter>::~QQmlElement()':
    /home/m/Builds/build_PY-Raspi_Buster-Debug/../../../x/workspace/gateway-setup/qt-for-raspi4-root-buster-pi3-vc4-tc5/qt5pi/include/QtQml/qqmlprivate.h:108: undefined reference to `QZXingFilter::~QZXingFilter()'
    

    Do you know what I am doing wrong?


  • Lifetime Qt Champion

    @Melle_87 said in After convertion QVideoFrame to QImage, the QImage is zoomed and streched:

    MyQZXingFilter

    Is it derived from QObject?



  • @jsulm: It is derived from QAbstractVideoFilter.


  • Lifetime Qt Champion

    @Melle_87 Then I guess you do not build/link moc_*.cpp file for MyQZXingFilter

    undefined reference to `QZXingFilter::QZXingFilter(QObject*)' - actually you seem to not link QZXingFilter at all


  • Qt Champions 2018

    How did you add QZXing to your project?
    https://github.com/ftylitak/qzxing#howToInclude has instructions on how to do it



  • @GrecKo: Okay. Now I am able to use the QZXingFilter in VideoOutput.
    But the conversion from QVideoFrame to QImage is not successful.
    QZXingFilter says:

    QZXingFilterRunnable error: Cant create image file to process.
    Maybe it was a format conversion problem? 
    VideoFrame format:  Format_YUYV
    Image corresponding format:  QImage::Format_Invalid
    

    What should I do now???



  • It was my fault. I pulled the latest version of QZXing. Now the conversion from QVideoFrame to QImage is working.


Log in to reply