Preprocessing images from Camera before Viewfinder



  • Hello!
    So i have a camera and i want to do these simple steps

    1. preprocess every frame from camera

    2. display preprocessed

    3. paint some lines and text over it

    i guess i should use QGraphicsVideoItem for diplaying and draw over it
    and QAbstractVideoFilter can help me to process each frame

    But i can't find any way to combine these two


  • Lifetime Qt Champion

    Hi,

    What kind of pre-processing do you have in mind ?

    QtQuick might be a better choice depending on what you want to render.



  • Yeah, I switched to QML
    For now i have a problem when format of QVideoFrame is YUYV and QImage doesn't support it.

    I managed to load image directly from QVideoFrame into opencv::Mat but i can't return proper QVideoFrame from
    my implementation of

    QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {
    

    because as far as i understand I need to return QVideoFrame in YUYV but i can't create it cause QImage doesn't support it


  • Lifetime Qt Champion

    What operation do you want to apply to the video frame ?



  • Right now i have a very strange situation
    ex1:
    please look at code

    class MyFilterRunnable : public QVideoFilterRunnable {
    public:
        QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {
    
            if (input->isValid()) {
                //reading from QVideoFrame
                QVideoFrame cloneFrame(*input);
                cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
    
                cv::Mat mat(cloneFrame.height(), cloneFrame.width(), CV_8UC2, cloneFrame.bits(), cloneFrame.bytesPerLine());
    
                cv::Mat converted_image = cv::Mat(mat);
                //converting to RGB
                cvtColor(mat, converted_image, CV_YUV2RGB_YUYV);
                //drawing dummy circle just as an example
                cv::circle(converted_image, cv::Point(100, 100), 35, cv::Scalar(0,0,255), 2, CV_AA );
                //constructing image
                QImage imageToReturn = QImage((uchar*)converted_image.data, converted_image.cols, converted_image.rows, QImage::Format_RGB888);
    
                //closing video frame mapping
                cloneFrame.unmap();
                imageToReturn.save("file.bmp","bmp",100);
    
                auto videoFrameToReturn = QVideoFrame(imageToReturn);
    
                qDebug() << v.isValid(); // always prints "true"
                return videoFrameToReturn;
    }
    

    So as you can see i'm creating an cv::Mat object drawing plain circle and packing it into QImage.
    After that i'm creating QVideFrame from QImage and **the view in QML is grey **
    But!
    In code number 2
    Everything shows as it should be

    class MyFilterRunnable : public QVideoFilterRunnable {
    public:
        QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {
    
            if (input->isValid()) {
    
    //            tmp.save("file.bmp","bmp",100);
    
                QImage imageToReturn = QImage("file.bmp","bmp");
    
                auto videoFrameToReturn = QVideoFrame(imageToReturn);
    }
    

    I even can write to file my QImage, instantly read it into another QImage and everything works


  • Lifetime Qt Champion

    In your second code, you don't do anything to input so I'm not sure what you are expecting there.



  • Ok.
    I have found a solution

    QImage imageToReturn = QImage((uchar*)converted_image.data, converted_image.cols, converted_image.rows, QImage::Format_RGB888);
    
    imageToReturn.convertToFormat(QImage::Format_RGB32);
    

    I have found that when i started to compare metadata between QImage constructed from cv::Mat data and QImage constructed as in second example


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.