Preprocessing images from Camera before Viewfinder

sergboec

Hello!
So i have a camera and i want to do these simple steps

preprocess every frame from camera
display preprocessed
paint some lines and text over it

i guess i should use QGraphicsVideoItem for diplaying and draw over it
and QAbstractVideoFilter can help me to process each frame

But i can't find any way to combine these two

SGaist

Hi,

What kind of pre-processing do you have in mind ?

QtQuick might be a better choice depending on what you want to render.

sergboec

Yeah, I switched to QML
For now i have a problem when format of QVideoFrame is YUYV and QImage doesn't support it.

I managed to load image directly from QVideoFrame into opencv::Mat but i can't return proper QVideoFrame from
my implementation of

QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {

because as far as i understand I need to return QVideoFrame in YUYV but i can't create it cause QImage doesn't support it

SGaist

What operation do you want to apply to the video frame ?

sergboec

Right now i have a very strange situation
ex1:
please look at code

class MyFilterRunnable : public QVideoFilterRunnable {
public:
    QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {

        if (input->isValid()) {
            //reading from QVideoFrame
            QVideoFrame cloneFrame(*input);
            cloneFrame.map(QAbstractVideoBuffer::ReadOnly);

            cv::Mat mat(cloneFrame.height(), cloneFrame.width(), CV_8UC2, cloneFrame.bits(), cloneFrame.bytesPerLine());

            cv::Mat converted_image = cv::Mat(mat);
            //converting to RGB
            cvtColor(mat, converted_image, CV_YUV2RGB_YUYV);
            //drawing dummy circle just as an example
            cv::circle(converted_image, cv::Point(100, 100), 35, cv::Scalar(0,0,255), 2, CV_AA );
            //constructing image
            QImage imageToReturn = QImage((uchar*)converted_image.data, converted_image.cols, converted_image.rows, QImage::Format_RGB888);

            //closing video frame mapping
            cloneFrame.unmap();
            imageToReturn.save("file.bmp","bmp",100);

            auto videoFrameToReturn = QVideoFrame(imageToReturn);

            qDebug() << v.isValid(); // always prints "true"
            return videoFrameToReturn;
}

So as you can see i'm creating an cv::Mat object drawing plain circle and packing it into QImage.
After that i'm creating QVideFrame from QImage and **the view in QML is grey **
But!
In code number 2
Everything shows as it should be

class MyFilterRunnable : public QVideoFilterRunnable {
public:
    QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {

        if (input->isValid()) {

//            tmp.save("file.bmp","bmp",100);

            QImage imageToReturn = QImage("file.bmp","bmp");

            auto videoFrameToReturn = QVideoFrame(imageToReturn);
}

I even can write to file my QImage, instantly read it into another QImage and everything works

SGaist

In your second code, you don't do anything to input so I'm not sure what you are expecting there.

sergboec

Ok.
I have found a solution

QImage imageToReturn = QImage((uchar*)converted_image.data, converted_image.cols, converted_image.rows, QImage::Format_RGB888);

imageToReturn.convertToFormat(QImage::Format_RGB32);

I have found that when i started to compare metadata between QImage constructed from cv::Mat data and QImage constructed as in second example