How to get ROI of QVideoFrame?

[MHermann]

@jsulm:
No image is shown in qml. It is black.
If I don't edit *input, than the image is shown correctly.

[jsulm]

@MHermann said in How to get ROI of QVideoFrame?:

*input = QVideoFrame();
*input = frameRoi;

1. The first line above is not needed (why do you create a new QVideoFrame, assign it to input and then assign frameRoi to input?). The input has to point to QVideoFrame instance already anyway if you want to assign another QVideoFrame to it.
2. Did you check whether the created QImage(s) are valid?

[MHermann]

@jsulm

Yes, I checked the imges. The images are correct.

[jsulm]

@MHermann What does https://doc.qt.io/qt-6/qvideoframe.html#isValid return?
It could also be that the QImage has to be converted to a supported video format first.

[MHermann]

@jsulm
The edited QVideoFrame is invalid ...

Any suggestions how I convert the QImage correctly to QVideoFrame?

[jsulm]

@MHermann You should check which formats QVideoFrame supports and what format you have in your QImage. Could be an issue converting different formats.
See the documentation (https://doc.qt.io/qt-6/qvideoframe.html#QVideoFrame-2):
"Otherwise, if the QImage::Format matches none of video formats, the image is first converted to a supported (A)RGB format using QImage::convertedTo() with the Qt::AutoColor flag."

[SGaist]

Why not just return frameRoi; ?

As @jsulm wrote, mutating the input frame does not make sense and is not how the video filter works AFAIK.

[MHermann]

@SGaist :
I tried that, but then it had no effect on the shown images in the GUI.
Thats why I think I have to do the changes on the input frame.

[SGaist]

Which version of Qt are you using ?

[MHermann]

@SGaist: I am using Qt 5.12.7

[SGaist]

There's an example application here that might help you get further.

[MHermann]

If somebody is interested in the soultion:

#include <QObject>
#include <QAbstractVideoFilter>
#include <QVideoFilterRunnable>
#include <QAbstractVideoFilter>
#include <QVideoFrame>
#include <QDebug>
#include <QThread>
#include <QFuture>
#include <QtConcurrent/QtConcurrent>
#include <QOpenGLContext>
#include <QOpenGLFunctions>
#include <QQmlContext>
#include "private/qvideoframe_p.h"

...

QImage MyVideoFilterRunnable::QVideoFrameToQImage(const QVideoFrame& videoFrame)
{
    if ( videoFrame.handleType() == QAbstractVideoBuffer::NoHandle )
    {
        QImage image = qt_imageFromVideoFrame( videoFrame );
        if ( image.isNull() )
        {
            return QImage();
        }

        if ( image.format() != QImage::Format_ARGB32 )
        {
            image = image.convertToFormat( QImage::Format_ARGB32 );
        }

        return image;
    }

    if ( videoFrame.handleType() == QAbstractVideoBuffer::GLTextureHandle )
    {
        QImage image( videoFrame.width(), videoFrame.height(), QImage::Format_ARGB32 );
        GLuint textureId = static_cast<GLuint>( videoFrame.handle().toInt() );
        QOpenGLContext* ctx = QOpenGLContext::currentContext();
        QOpenGLFunctions* f = ctx->functions();
        GLuint fbo;
        f->glGenFramebuffers( 1, &fbo );
        GLint prevFbo;
        f->glGetIntegerv( GL_FRAMEBUFFER_BINDING, &prevFbo );
        f->glBindFramebuffer( GL_FRAMEBUFFER, fbo );
        f->glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureId, 0 );
        f->glReadPixels( 0, 0,  videoFrame.width(),  videoFrame.height(), GL_RGBA, GL_UNSIGNED_BYTE, image.bits() );
        f->glBindFramebuffer( GL_FRAMEBUFFER, static_cast<GLuint>( prevFbo ) );
        return image.rgbSwapped();
    }

    return QImage();
}

QVideoFrame MyVideoFilterRunnable::run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags)
{
   ...

   m_filter->m_frame.copyData(*input);

   QImage img = QVideoFrameToQImage( *input );

   QImage imgRoi = img.copy(xMin, yMin, width, height);
   QVideoFrame frameRoi = QVideoFrame(imgRoi);

   *input = frameRoi;

    m_filter->m_processThread = QtConcurrent::run(this, &MyVideoFilterRunnable::processVideoFrame, m_filter->m_frame);

    return *input;
}