Play Video using Gstreamer and QML

vicky_mac

Hi,

I am trying to play video in QML using gstreamer with app sink. The video is playing but i have few concerns listed below:

**1. Hardware five YUV NV12 format decoded frame which i converting to RGB using OpenVC. Need other suitable options if available.

While running the application i see the memory consumed by the application is increasing overtime and finally crashes due to huge memory usage. Need to know why it's happening.
Sometime the playback hangs for no reason,**

Please find below program for reference:

//Class to pull decoded data from Gstreamer pipeline
int ImageProvider::width=1920;
int ImageProvider::height=1080;
GstSample* ImageProvider::sample =nullptr;
GstBuffer* ImageProvider::sampleBuffer=nullptr;
GstMapInfo ImageProvider::bufferInfo;
cv::Mat ImageProvider::rgbMat(height + height/2,width,CV_8UC3,cv::Scalar(0,0,255));
cv::Mat ImageProvider::yuvMat(height + height/2,width,CV_8UC1,cv::Scalar(0,0,255));
cv::Mat ImageProvider::scaledMat(300,500,CV_8UC3,cv::Scalar(0,0,255));
ImageProvider::ImageProvider(QObject *parent)
{
    m_image = QImage(500,300,QImage::Format_RGB888);
    m_image.fill(Qt::black);
}
GstFlowReturn ImageProvider::newSample(GstAppSink *sink, gpointer gSelf)
{
    ImageProvider* self = static_cast<ImageProvider* >(gSelf);
    sample = gst_app_sink_pull_sample(GST_APP_SINK(sink));
    if(sample != NULL)
    {
        sampleBuffer = gst_sample_get_buffer(sample);
        if(sampleBuffer != NULL)
        {
            gst_buffer_map(sampleBuffer, &bufferInfo, GST_MAP_READ);
            yuvMat = cv::Mat(height + height/2,width,CV_8UC1,bufferInfo.data);
            cv::cvtColor(yuvMat,rgbMat,cv::COLOR_YUV2RGB_NV12);
            cv::resize(rgbMat,scaledMat,cv::Size(500,300));
            self->m_image = QImage(scaledMat.data,scaledMat.cols,scaledMat.rows,QImage::Format_RGB888);
            if(self->m_image.isNull())
            {
                qDebug() << "Image is null";
            }
            else
            {
                 emit self->imageChanged();
            }
            gst_buffer_unmap(sampleBuffer, &bufferInfo);
    }
    gst_sample_unref(sample);
}
return GST_FLOW_OK;
}

//QPaint class
 LiveVideoImage::LiveVideoImage(QQuickItem *parent)
{
    m_image = QImage(500,300,QImage::Format_RGB888);
    m_image.fill(Qt::black);
}

void LiveVideoImage::setImage(const QImage &image)
{
        m_image = image;
        if(m_image.size().width() > 0)
            update();
}

void LiveVideoImage::paint(QPainter *painter)
{
            painter->drawImage(0,0,m_image);
}
 

// In main.cpp
    QQmlContext* context = engine.rootContext();
    qmlRegisterType<LiveVideoImage>("Test.VideoImage",1,0,"LiveImage");


//QML File

import Test.VideoImage 1.0
    Rectangle{
        width: 500
        height: 300
        id:videoPlayer
        x:parent.width*0.24
        y:parent.height*0.3
        LiveImage{
            width: 500
            height: 300
            image:LiveImageProvider.image
        }
    }

Using Qt5.13.2
@SGaist
Please let me know if you need any other details

vicky_mac

@vicky_mac any one?

JoeCFD

@vicky_mac The old gstreamer versions have memory leaks. Which gstreamer version are you using? Try to upgrade to the latest gstreamer.
Your Qt is old as well. Upgrade your Qt to at least 5.15.2.