Get each video's frames as an QImage using GStreamer

[Daniellopes]

I'm starting a project that consists of receive video from an RTSP server and showing it in a panel using QT c++ and GSTreamer. I want to receive every frame as a QImage object to run some features that I need. I know that we can use QTMultimedia to link the Gstreamer and view the video, but in this particular case, I would like to have QImages. I found a way (or i think i did) to get the samples from the video by calling the function "gst_app_sink_pull_sample". We get a GSTSample from it. However, i did not find any way to convert this into, for example, a JPG raw data which is easy to convert into a QImage. I

I also found a way to access the data from GSTSample from here: "How to get video stream frame-by-frame from Gstreamer pipeline? (without OpenCV)" but, once again, i've no idea how can i convert this data into a QImage.

/* Initialize GStreamer */
gst_init (NULL, NULL);

    /* Create the elements */
    GstElement * source = gst_element_factory_make ("videotestsrc", "source"); // HERE I'M USING THE GSTREAMER SOURCE TEST
    GstElement * sink = gst_element_factory_make ("appsink", "sink");
    GstAppSink *appsink = GST_APP_SINK(sink);

    /* Create the empty pipeline */
     GstElement * pipeline = gst_pipeline_new ("test-pipeline");

    if (!pipeline || !source || !sink) {
        g_printerr ("Not all elements could be created.\n");
        return ;
    }

    /* Build the pipeline */
    gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
    if (gst_element_link (source, sink) != TRUE) {
        g_printerr ("Elements could not be linked.\n");
        gst_object_unref (pipeline);
        return ;
    }

    /* Modify the source's properties */
    g_object_set (source, "pattern", 0, NULL);


    /* Start playing */
    ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
    if (ret == GST_STATE_CHANGE_FAILURE) {
        g_printerr ("Unable to set the pipeline to the playing state.\n");
        gst_object_unref (pipeline);
        return;
    }


    // My question starts HERE
    GstSample *sample = nullptr;

    do{

        sample = gst_app_sink_pull_sample(appsink); // Get the frame
        if (!sample) {
            printf("sample is NULL\n");
        }else{
            
            // Get the raw data (or trying to...)
            GstBuffer * buffer = gst_sample_get_buffer(sample);
            GstMapInfo info;
            gst_buffer_map(buffer, &info, GST_MAP_READ);

            // Dumb way to check if the frame is being received properly
            QImage imageAux((const unsigned char*)info.data, 100, 100, QImage::Format_RGB16);
            imageAux.save("out.jpg"); //returns garbage

        }


    }while(sample);

[SGaist]

Hi and welcome to devnet,

What is the exact video characteristics ?

[Daniellopes]

@SGaist In fact i've no idea. i'm using the "videotestsrc" module. I'm not sure but i think it returns AYUV video format.

[Daniellopes]

I found an example that seems to be my solution however, it also uses the Qt5GStreamer libraries:
Example
I've installed the libqt5gstreamer-dev and qtgstreamer-plugins-qt5 and I added the PKGCONFIG += Qt5GStreamer-1.0 Qt5GStreamerUi-1.0 gstreamer-1.0 gstreamer-video-1.0 to .pro file.

The problem now is that even if I don't add any code related to Qt5GStreamer, my compiler gives me several issues related to the system's header files and others. The project compiles successfully if I remove the pkgconfig. The version of my QT is the QT5.15.2 and the Qt5GStreamer is the 1.2.

Any idea?

[SGaist]

You should rather use the Qt version provided by your distribution since you installed the Qt5Gstreamer libraries through it.