Error while displaying GStreamer video from UDP using Qt Multimedia

[ADGB]

Hi.

I would like to use Qt Multimedia to display a video stream.

Although I managed to create a working example with videotestsrc, it fails with an ambiguous error using UDP stream, continuously printing appsrc: push buffer wrong state instead of displaying the video.

Do you know what could cause this error and how to fix it?

Locally, you can run these commands in two terminal windows:

# Server
gst-launch-1.0 videotestsrc ! x264enc ! rtph264pay ! udpsink host=localhost port=5000

# Client
gst-launch-1.0 udpsrc port=5000 ! "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" ! rtph264depay ! decodebin ! autovideosink

This works fine, and video is displayed correctly.

Now, launch the server and run this Qt application.

#include <QApplication>
#include <QMediaPlayer>
#include <QWidget>
#include <QVideoWidget>
#include <QBoxLayout>
#include <QProcess>
#include <QDebug>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    QWidget *window = new QWidget;
    QVideoWidget *videoWidget = new QVideoWidget;
    QBoxLayout *layout = new QVBoxLayout;
    QMediaPlayer *player = new QMediaPlayer;
    QProcess *process = new QProcess;

    layout->addWidget(videoWidget);
    window->setLayout(layout);
    window->show();
    player->setVideoOutput(videoWidget);

    QString program = "gst-launch-1.0";
    QStringList arguments;

    /* This is working fine, try it! */
    // arguments << "-v" << "videotestsrc" << "!" << "x264enc" << "!" << "filesink" << "location=/dev/stderr";

    /* This is not working :'( */
    arguments << "-v" << "udpsrc" << "port=5000" << "!" << "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" << "!" << "rtph264depay" << "!" << "filesink" << "location=/dev/stderr";

    /* Strangely, this is working half a second */
    // arguments << "-v" << "udpsrc" << "port=5000" << "!" << "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" << "!" << "rtph264depay" << "!" << "decodebin" << "!" << "x264enc" << "!" << "filesink" << "location=/dev/stderr";

    process->setReadChannel(QProcess::StandardError);
    process->start(program, arguments);

    while (!process->waitForReadyRead()) {}

    player->setMedia(QMediaContent(), process);
    player->play();

    return a.exec();
}

Note that stream is not explicitly decoded intentionally as QMediaPlayer expects a binary media (like a video file).

I know that there exists a QtGStreamer module but it installation requires dependencies we are trying to avoid, and it seemed easier to just rely on command line.
I am using Qt 5.11.1 on Ubuntu 16.04.

[SGaist]

Hi,

What about the qmlglsink that is mentioned here ?

[ADGB]

@SGaist Thanks for your suggestion, I did not know about this GStreamer plugin.
However, I am not sure how to use it nor if it would integrate very well with my current QtMultimedia + QProcess pipeline.

Nonetheless, thanks to Qt being open-source and easy to compile, I was able to re-install it from source while modifying parts of the module to better understand what caused the push wrong buffer state issue.

As you can see here, if the stream (my QProcess) does not have any bytesAvailable, the appsrc is closed and no longer accept data to be displayed to the QtMultimedia endpoint.
If I comment the sendEOS() call, my program seems to work fine.
I suppose that data is sometime consumed faster that the QProcess delivers it, resulting in this error.

As a workaround, I thought of inheriting from QProcess and make bytesAvailable() a blocking method rather than returning 0. This is not ideal, but I guess it should work?

[SGaist]

It's worth a try.

[VaL Doroshchuk]

QtGStreamer is not part of Qt.
QMediaPlayer with GStreamer supports udp url schema.
like:
MediaPlayer {
source: "udp://127.0.0.1:5001/?udpsrc.caps=application/x-rtp,media=video,clock-rate=90000,encoding=JPEG,payload=26"
}

[Pooja Bhusare]

could you please share working Qt app which display video correctly.