Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Error while displaying GStreamer video from UDP using Qt Multimedia



  • Hi.

    I would like to use Qt Multimedia to display a video stream.

    Although I managed to create a working example with videotestsrc, it fails with an ambiguous error using UDP stream, continuously printing appsrc: push buffer wrong state instead of displaying the video.

    Do you know what could cause this error and how to fix it?

    Locally, you can run these commands in two terminal windows:

    # Server
    gst-launch-1.0 videotestsrc ! x264enc ! rtph264pay ! udpsink host=localhost port=5000
    
    # Client
    gst-launch-1.0 udpsrc port=5000 ! "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" ! rtph264depay ! decodebin ! autovideosink
    

    This works fine, and video is displayed correctly.

    Now, launch the server and run this Qt application.

    #include <QApplication>
    #include <QMediaPlayer>
    #include <QWidget>
    #include <QVideoWidget>
    #include <QBoxLayout>
    #include <QProcess>
    #include <QDebug>
    
    int main(int argc, char *argv[])
    {
        QApplication a(argc, argv);
    
        QWidget *window = new QWidget;
        QVideoWidget *videoWidget = new QVideoWidget;
        QBoxLayout *layout = new QVBoxLayout;
        QMediaPlayer *player = new QMediaPlayer;
        QProcess *process = new QProcess;
    
        layout->addWidget(videoWidget);
        window->setLayout(layout);
        window->show();
        player->setVideoOutput(videoWidget);
    
        QString program = "gst-launch-1.0";
        QStringList arguments;
    
        /* This is working fine, try it! */
        // arguments << "-v" << "videotestsrc" << "!" << "x264enc" << "!" << "filesink" << "location=/dev/stderr";
    
        /* This is not working :'( */
        arguments << "-v" << "udpsrc" << "port=5000" << "!" << "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" << "!" << "rtph264depay" << "!" << "filesink" << "location=/dev/stderr";
    
        /* Strangely, this is working half a second */
        // arguments << "-v" << "udpsrc" << "port=5000" << "!" << "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" << "!" << "rtph264depay" << "!" << "decodebin" << "!" << "x264enc" << "!" << "filesink" << "location=/dev/stderr";
    
        process->setReadChannel(QProcess::StandardError);
        process->start(program, arguments);
    
        while (!process->waitForReadyRead()) {}
    
        player->setMedia(QMediaContent(), process);
        player->play();
    
        return a.exec();
    }
    

    Note that stream is not explicitly decoded intentionally as QMediaPlayer expects a binary media (like a video file).

    I know that there exists a QtGStreamer module but it installation requires dependencies we are trying to avoid, and it seemed easier to just rely on command line.
    I am using Qt 5.11.1 on Ubuntu 16.04.


  • Lifetime Qt Champion

    Hi,

    What about the qmlglsink that is mentioned here ?



  • @SGaist Thanks for your suggestion, I did not know about this GStreamer plugin.
    However, I am not sure how to use it nor if it would integrate very well with my current QtMultimedia + QProcess pipeline.

    Nonetheless, thanks to Qt being open-source and easy to compile, I was able to re-install it from source while modifying parts of the module to better understand what caused the push wrong buffer state issue.

    As you can see here, if the stream (my QProcess) does not have any bytesAvailable, the appsrc is closed and no longer accept data to be displayed to the QtMultimedia endpoint.
    If I comment the sendEOS() call, my program seems to work fine.
    I suppose that data is sometime consumed faster that the QProcess delivers it, resulting in this error.

    As a workaround, I thought of inheriting from QProcess and make bytesAvailable() a blocking method rather than returning 0. This is not ideal, but I guess it should work?


  • Lifetime Qt Champion

    It's worth a try.



  • QtGStreamer is not part of Qt.
    QMediaPlayer with GStreamer supports udp url schema.
    like:
    MediaPlayer {
    source: "udp://127.0.0.1:5001/?udpsrc.caps=application/x-rtp,media=video,clock-rate=90000,encoding=JPEG,payload=26"
    }


Log in to reply