Qt Forum

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Unsolved

    Call for Presentations - Qt World Summit

    QProcess killed with SIGPIPE

    General and Desktop
    3
    12
    7072
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      alajovic last edited by

      I am trying to make an application that starts a subprocess, reads raw video frames from its standard output, and processes them. However, after some frames, my subprocess gets killed with SIGPIPE for no obvious reason. Here is a minimal non-working example:

      app.cpp
      @#include <QCoreApplication>
      #include <QProcess>
      #include <QDebug>

      class MyApp : public QObject {
      Q_OBJECT

      private:
      static const quint64 frameLength = 614400;
      QProcess process;
      char* buffer;

      signals:
      void wantNewFrame(); // gets emitted when a new frame is needed
      void frameReady(); // gets emitted when a frame is ready for processing

      public:
      MyApp() {
      // Start a trivial process that never runs out of data.
      process.start("cat /dev/zero");
      buffer = new char[frameLength];
      // Connect the signals - note that the application is single threaded
      // so queued connections are used to avoid infinite recursion.
      connect(this, SIGNAL(wantNewFrame()), SLOT(readFrame()), Qt::QueuedConnection);
      connect(this, SIGNAL(frameReady()), SLOT(frameHandler()), Qt::QueuedConnection);
      // Initiate the reading of the first frame.
      emit wantNewFrame();
      }

      ~MyApp() { delete[] buffer; }

      public slots:
      // This method accumulates the data coming from the QProcess' standard
      // output until there is enough to form a frame.
      void readFrame() {
      qint64 bytesNeeded = frameLength;
      qint64 bytesRead = 0;
      char* ptr = buffer;
      while (bytesNeeded > 0) {
      process.waitForReadyRead();
      bytesRead = process.read(ptr, bytesNeeded);
      if (bytesRead == -1) {
      qDebug() << "process state" << process.state();
      qDebug() << "process error" << process.error();
      qDebug() << "QIODevice error" << process.errorString();
      QCoreApplication::quit();
      return;
      }
      ptr += bytesRead;
      bytesNeeded -= bytesRead;
      }
      emit frameReady(); // will eventually invoke frameHandler()
      }

      // A trivial data processor - it only counts frames.
      void frameHandler() {
      static qint64 frameno = 0;
      qDebug() << "frame" << frameno++;
      emit wantNewFrame(); // will eventually invoke readFrame()
      }
      };

      int main(int argc, char** argv) {
      QCoreApplication coreapp(argc, argv);
      MyApp a;
      return coreapp.exec();
      }

      #include "moc_app.cpp"
      @

      The results are not what one would expect:

      @
      $ ./app
      frame 0
      ...
      frame 249
      process state 0
      process error 1
      QIODevice error "Process crashed"
      @

      The number of frames read varies: I've seen anything from about ten to a few thousand, but eventually, the subprocess gets killed with SIGPIPE. This obviously means that the parent had closed the pipe on it. But why? And where? Due to the randomness observed, I would say that I am almost certainly racing against something, but—against what?

      Note that in this particular simplified case, there would actually be no need to use the signal/slot mechanism to implement the reading/processing loop; I retained the event driven approach because it features in my original application. Incidentally, I tried rewriting the above example without using signals and slots, but having a simple loop like this instead:

      @
      forever {
      readFrame();
      processFrame();
      }
      @

      In this case, the program worked fine. So what is the problem with the original implementation? Am I misusing the signal/slot mechanism in any way?

      Qt 4.8.4, x86_64 quad, Gentoo Linux

      1 Reply Last reply Reply Quote 0
      • SGaist
        SGaist Lifetime Qt Champion last edited by

        Hi and welcome to DevNet,

        Could try something ? Replace your emit with
        @QTimer::singleShot(0, this, readFrame)
        QTimer::singleShot(0, this, processFrame)
        @

        Does the crash still occur ?

        Interested in AI ? www.idiap.ch
        Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

        1 Reply Last reply Reply Quote 0
        • A
          alajovic last edited by

          I did the following two replacements:
          @
          // emit frameReady(); // will eventually invoke frameHandler()
          QTimer::singleShot(0, this, SLOT(frameHandler()));
          @
          and
          @
          // emit wantNewFrame(); // will eventually invoke readFrame()
          QTimer::singleShot(0, this, SLOT(readFrame()));
          @

          The problem persists.

          1 Reply Last reply Reply Quote 0
          • A
            alajovic last edited by

            Another test came to my mind: to check whether I had any problems with the way I'm using signals and slots, I replaced QProcess with QFile and opened /dev/zero directly. This worked fine. I also tried opening a FIFO (named pipe) that had a "cat /dev/zero" running on the other end. This also worked fine.

            I am beginning to think that I might have hit some kind of a bug in QProcess.

            1 Reply Last reply Reply Quote 0
            • M
              MuldeR last edited by

              I think you'll need to install your own signal handler for SIGPIPE then (or simply ignore the signal).

              @signal(SIGPIPE, SIG_IGN);@

              My OpenSource software at: http://muldersoft.com/

              Qt v4.8.6 MSVC 2013, static/shared: http://goo.gl/BXqhrS

              Go visit the coop: http://youtu.be/Jay...

              1 Reply Last reply Reply Quote 0
              • A
                alajovic last edited by

                Thank you for the suggestion, but I think that one should resort to ignoring signals only when there exists a good reason for doing so. In my opinion, ignoring SIGPIPE in this particular case would just serve to cover up a potentially deeper problem.

                Besides that, it does not work. I tried substituting the "cat /dev/zero" process with the following small program:

                @
                #include <cstdio>
                #include <csignal>

                int main() {
                signal(SIGPIPE, SIG_IGN);
                while (true)
                putc('A', stdout);
                return 0;
                }
                @

                The proces does not die in this case (because it ignores SIGPIPE), but after outputting a few frames, putc() suddenly fails with EPIPE (i.e., "Broken pipe").

                The essential problem therefore does not lie in the process being killed with SIGPIPE, but rather in Qt closing the pipe to the subprocess, which causes SIGPIPE to be sent. However, the question remains: why does Qt close the pipe?

                1 Reply Last reply Reply Quote 0
                • M
                  MuldeR last edited by

                  What QProcess::ProcessChannelMode have you set? Maybe try a different one then?

                  http://qt-project.org/doc/qt-4.8/qprocess.html#ProcessChannelMode-enum

                  My OpenSource software at: http://muldersoft.com/

                  Qt v4.8.6 MSVC 2013, static/shared: http://goo.gl/BXqhrS

                  Go visit the coop: http://youtu.be/Jay...

                  1 Reply Last reply Reply Quote 0
                  • A
                    alajovic last edited by

                    I did not set the channel mode explicitly, because it is set to QProcess::SeparateChannels by default; and that's exactly what I need. The other two modes are not suitable for me: I need to capture the data on process' stdout (which rules out QProcess::ForwardedChannels) and not mix in anything else (which rules out QProcess::MergedChannels).

                    1 Reply Last reply Reply Quote 0
                    • M
                      MuldeR last edited by

                      ...still, would be interesting to see if QProcess::ForwardedChannels fixes the issue.

                      Also, what is the purpose of:
                      cat /dev/zero

                      Why not read from /dev/zero directly? Or even better, create a buffer full of "zero" bytes yourself?

                      My OpenSource software at: http://muldersoft.com/

                      Qt v4.8.6 MSVC 2013, static/shared: http://goo.gl/BXqhrS

                      Go visit the coop: http://youtu.be/Jay...

                      1 Reply Last reply Reply Quote 0
                      • A
                        alajovic last edited by

                        As I said, this is a minimal non-working example intended to demonstrate an issue that I am facing in the context of a larger application. It is not meant to do anything useful. The "cat /dev/zero" command only serves here as a convenient source of data; in reality, this command would be replaced with another process that outputs raw video frames.

                        Since I need to capture the command's output, there is no point in setting QProcess::ForwardedChannels as that would cause the output of the QProcess to be forwarded directly to the standard output of the parent application. There would be no way of capturing it then.

                        1 Reply Last reply Reply Quote 0
                        • M
                          MuldeR last edited by

                          [quote author="alajovic" date="1365361656"]It is not meant to do anything useful.[/quote]

                          Okay, I see.

                          [quote author="alajovic" date="1365361656"]there is no point in setting QProcess::ForwardedChannels as that would cause the output of the QProcess to be forwarded directly to the standard output of the parent application. There would be no way of capturing it then.[/quote]

                          The point is not to get your program working, but to track down the problem. Then, if it turns out something is wrong inside Qt, get the bug fixed. I think it would be interesting to know whether this is related to redirecting the standard streams...

                          My OpenSource software at: http://muldersoft.com/

                          Qt v4.8.6 MSVC 2013, static/shared: http://goo.gl/BXqhrS

                          Go visit the coop: http://youtu.be/Jay...

                          1 Reply Last reply Reply Quote 0
                          • A
                            alajovic last edited by

                            Okay, it's an easy test anyway.

                            I tried putting

                            @
                            process.setProcessChannelMode(QProcess::ForwardedChannels);
                            @

                            before process.start(). The subprocess then ran uninterrupted and its standard output was forwarded to the standard output of the test application. Meanwhile, the application itself was blocking on process.waitForReadyRead() call (line 40 in my original post).

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post