QProcess killed with SIGPIPE

[alajovic]

I am trying to make an application that starts a subprocess, reads raw video frames from its standard output, and processes them. However, after some frames, my subprocess gets killed with SIGPIPE for no obvious reason. Here is a minimal non-working example:

app.cpp
@#include <QCoreApplication>
#include <QProcess>
#include <QDebug>

class MyApp : public QObject {
Q_OBJECT

private:
static const quint64 frameLength = 614400;
QProcess process;
char* buffer;

signals:
void wantNewFrame(); // gets emitted when a new frame is needed
void frameReady(); // gets emitted when a frame is ready for processing

public:
MyApp() {
// Start a trivial process that never runs out of data.
process.start("cat /dev/zero");
buffer = new char[frameLength];
// Connect the signals - note that the application is single threaded
// so queued connections are used to avoid infinite recursion.
connect(this, SIGNAL(wantNewFrame()), SLOT(readFrame()), Qt::QueuedConnection);
connect(this, SIGNAL(frameReady()), SLOT(frameHandler()), Qt::QueuedConnection);
// Initiate the reading of the first frame.
emit wantNewFrame();
}

~MyApp() { delete[] buffer; }

public slots:
// This method accumulates the data coming from the QProcess' standard
// output until there is enough to form a frame.
void readFrame() {
qint64 bytesNeeded = frameLength;
qint64 bytesRead = 0;
char* ptr = buffer;
while (bytesNeeded > 0) {
process.waitForReadyRead();
bytesRead = process.read(ptr, bytesNeeded);
if (bytesRead == -1) {
qDebug() << "process state" << process.state();
qDebug() << "process error" << process.error();
qDebug() << "QIODevice error" << process.errorString();
QCoreApplication::quit();
return;
}
ptr += bytesRead;
bytesNeeded -= bytesRead;
}
emit frameReady(); // will eventually invoke frameHandler()
}

// A trivial data processor - it only counts frames.
void frameHandler() {
static qint64 frameno = 0;
qDebug() << "frame" << frameno++;
emit wantNewFrame(); // will eventually invoke readFrame()
}
};

int main(int argc, char** argv) {
QCoreApplication coreapp(argc, argv);
MyApp a;
return coreapp.exec();
}

#include "moc_app.cpp"
@

The results are not what one would expect:

@
$ ./app
frame 0
...
frame 249
process state 0
process error 1
QIODevice error "Process crashed"
@

The number of frames read varies: I've seen anything from about ten to a few thousand, but eventually, the subprocess gets killed with SIGPIPE. This obviously means that the parent had closed the pipe on it. But why? And where? Due to the randomness observed, I would say that I am almost certainly racing against something, but—against what?

Note that in this particular simplified case, there would actually be no need to use the signal/slot mechanism to implement the reading/processing loop; I retained the event driven approach because it features in my original application. Incidentally, I tried rewriting the above example without using signals and slots, but having a simple loop like this instead:

@
forever {
readFrame();
processFrame();
}
@

In this case, the program worked fine. So what is the problem with the original implementation? Am I misusing the signal/slot mechanism in any way?

Qt 4.8.4, x86_64 quad, Gentoo Linux

[SGaist]

Hi and welcome to DevNet,

Could try something ? Replace your emit with
@QTimer::singleShot(0, this, readFrame)
QTimer::singleShot(0, this, processFrame)
@

Does the crash still occur ?

[alajovic]

I did the following two replacements:
@
// emit frameReady(); // will eventually invoke frameHandler()
QTimer::singleShot(0, this, SLOT(frameHandler()));
@
and
@
// emit wantNewFrame(); // will eventually invoke readFrame()
QTimer::singleShot(0, this, SLOT(readFrame()));
@

The problem persists.

[alajovic]

Another test came to my mind: to check whether I had any problems with the way I'm using signals and slots, I replaced QProcess with QFile and opened /dev/zero directly. This worked fine. I also tried opening a FIFO (named pipe) that had a "cat /dev/zero" running on the other end. This also worked fine.

I am beginning to think that I might have hit some kind of a bug in QProcess.

[MuldeR]

I think you'll need to install your own signal handler for SIGPIPE then (or simply ignore the signal).

@signal(SIGPIPE, SIG_IGN);@

[alajovic]

Thank you for the suggestion, but I think that one should resort to ignoring signals only when there exists a good reason for doing so. In my opinion, ignoring SIGPIPE in this particular case would just serve to cover up a potentially deeper problem.

Besides that, it does not work. I tried substituting the "cat /dev/zero" process with the following small program:

@
#include <cstdio>
#include <csignal>

int main() {
signal(SIGPIPE, SIG_IGN);
while (true)
putc('A', stdout);
return 0;
}
@

The proces does not die in this case (because it ignores SIGPIPE), but after outputting a few frames, putc() suddenly fails with EPIPE (i.e., "Broken pipe").

The essential problem therefore does not lie in the process being killed with SIGPIPE, but rather in Qt closing the pipe to the subprocess, which causes SIGPIPE to be sent. However, the question remains: why does Qt close the pipe?

[MuldeR]

What QProcess::ProcessChannelMode have you set? Maybe try a different one then?

http://qt-project.org/doc/qt-4.8/qprocess.html#ProcessChannelMode-enum

[alajovic]

I did not set the channel mode explicitly, because it is set to QProcess::SeparateChannels by default; and that's exactly what I need. The other two modes are not suitable for me: I need to capture the data on process' stdout (which rules out QProcess::ForwardedChannels) and not mix in anything else (which rules out QProcess::MergedChannels).

[MuldeR]

...still, would be interesting to see if QProcess::ForwardedChannels fixes the issue.

Also, what is the purpose of:
cat /dev/zero

Why not read from /dev/zero directly? Or even better, create a buffer full of "zero" bytes yourself?

[alajovic]

As I said, this is a minimal non-working example intended to demonstrate an issue that I am facing in the context of a larger application. It is not meant to do anything useful. The "cat /dev/zero" command only serves here as a convenient source of data; in reality, this command would be replaced with another process that outputs raw video frames.

Since I need to capture the command's output, there is no point in setting QProcess::ForwardedChannels as that would cause the output of the QProcess to be forwarded directly to the standard output of the parent application. There would be no way of capturing it then.

[MuldeR]

[quote author="alajovic" date="1365361656"]It is not meant to do anything useful.[/quote]

Okay, I see.

[quote author="alajovic" date="1365361656"]there is no point in setting QProcess::ForwardedChannels as that would cause the output of the QProcess to be forwarded directly to the standard output of the parent application. There would be no way of capturing it then.[/quote]

The point is not to get your program working, but to track down the problem. Then, if it turns out something is wrong inside Qt, get the bug fixed. I think it would be interesting to know whether this is related to redirecting the standard streams...

[alajovic]

Okay, it's an easy test anyway.

I tried putting

@
process.setProcessChannelMode(QProcess::ForwardedChannels);
@

before process.start(). The subprocess then ran uninterrupted and its standard output was forwarded to the standard output of the test application. Meanwhile, the application itself was blocking on process.waitForReadyRead() call (line 40 in my original post).