QProcess running out of file descriptors

DanJChv

When creating a large number of QProcess's, I get messages like these:

QProcessPrivate::createPipe: Cannot create pipe 0x2af11c285600: Too many open files
QSocketNotifier: Invalid socket specified

The problem is I get these error messages, but I haven't found a way via the API to detect that this error has happened. There are apparently no signals and no function calls that I can find that get relevant info. And QProcessPrivate appears not available to the user.

Ideas?

asanka424

Hi,

I think this should be dealt in OS level. You can increase number of file descriptors in your OS ( I assume you are on Linux. even for windows I think it can)

Thanks

kshegunov

@DanJChv
Hello,
For this to happen you'd need to have an insane amount of open processes, maybe you should consider switching to a different design. However, I'm with @asanka424, this is a system limitation and doesn't have a Qt solution.

Kind regards.

DanJChv

@kshegunov

The underlying problem can be fixed on the system level; however, that's not the goal here. The goal is for the program to KNOW there is a problem with the underlying system. If the program runs into this limit, it can request the user contact the system administrators to increase the default limit on number of file descriptors. The most user friendly way is to notify the user only if they are doing something that goes beyond the limit.

I agree a design that uses fewer QProcesses would be better, but that would be a major revision which I don't have time for. The current problem is occurring with only about 200 QProcesses on a system with a file descriptor limit of 1024.

kshegunov

@DanJChv
Hello,
What about QProcess::error or the corresponding signal QProcess::error(QProcess::ProcessError error)? Here the private object sets the error, emits the signal and breaks execution.

The current problem is occurring with only about 200 QProcesses on a system with a file descriptor limit of 1024.

You're not the only one requesting a descriptor. What about all the sockets on the system (many if not all daemons use them) and Qt uses some internally as well. So don't be surprised to have run out of them with only with a thousand to start with ... on my system cat /proc/sys/fs/file-max returns 1631401.

Kind regards.

DanJChv

I have checked the QProcess::error signal, but it isn't called for the case of an invalid file descriptor amazingly.

kshegunov

@DanJChv
Hm, that's strange. The error and status properties could check for these? What version of Qt are you running?

? Offline

Hi everyone. This was discussed in QTBUG-18934 and not considered a bug.

kshegunov

@Wieland
Hello,
If you take a look at the attached gerrit patch (which seems to have been integrated) I believe the bug's supposed to be fixed. Am I missing something?

Kind regards.

? Offline

@kshegunov If I understand the bug report correctly then QProcess crashed the application (SIGSEGV) when running out of file descriptors. With the patch it doesn't crash anymore but enters some bogus state. Thiago said the patch wouldn't make much sense but he'd integrate it anyhow.

kshegunov

@Wieland
Maybe I'm in error, I'm by no means an expert. I just follow the code, and it looks like (not necessarily behaves that way though) it should set the error and emit the signal.

? Offline

@kshegunov Ok, might be true. I only read the comments under the bug report.