QProcess running out of file descriptors
-
wrote on 11 Feb 2016, 17:45 last edited by
When creating a large number of QProcess's, I get messages like these:
QProcessPrivate::createPipe: Cannot create pipe 0x2af11c285600: Too many open files
QSocketNotifier: Invalid socket specifiedThe problem is I get these error messages, but I haven't found a way via the API to detect that this error has happened. There are apparently no signals and no function calls that I can find that get relevant info. And QProcessPrivate appears not available to the user.
Ideas?
-
wrote on 11 Feb 2016, 18:00 last edited by
Hi,
I think this should be dealt in OS level. You can increase number of file descriptors in your OS ( I assume you are on Linux. even for windows I think it can)
Thanks
-
When creating a large number of QProcess's, I get messages like these:
QProcessPrivate::createPipe: Cannot create pipe 0x2af11c285600: Too many open files
QSocketNotifier: Invalid socket specifiedThe problem is I get these error messages, but I haven't found a way via the API to detect that this error has happened. There are apparently no signals and no function calls that I can find that get relevant info. And QProcessPrivate appears not available to the user.
Ideas?
@DanJChv
Hello,
For this to happen you'd need to have an insane amount of open processes, maybe you should consider switching to a different design. However, I'm with @asanka424, this is a system limitation and doesn't have a Qt solution.Kind regards.
-
@DanJChv
Hello,
For this to happen you'd need to have an insane amount of open processes, maybe you should consider switching to a different design. However, I'm with @asanka424, this is a system limitation and doesn't have a Qt solution.Kind regards.
wrote on 11 Feb 2016, 20:45 last edited byThe underlying problem can be fixed on the system level; however, that's not the goal here. The goal is for the program to KNOW there is a problem with the underlying system. If the program runs into this limit, it can request the user contact the system administrators to increase the default limit on number of file descriptors. The most user friendly way is to notify the user only if they are doing something that goes beyond the limit.
I agree a design that uses fewer QProcesses would be better, but that would be a major revision which I don't have time for. The current problem is occurring with only about 200 QProcesses on a system with a file descriptor limit of 1024.
-
The underlying problem can be fixed on the system level; however, that's not the goal here. The goal is for the program to KNOW there is a problem with the underlying system. If the program runs into this limit, it can request the user contact the system administrators to increase the default limit on number of file descriptors. The most user friendly way is to notify the user only if they are doing something that goes beyond the limit.
I agree a design that uses fewer QProcesses would be better, but that would be a major revision which I don't have time for. The current problem is occurring with only about 200 QProcesses on a system with a file descriptor limit of 1024.
Moderatorswrote on 11 Feb 2016, 21:14 last edited by kshegunov 2 Nov 2016, 21:16@DanJChv
Hello,
What about QProcess::error or the corresponding signal QProcess::error(QProcess::ProcessError error)? Here the private object sets the error, emits the signal and breaks execution.The current problem is occurring with only about 200 QProcesses on a system with a file descriptor limit of 1024.
You're not the only one requesting a descriptor. What about all the sockets on the system (many if not all daemons use them) and Qt uses some internally as well. So don't be surprised to have run out of them with only with a thousand to start with ... on my system
cat /proc/sys/fs/file-maxreturns 1631401.Kind regards.
-
wrote on 11 Feb 2016, 23:42 last edited by
I have checked the QProcess::error signal, but it isn't called for the case of an invalid file descriptor amazingly.
-
I have checked the QProcess::error signal, but it isn't called for the case of an invalid file descriptor amazingly.
@DanJChv
Hm, that's strange. Theerrorandstatusproperties could check for these? What version of Qt are you running? -
wrote on 12 Feb 2016, 05:14 last edited by
Hi everyone. This was discussed in QTBUG-18934 and not considered a bug.
-
Hi everyone. This was discussed in QTBUG-18934 and not considered a bug.
@Wieland
Hello,
If you take a look at the attached gerrit patch (which seems to have been integrated) I believe the bug's supposed to be fixed. Am I missing something?Kind regards.
-
wrote on 12 Feb 2016, 06:47 last edited by
@kshegunov If I understand the bug report correctly then QProcess crashed the application (SIGSEGV) when running out of file descriptors. With the patch it doesn't crash anymore but enters some bogus state. Thiago said the patch wouldn't make much sense but he'd integrate it anyhow.
-
@kshegunov If I understand the bug report correctly then QProcess crashed the application (SIGSEGV) when running out of file descriptors. With the patch it doesn't crash anymore but enters some bogus state. Thiago said the patch wouldn't make much sense but he'd integrate it anyhow.
@Wieland
Maybe I'm in error, I'm by no means an expert. I just follow the code, and it looks like (not necessarily behaves that way though) it should set the error and emit the signal. -
@Wieland
Maybe I'm in error, I'm by no means an expert. I just follow the code, and it looks like (not necessarily behaves that way though) it should set the error and emit the signal.wrote on 12 Feb 2016, 06:55 last edited by@kshegunov Ok, might be true. I only read the comments under the bug report.
9/12