QThread::start: Thread creation error: Resource temporarily unavailable

mishas

Hello.
When I try to start over 500 threads using phantomjs the error is thrown:
[WARNING] QThread :: start: Thread creation error: Resource temporarily unavailable

Tell me what needs to be done to avoid this error?

Thanks in advance

$ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 514564
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1048576
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 24576000
cpu time (seconds, -t) unlimited
max user processes (-u) 1000000
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

$uname -a
Linux debian 4.19.0-12-amd64 #1 SMP Debian 4.19.152-1 (2020-10-18) x86_64 GNU/Linux

$free
total used free shared buff/cache available
Mem: 131960164 18548424 104018312 1575544 9393428 110874136
Swap: 134152188 0 134152188

$lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 24
On-line CPU(s) list: 0-23
Thread(s) per core: 2
Core(s) per socket: 6
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Model name: Intel(R) Xeon(R) CPU E5-2430 0 @ 2.20GHz
Stepping: 7
CPU MHz: 2458.472

Christian Ehrlicher

@mishas said in QThread::start: Thread creation error: Resource temporarily unavailable:

Tell me what needs to be done to avoid this error?

Don't start so many threads - or do you have 500 cores?

mishas

my task needs to run at least 700 threads.
hardware allows it..
just tell me, what "resource" I need to increase?
Thank you.

htop:

when I using php cUrl, I can start over 1000 threads, but I need to run phantomjs...

JonB

@mishas said in QThread::start: Thread creation error: Resource temporarily unavailable:

[WARNING] QThread :: start: Thread creation error: Resource temporarily unavailable

I would read e.g. https://forum.qt.io/topic/22135/qthread-start-thread-creation-error-resource-temporarily-unavailable. First it discusses why it's not a great idea and what the issue might be.

By the time you get to https://forum.qt.io/topic/22135/qthread-start-thread-creation-error-resource-temporarily-unavailable/8 :

After modifying /etc/security/limits.conf
--with
<myusername> soft nofile 150000
<myusername> hard nofile 150000
<myusername> soft nproc 150000
<myusername> hard nproc 150000

I can open about 34000 threads from the class QThread

I believe that user is saying he went from 290 to 34,000, so might address your requirement. Then he says:

after opening those threads,I can not execute any more commands in shell

I get the error "bash: fork: Cannot allocate memory"

:D

Christian Ehrlicher

@mishas said in QThread::start: Thread creation error: Resource temporarily unavailable:

my task needs to run at least 700 threads.

What are you doing?

mishas

@JonB said in QThread::start: Thread creation error: Resource temporarily unavailable:

@mishas said in QThread::start: Thread creation error: Resource temporarily unavailable:

[WARNING] QThread :: start: Thread creation error: Resource temporarily unavailable

I would read e.g. https://forum.qt.io/topic/22135/qthread-start-thread-creation-error-resource-temporarily-unavailable. First it discusses why it's not a great idea and what the issue might be.

By the time you get to https://forum.qt.io/topic/22135/qthread-start-thread-creation-error-resource-temporarily-unavailable/8 :
After modifying /etc/security/limits.conf
--with
<myusername> soft nofile 150000
<myusername> hard nofile 150000
<myusername> soft nproc 150000
<myusername> hard nproc 150000

I can open about 34000 threads from the class QThread
I believe that user is saying he went from 290 to 34,000, so might address your requirement. Then he says:

after opening those threads,I can not execute any more commands in shell

I get the error "bash: fork: Cannot allocate memory"

:D

I read this post before posting.
As you can see my
$ulimit -a
a lot more then in post:
open files (-n) 1048576
max user processes (-u) 1000000