1200 QThreads need create

[A Former User]

this code doesn't work on linux.
it makes
QThread::start: Thread creation error: resource temporarily unavailable

after ulimit -s 1024
GLib-ERROR **: Cannot create pipe main loop wake-up: too many files open

@#include <QtGui>

class MyThread : public QThread {
Q_OBJECT
private:
int m_nValue;

public:
MyThread() : m_nValue(1000)
{
}

void run()
{

while (true) {
msleep(m_nValue);
//qDebug() << QThread::currentThreadId();

}
}
};

int main(int argc, char** argv)
{
QApplication app(argc, argv);

MyThread     *thread;

for (int i=0;i<1201;++i){

thread=new MyThread();
thread->start();
qDebug() << i;
}

return app.exec();

}

#include "main.moc"
@

[Franzk]

Well, 1200 threads is overkill of course and it certainly won't do much for your application. It's better to use a thread pool (usually processor count or processor count + 1 threads) and handle the tasks there. Look into QtConcurrent if you just want a lot of tasks done.

[A Former User]

in Windows that code is run good.
my app needs more than 1200 threads, 1200 just for test.

and.. above code runs on linux very well, and build 5000 threads!

@#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <signal.h>

#define NUM_THREADS 5000

volatile int while_condition = 1;

void sig_handler(int signal)
{
while_condition = 0;
}

void *do_work(void *ptr) {
size_t i = (size_t)ptr;
while (while_condition) {
sleep(1);
}
pthread_exit(NULL);
}

int main(int argc, char** argv) {
pthread_t threads[NUM_THREADS];
pthread_attr_t attr;
void *ptr;
size_t i;

signal(SIGINT, sig_handler);

/* Initialize and set thread detached attribute */
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);

for (i = 0; i < NUM_THREADS; ++i)
{
    printf("In main: creating thread %ld\n", i);
    int rc = pthread_create(&threads[i], &attr, do_work, (void *)i);
    if (rc) {
        printf("ERROR; return code from pthread_create() is %d\n", rc);
        exit(-1);
    }
}

/* Free attribute and wait for the other threads */
pthread_attr_destroy(&attr);

for (i = 0; i < NUM_THREADS; ++i)
{
    int rc = pthread_join(threads[i], &ptr);
    if (rc) {
        printf("ERROR; return code from pthread_join() is %d\n", rc);
        exit(-1);
    }
    printf("In main: stopped thread %ld\n", i);
}

return (EXIT_SUCCESS);

}@

[tobias.hunger]

Of course you should be able to run an arbitrary number of threads at the same time. But you can never ever run more than threads at the same time than you have processing units. It does sometimes make sense to run more than that number of threads, but only if the threads are not CPU bound (e.g. have to wait for data from the HDD).

You further need to be aware that creating and destructing threads is a somewhat expensive operation. That is the reason for systems using thread pools consisting of a fixed number of threads. These threads are then used to process jobs that get queued with the system. This allows to queue lots of jobs while keeping the thread management overhead low.

QtConcurrent is such a thread pool concept.

[goetz]

See the forum thread "Help With this ERROR:QThread::start: Thread creation error: Resource temporarily unavailable":http://developer.qt.nokia.com/forums/viewthread/1496 for a discussion of the same problem.

There is a limit on the maximum number of threads a single application and/or a user can start.

As Tobias already mentioned, there it almost always does not make sense to start significantly more threads than you have CPU cores. In the worst case, the overhead of managing that much threads (by your app, by the OS) might be contra productive.

[AlterX]

Linux has a default limit to 1024 max opened files...if you want to change it, just see /etc/security/limits.conf to change per user max files, or if you want a whole max files setting, take a look at
/proc/sys/fs/file-max and persists that in /etc/sysctl.conf

Bye

[yshurik]

Are you sure you need exactly "threads"? Because it is stressful for OS and your application. Maybe you need something like "stackless" approach? Or on other side you may try CSP approach (lib for example: http://www.cs.kent.ac.uk/projects/ofa/c++csp/) - that is extremely effective to bypass threads limitations.

[andre]

Guys... this question is almost 3 years old now... Why pull it from the grave again?

[AlterX]

You're right, but I didn't see a valid response to resolve that problem...so in the future my response can help someone else.