Qthread creation error

[rafaelSorel]

Well I found out where the limitation came from, indeed the default thread stack size is set by the system is:

stack size              (kbytes, -s) 8192

I have wrote a simple C program to see what is the limit of the thread creation, and even if I create the thread and destroy it, and create the next one , the max is always 251, the system does not decrement the total allowed thread value. to increase the thread number , we need to decrease the default stack size set by the system.
Typically each of the created thread will get this amount of memory (8MB) assigned for it's stack. With a 32bit program and a maximum Virtual mem address space of 2GB, that is a maximum of only 2048MB / 8MB = 256 threads ! Minus program code, minus heap-space will probably lead to the observed max. of 251 threads.

I don't understand Why even if I destroy the previous created thread it does not take this into consideration.

[jsulm]

@rafaelSorel You can simply use QThreadPool as @SGaist suggested

[rafaelSorel]

@jsulm said in Qthread creation error:

QThreadPool

Okey I will give a try, using the QThreadpool, the thing is that I have to implment my self all the QEventQueu that I am actually using in my Qthread runing obj which is auto handled in QThread classes.

[SGaist]

Why would you need to do that ?

[rafaelSorel]

@SGaist Need to do what ?

[SGaist]

I was referring to that:

@rafaelSorel said in Qthread creation error:

the thing is that I have to implment my self all the QEventQueu that I am actually using in my Qthread runing obj which is auto handled in QThread classes

[rafaelSorel]

Well I have tried to use QThreadpool and it appears that this class is provided for quick task and in my case it is not the case at all, I have many slots and signals on my worker class, so it is painful to use QRunnable in order to suit all my case.

[SGaist]

Can you describe what your class does ?

[rafaelSorel]

My class fetches stream data from a remote server (basically video data) and send this data o our local decoder. so the class itself look like this:

class inputStreamHttp : public QObject
{
    Q_OBJECT
public:
    enum
    {
        STREAMING_ACCEPTING,
        STREAMING_IN_PROGRESS,
        STREAMING_STOPPING,
        STREAMING_STOPPED
    };

    typedef struct
    {
        int socket;
        int state;
    } streaming_server_t;

    inputStreamHttp(QString);
    ~inputStreamHttp();
    void Start();
    void Close();
    void execGetRequest();

signals:
    void httpStreamInterrupted();
    void finished();
    void httpStreamReady(QString);

public slots:
    void onNewLinkAvailable();
    void onGetReqTimeout();
    void onServerReady();

private slots:
    void data_ready();
    void finish_get();
    void onCurrentSessionError(QNetworkReply::NetworkError);

private:
    void startget();
    void cleanUp();

    HttpServer * m_http_server;
    QTimer * m_getReqTimer;
    QNetworkReply * m_network_reply;
    QString m_main_url_str;
    QNetworkAccessManager * m_http_headers_obj;
    QMutex m_stop_pending_mutex;
    streaming_server_t m_server;
    bool m_httpPendingStop;
};

I found a way to bypass this limitation, by creating two threads and queue them, then alternete between those two threads each time a new connection came:

#define MAX_THREAD_COUNT 2

    for(unsigned int i=0; i<MAX_THREAD_COUNT; i++)
    {
        QThread * httpthread = new QThread();
        m_thread_queue.enqueue(httpthread);
    }

After that, I could use the thread queue to alternate between the two threads:

    m_httpMediaProc = new inputStreamHttp(*httpStream);
    m_httpthread = m_thread_queue.dequeue();
    m_httpMediaProc->moveToThread(m_httpthread);

and of course enqueue the already used one when session is closed.

[SGaist]

Out of curiosity, why do you need a QNetworkAccessManager per thread ?

[rafaelSorel]

@rafaelSorel said in Qthread creation error:

m_http_headers_obj

This is how it was designed at first place, any way the obj itself is deleted at each new connection.
Here is the Qthreadpool like that I have made but using real Qthreads:

class OMX_QThreadPool: public QObject
{
    Q_OBJECT
public:
    OMX_QThreadPool():
        QObject()
      , m_active_thread(NULL)
      , m_reserved_thread(0)
    {}
    ~OMX_QThreadPool()
    {m_thread_queue.clear();
    }

    inline void setMaxThreadCount(int maxThreadCount)
    {
        for (int i = 0; i < maxThreadCount; i++) {
            QThread * httpthread = new QThread();
            m_thread_queue.enqueue(httpthread);
            m_reserved_thread++;
        }
    }

    inline void reserveThread()
    {
        QMutexLocker locker(&m_mutex);
        if (m_thread_queue.isEmpty())
        {
            QThread * fallbackthread = new QThread();
            m_reserved_thread++;
            m_active_thread = fallbackthread;
        }
        else
        {
            m_active_thread = m_thread_queue.dequeue();
        }
    }

    inline void start(QObject * obj)
    {
        QMutexLocker locker(&m_mutex);
        obj->moveToThread(m_active_thread);
        if(!m_active_thread->isRunning())
            m_active_thread->start();
    }

    inline void releaseThread()
    {
        QMutexLocker locker(&m_mutex);
        if (!m_thread_queue.contains(m_active_thread))
            m_thread_queue.enqueue(m_active_thread);
    }

    inline int activeThreadCount()
    {
        QMutexLocker locker(&m_mutex);
        return m_reserved_thread;
    }
private:
    QQueue<QThread *> m_thread_queue;
    QThread * m_active_thread;
    QMutex m_mutex;
    unsigned int m_reserved_thread;
};