Synchronizing main thread with multiple QThread object finishes



  • I have a need to create checksums for each file in a folder. The QApplication (main) thread creates and starts the process using a pool of 8 array indexes for QThreads. Creating each checksum is performed in the run() function of each QThread. When each QThread finishes, and there are still more files to checksum, a new QThread is constructed, its pointer placed into the available array index, and is start()-ed (within the finish() handler) with a new file name.

    Problem: if I allow the above to run asynchronously with the main thread, all files' checksums are created, and all threads correctly finish. But the main thread has no idea when this is completed.

    I would like to block (with cpu yielding) the main thread until all QThreads have finished. However, if I attempt to place any blocking code (based on QSemaphores, QWaitEvents or what have you), no thread finishes (i.e. the finish() slot is never entered--verified by posting a message if it is entered). There is no race condition (verified with system monitor). The main thread simply hangs and prevents finish() handling of any start()-ed QThread. There is no CPU utilization.

    Is there a way I can synchronize the main thread to wait for completion of all created/started QThreads?

    Here's the code [main thread]:
    @
    if (currentImageCount > 0) //create image info for uncached images
    {
    for (t = 0; t < THREADS; t++)//initialize array of QThreads; THREADS is 8
    pThread[t] = NULL;

    finx = 0;

    for (t = 0; t < THREADS; t++)
    {
    finxes[t] = finx;
    if (finx < currentImageCount)
    {

     pThread[t] = new CreateChecksumThread(destinationPath + "/" + currentImageNames.at(finx), &(crcs[t]));
    
     connect(pThread[t], SIGNAL(finished()), this, SLOT(slotSumFinished()));
     pThread[t]->start();
     finx++;
    }
    

    }
    // the above works fine until I attempt to put blocking/waiting/cpu yielding code here

    }
    @
    Here's the finish() handler:
    @
    slotSumFinished()
    // find out which thread finished
    for (t = 0; t < THREADS; t++)
    {
    if (pThread[t] != NULL)
    {
    if (pThread[t]->isFinished())
    {
    disconnect(pThread[t], SIGNAL(finished()), 0, 0);
    delete pThread[t];
    pThread[t] = NULL;

    checksums.append((qint64)crcs[t]);

    if ((finxes[t] = finxes[t] + THREADS) < currentImageCount)
    {
    // there are more files to checksum; reuse the index t in the array of QThreads
    pThread[t] = new CreateChecksumThread(destinationPath /imagePath/ + "/" + currentImageNames.at(finxes[t]), &(crcs[t]));
    connect(pThread[t], SIGNAL(finished()), this, SLOT(slotSumFinished()));
    pThread[t]->start();
    }
    else
    {
    for (t = 0; t < THREADS; t++)
    {
    if (pThread[t] != NULL) // some QThread(s) are still running
    return;
    }
    // all threads have completed; here is where I would like to wake up the main thread
    }

    }
    @



  • Hmmmmm.... I think I just (finally) figured out why the hang occurrs. The finish() handler is run in the main thread, but I've blocked it. DUH.

    Still, it seems a common enough desire to start a bunch of QThreads from a main thread and then wait (in the main thread) for them to complete (without using the QThread::wait()--since I'm running a bazillion threads 8 at a time. Maybe use of QThread::wait() is the only way. I'll try restructuring the program to use it.....


  • Lifetime Qt Champion

    Hi,

    You might be interest in the QtConcurrent module. It provides the base blocks for exactly your use case.

    Hope it helps



  • Thanks very much for your helpful response. I'll certainly check out QtConcurrent.

    In the meanwhile, I got my situation working by removing the finish() handler and putting its code behind a wait() in the main thread. Here is the relevant portion (in case someone else happens to find it helpful):
    @
    if (currentImageCount > 0) //create image info for remaining uncached images
    {
    finx = 0;

    for (t = 0; t < THREADS; t++)
    {
    pThread[t] = NULL;
    finxes[t] = finx;
    if (finx < currentImageCount)
    {

     pThread[t] = new CreateChecksumThread(destinationPath + "/" + currentImageNames.at(finx), &(crcs[t]));
    
     pThread[t]->start();
     finx++;
    }
    

    }

    do
    {
    bActiveThread = false;

    for (t = 0; t < THREADS; t++)
    {
     if (pThread[t] != NULL)
     {
      pThread[t]->wait();
      delete pThread[t];
      pThread[t] = NULL;
    
      if ((finxes[t] = finxes[t] + THREADS) < currentImageCount)
      {
    
       pThread[t] = new CreateChecksumThread(destinationPath /*imagePath*/ + "/" + currentImageNames.at(finxes[t]), &(crcs[t]));
    
       pThread[t]->start();
      }
    
      bActiveThread = true;
      continue;
     }
    }
    

    }
    while (bActiveThread);
    @

    PS: If you think the above is terrible practice for some reason, please let me know. It does work, however.

    Thanks again!


  • Moderators

    Hi,

    Using Qt Concurrent can produce much simpler code AND more efficient processing.

    Using wait() in your loop this way can result in idle threads. If the program starts waiting on thread X, but all the other threads finish before thread X, they can't be replaced with new threads until thread X finishes.

    Also, if you're waiting in the main thread, your GUI would be frozen while you're calculating the checksums.



  • Again, many thanks for the response. Yes, I've understood that a wait() can leave threads idle--hence my reluctance to use it. I'll definitely check out QtConcurrent.


  • Lifetime Qt Champion

    To add to JKSH and if you want to stay with your current design, you should rather use QThreadPool to provide you with threads



  • Thanks to your responses, I'm getting a ton of good info here. I'm working solo on a project that provides image archive management and image editing (see http://sourceforge.net/projects/cimphoto/?source=directory).

    The project now amounts to nearly 29,000 lines of code, and I've often settled for "good enough to work" rather than doing extensive research and optimization. BTW, I reported QtBug 30251, having to do with QThreads sending the finished() signal before setting the finished attribute to true.

    Again, many thanks for your kind and informative help.


  • Moderators

    You're welcome. :)

    For your interest: Qt Concurrent uses QThreadPool behind-the-scenes, but you can't write code that combines them (yet). So for now, Qt Concurrent and QThreadPool can be treated as separate technologies.

    Here is a page from the official documentation that describes the different ways to do parallel processing in Qt: http://qt-project.org/doc/qt-5/threads-technologies.html


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.