Starting multiple threads - at once or with a small break in between?

[d2uriel]

Dear Qt community,

I'm writing an application which is using multiple threads to load some data to keep the GUI responsive. Number of threads and work time for each thread is unknown - depends on the amount of data to load. At first I'm calculating the amount of threads needed to load the data. This number may be between 1 and 100. Now the question is - should I start all of those threads in one loop like:
@for(int i = 0; i < amountOfThreadsNeeded; i++) {
MyThread* t = new MyThread();
connect(t, SIGNAL(finished()), SLOT(threadFinished()));
t->start();
}@

or should I do it using a QTimer like:

@void startThread() {
MyThread* t = new MyThread();
connect(t, SIGNAL(finished()), SLOT(threadFinished()));
t->start();
threadsStarted++;
if(threadsStarted == amountOfThreadsNeeded) {
timer.stop();
}
}

int main() {
QTimer timer;
timer.setInterval(50);
connect(timer, SIGNAL(timeout()), SLOT(startThread()));
timer.start();
}@

What's the best way? Or maybe there's another solution?

Thanks you in advance for your help.

[Jeroentjehome]

Hi,
The number of threads doesn't really matter in this case. When more threads then cores are created the threads will be run consecutive of each other. So maybe limit the number of threads to minimize the changes of threadraces etc.
Maybe take a look at QThreadPool "QThreadPool":http://qt-project.org/doc/qt-5.0/qtcore/qthreadpool.html#details
It is designed for this kind of implementation.

[d2uriel]

The thing is that as soon as the thread is done loading the data I need to get a pointer to a QList of pointers to objects that this thread created. As soon as the thread is deleted the list itself is deleted as well (not the objects pointed in the list). Therefore I need to get the list and copy all pointers from it to a different list before I delete the thread.

Let me put some actual code in so you can tell me if I'm doing things right.

RLT.h
@class RLT : public QThread {
Q_OBJECT

public:
explicit RLT(QString folder, int startFrom, QObject parent = 0);
~RLT();
QList<Reps>* reps() { return _reps; }
QString folder() { return _folder; }
void setToDelete() { _toDelete = true; }
bool toDelete() { return _toDelete; }

private:
bool _toDelete;
QString _folder;
int _startFrom;
QList<Reps*>* _reps;

protected:
virtual void run();
};@

RLT.cpp
@RLT::RLT(QString folder, int startFrom, QObject parent) : QThread(parent) {
_toDelete = false;
_folder = folder;
_startFrom = startFrom;
_reps = new QList<Rep>();
}

RLT::~RLT() {
delete _reps;
}

void RLT::run() {
QDirIterator dirIterator(_folder, QDirIterator::Subdirectories);
int counter = 0;
while(dirIterator.hasNext()) {
QString fileName = dirIterator.next();
if(counter >= _startFrom && counter < _startFrom + REPS_PER_THREAD) {
Rep *r = new Rep(fileName);
if(r->load()) _reps->append(r);
else delete r;
} else if(counter > _startFrom + REPS_PER_THREAD) {
break;
}
counter++;
}
}@

The above class is iterating through a directory and loads up files into memory. Each thread gets a startFrom parameter which tells it how many files should it skip before starting to load so that one file won't get loaded twice.

Then in my MainWindow class I have:
header parts
@QList<RLT*> rlts;
QList<Rep*> reps;
bool allThreadsFinished;
QTimer loadingThreadsCheckTimer;
QMutex* mutex;
@

implementation
@MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), ui(new Ui::MainWindow) {
mutex = new QMutex();
maxLoadingThreads = fileCount / REPS_PER_THREAD + 1;
loadingThreadsCheckTimer.setInterval(100);
connect(&loadingThreadsCheckTimer, SIGNAL(timeout()), SLOT(checkLoadingThreadsFinished()));
QTimer::singleShot(1000, this, SLOT(loadRepsThreaded()));
}

// Starts needed amount of loading threads
void MainWindow::loadRepsThreaded() {
tableWidget->setSortingEnabled(false);
ui->statusBar->showMessage("Loading...");
if(QDir(settings->folder()).exists()) {
clearRepList(reps);
clearTableWidget(tableWidget);
for(int i = 0; i < maxLoadingThreads; i++) {
startLoadingThread(settings->folder());
}
}
loadingThreadsCheckTimer.start();
}

// This one actually creates the thread and appends it to a QList so I have track of them
void MainWindow::startLoadingThread(QString path) {
RLT* r = new RLT(path, threadsStarted * 100);
connect(r, SIGNAL(finished()), SLOT(repsLoaded()));
rlts.append(r);
r->start();
}

// This method checks if all threads have finished their job. If so we can delete them. It's being called every 100ms by a QTimer.
void MainWindow::checkLoadingThreadsFinished() {
allThreadsFinished = true;
for(int i = 0; i < rlts.count(); i++) {
if(!rlts[i]->isFinished() || !rlts[i]->toDelete()) {
allThreadsFinished = false;
break;
}
}
if(allThreadsFinished) {
while(!rlts.isEmpty()) {
delete rlts.takeFirst();
}
tableWidget->setSortingEnabled(true);
tableWidget->resizeColumnsToContents();
((QTimer*)sender())->stop();
ui->statusBar->showMessage("Loading reps...done.");
}
}

// This slot is being called every time a thread emits it's finished() signal. We just copy the Rep object pointers from the list and mark the thread as deletable. There's a mutex here... but I guess it's not really needed, is it?
void MainWindow::replaysLoaded() {
RLT* t = ((RLT*)sender());
QList<Rep*>* loadedReps = t->reps();
if(reps->count() > 0) {
bool locked = false;
while(!locked) {
locked = mutex->tryLock();
}
while(!loadedReps->isEmpty()) {
Rep* r = reps->takeFirst();
reps->append(r);
insertTableWidgetItems(tableWidget, r);
}
mutex->unlock();
}
t->setToDelete();
}
@