QThreads, and constantly reusing the same Thread objects.

[Catherine Olsen]

Alright, what I am attempting to do is to get data into a database by using a pre-determined number of QThreads for scientific purposes.

My first attempts were unsuccessful due to my database handling class improperly allocating their constructors and therefore having duplicate connections, so I have fixed that in another side project update and now I'm attempting for the QThreads to basically "wait" for data to be passed to them after now successfully processing the first batch I pass to all the threads (btw, this does work beautifully up until this point), and not be destroyed... just looking to clean up the objects in my class and have them sit, wait, and not be in the running state. It seems I'm just destroying the threads.

The problem is the API is insanely vague on how to do this if any information at all, and it's driving me nuts.

I've renamed a few classes below to keep my proprietary code safe... while hopefully keeping the general flow ready to go just by using the header info. So apologies if this is vague.

class InfoDecodingHorse : public QObject
{
	Q_OBJECT
  public slots:
	  void InfoDecodingObject();
		//void threadFinished();

  public:
		void setFileName(QString &filename){ myFileName = filename; }
		void setDBHandle(databaseHandler &handle){ myHandleInfo = handle; }
		void setSubgridStr(QString &subgrid){ mSubgrid = subgrid; }
		void setThreadId(int num) { myThreadId = num; }

	signals:
		//this signal will only be emitted after the entire file is decoded
		void decodingComplete();

  private:
	  QString myFileName;
	  QByteArray gribStream;
	  databaseHandler myHandleInfo;
	  QString mSubgrid;
		int myThreadId;
};

class DataDecodingThread :public QObject
{
	Q_OBJECT
		QThread decodingThread;

  public:
		DataDecodingThread(int num) //default constructor
		{
			decoder = new DataDecodingHorse;
			threadNumber = num;
			decoder->setThreadId(num);
			decoder->moveToThread(&decodingThread);
			//connect(&decodingThread, &QThread::finished, decoder, );
			connect(this, SIGNAL(doDecoding()), decoder, SLOT(InfoDecodingObject()));
			connect(decoder, &DataDecodingHorse::decodingComplete, decoder, &QObject::deleteLater);
			connect(decoder, &DataDecodingHorse::decodingComplete, &decodingThread, &QThread::quit);
			connect(&decodingThread, SIGNAL(finished()), &decodingThread, SLOT(deleteLater()));
		};

		//parameterized constructor in case somehow I find a way to use this sucker
		DataDecodingThread(QString &fileName, QString &subgrid, int num)
		{
			decoder = new DataDecodingHorse;
			threadNumber = num;
			decoder->setFileName(fileName);
			decoder->setSubgridStr(subgrid);
			decoder->moveToThread(&decodingThread);
			//connect(&decodingThread, &QThread::finished, decoder, );
			connect(this, SIGNAL(doDecoding()), decoder, SLOT(InfoDecodingObject()));
			connect(decoder, &DataDecodingHorse::decodingComplete, this, &QObject::deleteLater);
		};

		~DataDecodingThread()
		{
			decodingThread.quit();
			decodingThread.wait();
		}

		void setInfoDecodingObjectFilename(QString &fileName){ (*decoder).setFileName(fileName); }
		void setInfoDecodingObjectSubgrid(QString &subgrid){ (*decoder).setSubgridStr(subgrid); }
		void setInfoDecodingObjectDBInfo(databaseHandler &handle){ (*decoder).setDBHandle(handle); }
		
		int getThreadNumber(void){ return threadNumber; }
		void startThread(void) { decodingThread.start(); }

		bool myThreadIsRunning(void){ return decodingThread.isRunning(); }

	//public slots:
	//  void uploadResultsToDB();

	signals:
		void doDecoding();
		void threadIsDone();

  private:
		DataDecodingHorse *decoder;
		int threadNumber;
};

Basically, this code follows this pesudocode:

• call threading process handler
  • declare thread objects with numerical constructor to handle db connections cleanly
  • see if compressed data streams exists in directory, if so....
    • Allocate filename and other gridded information into private variables of the thread handling class to keep mutexes to a minimum for speed.
    • Start thread, currently executes cleanly.
    • emit DecodingDone signal in the object

After emitted the finishing signal of the object, I just want to clean up the thread and the handlers private variables, and have the thread then wait for the next batch data if available/when it arrives.

I know, complicated. Need help! Let me know if I need to clarify things.

[kshegunov]

@Catherine-Olsen
Hello fellow scientist,
If you don't want your threads to quit, then just don't call their quit slot. In your code you connect the DataDecodingHorse::decodingComplete signal to the QThread::quit, so when decoding has completed your thread will exit.
This is the relevant part of your code:

// These two lines just close the thread and destroy the worker object
connect(decoder, &DataDecodingHorse::decodingComplete, decoder, &QObject::deleteLater);
connect(decoder, &DataDecodingHorse::decodingComplete, &decodingThread, &QThread::quit);

Something caught my eye though, this line:

connect(&decodingThread, SIGNAL(finished()), &decodingThread, SLOT(deleteLater()));

seems like an error, since you're creating your thread as a member and calling the deleteLater() slot will cause your program to segfault; you can't delete such objects, they are cleaned for you by the runtime. One additional remark here:

(*decoder).

is simply:

decoder->

Additionally, you should use QSqlDatabase and its friends from a single thread, as it's not reentrant. Consider this simple example how one could process things in batches with a running event loop:

class MyWorker : public QObject
{
    Q_OBJECT

signals:
    void workFinished(MyResult result);

slots:
    void doWork(MyInputData data)
    {
         // ... Do some work with data
         // ... Fill up a result of type MyResult and notify anyone that's interested
         emit workFinished(result);
     }
};

class MyControllerClass : public QObject
{
    Q_OBJECT
signals:
    void needWorkDone(MyInputData);
    void quit();

slots:
    void resultsAvailable(MyResult result)
    {
        // ... Do something with the result
    }

    void startBatch()
    {
        if (noMoreData)
            emit quit();

        // ... Get the needed input data
        emit needWorkDone(data);
        QObject::invokeMethod(this, "startBatch", Qt::QueuedConnection);
    }

    void workerFinished()
    {
        // ... The worker has finished, post quit to the application
        qApp->quit();
    }
};

int main(int argc, char ** argv)
{
    QApplication app(argc, argv);
    MyControllerClass controller;
    QObject::invokeMethod(&controller, "startBatch", Qt::QueuedConnection);

    // Start the worker thread and attach the worker object to it
    QThread workerThread;
    workerThread.start();

    MyWorker * worker = new MyWorker;
    worker->moveToThread(&workerThread);

    QObject::connect(&controller, &MyControllerClass::needWorkDone, worker, &MyWorker::doWork);
    QObject::connect(worker, &MyWorker::workFinished, &controller, &MyControllerClass::resultsAvailable);
    QObject::connect(&controller, &MyControllerClass::quit, &workerThread, &QThread::quit);
    QObject::connect(&workerThread, &QThread::finished, &controller, &MyControllerClass::workerFinished);

    return QApplication::exec();
}

I hope this helps.

Kind regards.

[Catherine Olsen]

After lengthy discussions on how this would work, this is solved!

[kshegunov]

@Catherine-Olsen
Could you mark it as such? The topic tools button is on the bottom and when you click it select "Mark as solved".