Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. QThread with SQLite blocks GUI
Forum Update on Monday, May 27th 2025

QThread with SQLite blocks GUI

Scheduled Pinned Locked Moved Solved General and Desktop
6 Posts 3 Posters 562 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • NiagarerN Offline
    NiagarerN Offline
    Niagarer
    wrote on last edited by Niagarer
    #1

    Hi,
    in my program, a second thread, meant to extract data from one database and store it in another, gets created.

    from mainwindow.cpp

    void MainWindow::processFile(){
        if(unfinishedFilenames->length() > 0){
            QString path = unfinishedFilenames->at(0);
            QString filename = path.mid(path.lastIndexOf("/")+1);
            qDebug() << "processing file: " << filename;
    
            CurrentWeatherWorker *cww = new CurrentWeatherWorker(path, filename, targetDir);
            connect(cww, SIGNAL(fileProcessingFinished(QString)),
                    this, SLOT(fileProcessingFinished(QString)));
            connect(cww, SIGNAL(updateProgress(int)),
                    this, SLOT(updateProgressBar(int)));
            connect(cww, SIGNAL(log(QString)),
                    this, SLOT(log(QString)));
            cww->start();
        }
    
    }
    

    currentweatherworker.cpp

    #include "CurrentWeatherWorker.h"
    
    #include "dataobjects.h"
    
    #include <QJsonDocument>
    #include <QJsonObject>
    #include <QJsonArray>
    #include <QtSql/QSqlDatabase>
    #include <QSqlQuery>
    #include <QSqlError>
    #include <QVariant>
    
    
    CurrentWeatherWorker::CurrentWeatherWorker(QString _filepath, QString _filename, QString _targetDir)
    {
        filepath = _filepath;
        filename = _filename;
        targetDir = _targetDir;
    }
    
    void CurrentWeatherWorker::run(){
        log("opening database...");
    
        QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
        db.setDatabaseName(filepath);
        if(!db.open()){
            return;
        }
    
        QSqlQuery query;
    
        QList<CurrentData_meta> metaList;
    
        int numRecords = -1;
        query.exec("SELECT ID FROM Data");
        query.last();
        numRecords = query.value("ID").toInt();
    
        query.exec("SELECT * FROM Data");
        int i=0;
        while(query.next()){
            CurrentData_meta r;
            r.key = query.value("Key").toString();
            // ...
    
            metaList.append(r);
    
            log("data object read");
    
            emit updateProgress((100*i++)/(100*numRecords));
        }
    
        QString day = "";
        for(int i=9; i<filename.length(); i++){
            day+=filename.at(i);
        }
        QString newFilename = targetDir+"meta_data_current_"+day;
    
        saveMetaData_currentWeather(metaList, newFilename);
    
        emit updateProgress(100);
    
        emit fileProcessingFinished(filepath);
    }
    
    
    void CurrentWeatherWorker::saveMetaData_currentWeather(QList<CurrentData_meta> metaList, QString filename){
    
        log("creating new database...");
    
        QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
    
        db.setDatabaseName(filename);
        if(!db.open()){
            return;
        }
    
        QSqlQuery query;
    
        // creating table
    
        // preparing query
        for(int i=0; i<metaList.length(); i++){
            CurrentData_meta r = metaList[i];
    
            query.bindValue(":Key", r.key);
            // ...
            query.exec();
    
            QString data = "";
            QMap<QString, QVariant> map = query.boundValues();
            for(auto e : map.keys()){
                data += e + ": " + map.value(e).toString() + "; ";
            }
    
            log("data inserted: "+data);
    
            emit updateProgress((100*i)/(100*metaList.length()));
        }
        db.close();
        log("successfully finished saving data");
    }
    
    

    The worker works, but blocks the mainwindow (program not responding is shown by Windows) while it is running. Normalls this is due to a dependency between the two classes, which is why I use Qt signals an slots for communication, nothing else and I can't find another. The dataobjects.h file does not include anything. Sometimes, the mainwindow responds again but just for a couple of seconds (which is strange), then it's gone again.
    Do you see something causing a connection between MainWindow and CurrentWeatherWorker responsible for the block? Does SQLite have something to do with it?
    Thanks for answers!

    CKurduC 1 Reply Last reply
    0
    • CKurduC Offline
      CKurduC Offline
      CKurdu
      wrote on last edited by
      #2

      @Niagarer said in QThread with SQLite blocks GUI:

      log("opening database...");

      Because I don't see where this log implemented, I am guessing this system-wide log is responsible for that. You can try to replace log calls like below.

      emit  log("data object read parsed successfully");
      

      You reap what you sow it

      NiagarerN 1 Reply Last reply
      0
      • CKurduC CKurdu

        @Niagarer said in QThread with SQLite blocks GUI:

        log("opening database...");

        Because I don't see where this log implemented, I am guessing this system-wide log is responsible for that. You can try to replace log calls like below.

        emit  log("data object read parsed successfully");
        
        NiagarerN Offline
        NiagarerN Offline
        Niagarer
        wrote on last edited by
        #3

        @CKurdu
        Oh, thank you, didn't see that!
        Unfortunately, it still blocks the GUI.

        1 Reply Last reply
        0
        • NiagarerN Niagarer

          Hi,
          in my program, a second thread, meant to extract data from one database and store it in another, gets created.

          from mainwindow.cpp

          void MainWindow::processFile(){
              if(unfinishedFilenames->length() > 0){
                  QString path = unfinishedFilenames->at(0);
                  QString filename = path.mid(path.lastIndexOf("/")+1);
                  qDebug() << "processing file: " << filename;
          
                  CurrentWeatherWorker *cww = new CurrentWeatherWorker(path, filename, targetDir);
                  connect(cww, SIGNAL(fileProcessingFinished(QString)),
                          this, SLOT(fileProcessingFinished(QString)));
                  connect(cww, SIGNAL(updateProgress(int)),
                          this, SLOT(updateProgressBar(int)));
                  connect(cww, SIGNAL(log(QString)),
                          this, SLOT(log(QString)));
                  cww->start();
              }
          
          }
          

          currentweatherworker.cpp

          #include "CurrentWeatherWorker.h"
          
          #include "dataobjects.h"
          
          #include <QJsonDocument>
          #include <QJsonObject>
          #include <QJsonArray>
          #include <QtSql/QSqlDatabase>
          #include <QSqlQuery>
          #include <QSqlError>
          #include <QVariant>
          
          
          CurrentWeatherWorker::CurrentWeatherWorker(QString _filepath, QString _filename, QString _targetDir)
          {
              filepath = _filepath;
              filename = _filename;
              targetDir = _targetDir;
          }
          
          void CurrentWeatherWorker::run(){
              log("opening database...");
          
              QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
              db.setDatabaseName(filepath);
              if(!db.open()){
                  return;
              }
          
              QSqlQuery query;
          
              QList<CurrentData_meta> metaList;
          
              int numRecords = -1;
              query.exec("SELECT ID FROM Data");
              query.last();
              numRecords = query.value("ID").toInt();
          
              query.exec("SELECT * FROM Data");
              int i=0;
              while(query.next()){
                  CurrentData_meta r;
                  r.key = query.value("Key").toString();
                  // ...
          
                  metaList.append(r);
          
                  log("data object read");
          
                  emit updateProgress((100*i++)/(100*numRecords));
              }
          
              QString day = "";
              for(int i=9; i<filename.length(); i++){
                  day+=filename.at(i);
              }
              QString newFilename = targetDir+"meta_data_current_"+day;
          
              saveMetaData_currentWeather(metaList, newFilename);
          
              emit updateProgress(100);
          
              emit fileProcessingFinished(filepath);
          }
          
          
          void CurrentWeatherWorker::saveMetaData_currentWeather(QList<CurrentData_meta> metaList, QString filename){
          
              log("creating new database...");
          
              QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
          
              db.setDatabaseName(filename);
              if(!db.open()){
                  return;
              }
          
              QSqlQuery query;
          
              // creating table
          
              // preparing query
              for(int i=0; i<metaList.length(); i++){
                  CurrentData_meta r = metaList[i];
          
                  query.bindValue(":Key", r.key);
                  // ...
                  query.exec();
          
                  QString data = "";
                  QMap<QString, QVariant> map = query.boundValues();
                  for(auto e : map.keys()){
                      data += e + ": " + map.value(e).toString() + "; ";
                  }
          
                  log("data inserted: "+data);
          
                  emit updateProgress((100*i)/(100*metaList.length()));
              }
              db.close();
              log("successfully finished saving data");
          }
          
          

          The worker works, but blocks the mainwindow (program not responding is shown by Windows) while it is running. Normalls this is due to a dependency between the two classes, which is why I use Qt signals an slots for communication, nothing else and I can't find another. The dataobjects.h file does not include anything. Sometimes, the mainwindow responds again but just for a couple of seconds (which is strange), then it's gone again.
          Do you see something causing a connection between MainWindow and CurrentWeatherWorker responsible for the block? Does SQLite have something to do with it?
          Thanks for answers!

          CKurduC Offline
          CKurduC Offline
          CKurdu
          wrote on last edited by CKurdu
          #4

          @Niagarer said in QThread with SQLite blocks GUI:

          emit updateProgress((100i++)/(100numRecords));

          Maybe this line the reason. Because you are sending many update progress messages to the main thread. Maybe you can implement like below code in two places ( run and saveMetaData_currentWeather methods)

          int progress = 0;
          const int range = 100;
          const int step = int(numRecords/range);
          while(query.next())
          {
          //... some codes
          if( (i%step )== 0 ) 
          {
               progress++;
               emit updateProgress(progress);
          }
          }
          

          You reap what you sow it

          1 Reply Last reply
          1
          • Christian EhrlicherC Offline
            Christian EhrlicherC Offline
            Christian Ehrlicher
            Lifetime Qt Champion
            wrote on last edited by
            #5

            QSqlQuery query;

            This is wrong when you're more than one db connection - you want 'QSqlQuery query(db)'

            ...saveMetaData_currentWeather()
            ...
            QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");

            Here you're opening the database again and kill the old connection.

            CurrentWeatherWorker *cww;

            cww is leaking...

            Apart from this I don't see what should block the ui thread apart from the stuff @CKurdu already mentioned.

            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
            Visit the Qt Academy at https://academy.qt.io/catalog

            1 Reply Last reply
            3
            • NiagarerN Offline
              NiagarerN Offline
              Niagarer
              wrote on last edited by Niagarer
              #6

              @CKurdu said in QThread with SQLite blocks GUI:

              Because you are sending many update progress messages to the main thread. Maybe you can implement like below code in two places ( run and saveMetaData_currentWeather methods)

              Oh yes, this is actually the problem. It's not the updateProgress, but this line above in CurrentWeatherWorker::run():

               emit log("data object read");
              

              Because reading the data is done really fast and about 170 000 calls in about 10 seconds seems too be to much to handle for the mainwindow (the message gets printed every time). The queue of the received signals is just too long.
              Anyway, changing the updateProgress to something like this:

                      if(i%(metaList.length()/100)==0)
                          emit updateProgress(qRound(100.0*(qreal)i/(qreal)metaList.length()));
              

              should increase performance sagnificantly while the result should stay the same.

              @Christian-Ehrlicher said in QThread with SQLite blocks GUI:

              This is wrong when you're more than one db connection - you want 'QSqlQuery query(db)'

              Yes, thanks. So far it even works the old way since I am only working on one database at a time yet, but I plan to run multiple workers simultaneously.

              Thank you!!

              1 Reply Last reply
              0

              • Login

              • Login or register to search.
              • First post
                Last post
              0
              • Categories
              • Recent
              • Tags
              • Popular
              • Users
              • Groups
              • Search
              • Get Qt Extensions
              • Unsolved