Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. Mobile and Embedded
  4. Too much query executed with QSqlQuery cause fatal signal if executed in different thread
Forum Updated to NodeBB v4.3 + New Features

Too much query executed with QSqlQuery cause fatal signal if executed in different thread

Scheduled Pinned Locked Moved Solved Mobile and Embedded
26 Posts 3 Posters 3.5k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • C Offline
    C Offline
    cmra
    wrote on last edited by
    #3

    It's not an SQL error in the query, it seam a memory leak problem or an other exception, but I can't discover what it is.

    1 Reply Last reply
    0
    • SGaistS Offline
      SGaistS Offline
      SGaist
      Lifetime Qt Champion
      wrote on last edited by
      #4

      At no point did I say the query was the problem. What I wrote is that you should open one connection to the database per thread and not use the same connection from all the threads.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      1 Reply Last reply
      0
      • C Offline
        C Offline
        cmra
        wrote on last edited by
        #5

        Yeah I already use QThreadPool to exec the queries in the new thread and already use a new connection in the new thread, but I don't create 300 thread but i create a new one thread and inside this I would run 33000 queries, but the app stop after about 300.

        1 Reply Last reply
        0
        • SGaistS Offline
          SGaistS Offline
          SGaist
          Lifetime Qt Champion
          wrote on last edited by
          #6

          Can you show how you do that ?

          Out of curiosity, why use SQLite for what seems to be a pretty intensive database use ? Something like PostgreSQL might be a better fit.

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          1 Reply Last reply
          1
          • C Offline
            C Offline
            cmra
            wrote on last edited by
            #7

            baseThreadStart(){
            if (thPool == nullptr){
            thPool = new QThreadPool(this);
            thPool->setExpiryTimeout(-1);
            }
            else {
            thPool->clear();
            }

            thPool->start(exQry);
            }

            QRunnableDerivedObject::run(){
            QSqlDatabase db = QSqlDatabase::database(connName);
            if (!db.isValid()){
            db = QSqlDatabase::addDatabase(tipoDb,connName);
            db.setDatabaseName(dbFullPath);
            }

            if (!db.open()){
                emit errore("Database non accessibile\n" +
                            db.lastError().text());
                db.removeDatabase(connName);
                return;
            }else {
                if (!queryList.empty()){
                    t = queryList.count();
                    emit totaleOp(c + 1, t);
            
                    ferma = false;
            
                    for (int i = c; i < t; i++) {
                        if (ferma)
                            break;
                        QString qrStr = queryList.at(i);
            
                        if (!qrStr.isEmpty()){
                            QSqlQuery qr(qrStr,db);
            

            #ifdef QT_DEBUG
            qDebug() << QString::number(i + 1) + ": " + qrStr;
            #endif

                            if (!qr.exec(qrStr)) {
            
                                emit errore("Impossibile completare un'operazione sul database: \n" +
                                            db.databaseName() + "\n" +
                                            "Errore: " +
                                            qr.lastError().text() +
                                            "\n" + qrStr);
                                emit log(qrStr);
                            }
                        }
                        emit totaleOp(i + 1,t);
                    }
                }
            

            ....sorry for delay but I need to wait 10 minutes between reply

            JonBJ 1 Reply Last reply
            0
            • C cmra

              baseThreadStart(){
              if (thPool == nullptr){
              thPool = new QThreadPool(this);
              thPool->setExpiryTimeout(-1);
              }
              else {
              thPool->clear();
              }

              thPool->start(exQry);
              }

              QRunnableDerivedObject::run(){
              QSqlDatabase db = QSqlDatabase::database(connName);
              if (!db.isValid()){
              db = QSqlDatabase::addDatabase(tipoDb,connName);
              db.setDatabaseName(dbFullPath);
              }

              if (!db.open()){
                  emit errore("Database non accessibile\n" +
                              db.lastError().text());
                  db.removeDatabase(connName);
                  return;
              }else {
                  if (!queryList.empty()){
                      t = queryList.count();
                      emit totaleOp(c + 1, t);
              
                      ferma = false;
              
                      for (int i = c; i < t; i++) {
                          if (ferma)
                              break;
                          QString qrStr = queryList.at(i);
              
                          if (!qrStr.isEmpty()){
                              QSqlQuery qr(qrStr,db);
              

              #ifdef QT_DEBUG
              qDebug() << QString::number(i + 1) + ": " + qrStr;
              #endif

                              if (!qr.exec(qrStr)) {
              
                                  emit errore("Impossibile completare un'operazione sul database: \n" +
                                              db.databaseName() + "\n" +
                                              "Errore: " +
                                              qr.lastError().text() +
                                              "\n" + qrStr);
                                  emit log(qrStr);
                              }
                          }
                          emit totaleOp(i + 1,t);
                      }
                  }
              

              ....sorry for delay but I need to wait 10 minutes between reply

              JonBJ Online
              JonBJ Online
              JonB
              wrote on last edited by
              #8

              @cmra
              I'm just throwing this in there: are you sure that where you re-use/re-create the QSqlDatabase in each thread will not run into race condition with other parallel threads doing the same? Just a thought....

              1 Reply Last reply
              0
              • C Offline
                C Offline
                cmra
                wrote on last edited by
                #9

                Yes I'm sure because I use a different thread only to import data from csv into db and therefore I have to run too much query.
                So I just do one import at time.

                JonBJ 1 Reply Last reply
                0
                • C cmra

                  Yes I'm sure because I use a different thread only to import data from csv into db and therefore I have to run too much query.
                  So I just do one import at time.

                  JonBJ Online
                  JonBJ Online
                  JonB
                  wrote on last edited by
                  #10

                  @cmra
                  OIC.
                  OK, so it's many queries in just one thread. What about putting in some kind of processEvents() call within your loop, to see whether if you give Qt a chance to clear up that causes the error to go away?

                  1 Reply Last reply
                  0
                  • SGaistS Offline
                    SGaistS Offline
                    SGaist
                    Lifetime Qt Champion
                    wrote on last edited by
                    #11

                    @cmra said in Too much query executed with QSqlQuery cause fatal signal if executed in different thread:

                    connName

                    Where is it defined ? What does it contain ?

                    Interested in AI ? www.idiap.ch
                    Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                    1 Reply Last reply
                    0
                    • C Offline
                      C Offline
                      cmra
                      wrote on last edited by
                      #12

                      The processEvents() calls does not change nothing (I call it at line number 300 and the error happen at line 315 in release build). The error happen again.
                      connName is a global QString variable defined in the QRunnableDerivedObject and it's value is "async_conn", but this connection name is used only with one QSqlDatabase at start of importing process, no more.

                      1 Reply Last reply
                      0
                      • SGaistS Offline
                        SGaistS Offline
                        SGaist
                        Lifetime Qt Champion
                        wrote on last edited by
                        #13

                        So there's your problem: you are creating connections with the exact same name in all of your threads. You should make the name unique per thread. For example add the thread id to the connection string.

                        Interested in AI ? www.idiap.ch
                        Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                        C 1 Reply Last reply
                        1
                        • C Offline
                          C Offline
                          cmra
                          wrote on last edited by
                          #14

                          No it doesn't...because as I explained before I have just one thread and so just one connection.. I create only one thread fot importing data and no other. The connections in the main app don't use thread to exec query and they have different names.
                          I don't beleave that QThreadPool create more than one thread itself.

                          1 Reply Last reply
                          0
                          • C Offline
                            C Offline
                            cmra
                            wrote on last edited by
                            #15

                            I'm developing for and running the app on an Android Device.

                            1 Reply Last reply
                            0
                            • SGaistS SGaist

                              So there's your problem: you are creating connections with the exact same name in all of your threads. You should make the name unique per thread. For example add the thread id to the connection string.

                              C Offline
                              C Offline
                              cmra
                              wrote on last edited by
                              #16

                              @SGaist I tried also to add to the connName variable the thread Id by QThread::currentThreadId() so the new name is (for example): async_connca6b1920 but this don't solve my problem.

                              1 Reply Last reply
                              0
                              • C Offline
                                C Offline
                                cmra
                                wrote on last edited by
                                #17

                                I tried also to re install qt - android sdk and ndk- changing testing phone...but nothing of these solutions solve the problem...

                                1 Reply Last reply
                                0
                                • SGaistS Offline
                                  SGaistS Offline
                                  SGaist
                                  Lifetime Qt Champion
                                  wrote on last edited by
                                  #18

                                  To be sure to understand you correctly, you are trying to execute 33000 queries on an Android phone ? What kind of device is that ?

                                  Interested in AI ? www.idiap.ch
                                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                  1 Reply Last reply
                                  1
                                  • C Offline
                                    C Offline
                                    cmra
                                    wrote on last edited by cmra
                                    #19

                                    Yes it is right....I tried on Samsung S6 and Samsung j6. But I already execute with success all the queries in the past, but now I can't find what wrong...if depend by an update of android sdk or ndk or I made a mistake in some place (the code is so simple that I exclude that).
                                    So I ask for help to you because may be some errors that I can't see.

                                    1 Reply Last reply
                                    0
                                    • C Offline
                                      C Offline
                                      cmra
                                      wrote on last edited by cmra
                                      #20

                                      And I think that the problem is not the high number of queries, because it fails after only 300... 300 is not too much.

                                      1 Reply Last reply
                                      0
                                      • C Offline
                                        C Offline
                                        cmra
                                        wrote on last edited by
                                        #21

                                        ....and FINALLY I FOUND THE SOLUTIONS that wasn't in this part of code (this is correct)...
                                        I use reference & to a global variable in the base app and assign it to a pointer * = &var.. When declare as pointer also the global variable all works good.
                                        Thanks too much all people help me, but the problem was not here. I'm sorry to busy your time...
                                        Thanks again.

                                        1 Reply Last reply
                                        0
                                        • SGaistS Offline
                                          SGaistS Offline
                                          SGaist
                                          Lifetime Qt Champion
                                          wrote on last edited by
                                          #22

                                          Glad you found out !

                                          Global variable ?

                                          They should be avoided as much as possible. What is it and what are you doing with it ?

                                          Interested in AI ? www.idiap.ch
                                          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                          1 Reply Last reply
                                          1

                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Users
                                          • Groups
                                          • Search
                                          • Get Qt Extensions
                                          • Unsolved