Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. Mobile and Embedded
  4. Too much query executed with QSqlQuery cause fatal signal if executed in different thread
Forum Updated to NodeBB v4.3 + New Features

Too much query executed with QSqlQuery cause fatal signal if executed in different thread

Scheduled Pinned Locked Moved Solved Mobile and Embedded
26 Posts 3 Posters 3.4k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • C Offline
    C Offline
    cmra
    wrote on last edited by cmra
    #1

    If I execute a large number of query stored in qstringlist with qsqlquery in new thread, the application stop execution with this message:

    " F libc : Fatal signal 7 (SIGBUS), code 1, fault addr 0x1e in tid 855 (qtMainLoopThrea)"

    while execution of queries on database (after about 300 queries in release mode and after about 700 queries in debug mode).
    The database is qsqlite and there is just one connection opened.
    Please help me!

    1 Reply Last reply
    0
    • SGaistS Offline
      SGaistS Offline
      SGaist
      Lifetime Qt Champion
      wrote on last edited by
      #2

      Hi,

      AFAIR, the correct way to do that is to open a new connection to the database per thread. On a side note, are you really creating 300 threads in parallel to execute queries ? A QThreadPool might be of interest in this case.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      1 Reply Last reply
      0
      • C Offline
        C Offline
        cmra
        wrote on last edited by
        #3

        It's not an SQL error in the query, it seam a memory leak problem or an other exception, but I can't discover what it is.

        1 Reply Last reply
        0
        • SGaistS Offline
          SGaistS Offline
          SGaist
          Lifetime Qt Champion
          wrote on last edited by
          #4

          At no point did I say the query was the problem. What I wrote is that you should open one connection to the database per thread and not use the same connection from all the threads.

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          1 Reply Last reply
          0
          • C Offline
            C Offline
            cmra
            wrote on last edited by
            #5

            Yeah I already use QThreadPool to exec the queries in the new thread and already use a new connection in the new thread, but I don't create 300 thread but i create a new one thread and inside this I would run 33000 queries, but the app stop after about 300.

            1 Reply Last reply
            0
            • SGaistS Offline
              SGaistS Offline
              SGaist
              Lifetime Qt Champion
              wrote on last edited by
              #6

              Can you show how you do that ?

              Out of curiosity, why use SQLite for what seems to be a pretty intensive database use ? Something like PostgreSQL might be a better fit.

              Interested in AI ? www.idiap.ch
              Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

              1 Reply Last reply
              1
              • C Offline
                C Offline
                cmra
                wrote on last edited by
                #7

                baseThreadStart(){
                if (thPool == nullptr){
                thPool = new QThreadPool(this);
                thPool->setExpiryTimeout(-1);
                }
                else {
                thPool->clear();
                }

                thPool->start(exQry);
                }

                QRunnableDerivedObject::run(){
                QSqlDatabase db = QSqlDatabase::database(connName);
                if (!db.isValid()){
                db = QSqlDatabase::addDatabase(tipoDb,connName);
                db.setDatabaseName(dbFullPath);
                }

                if (!db.open()){
                    emit errore("Database non accessibile\n" +
                                db.lastError().text());
                    db.removeDatabase(connName);
                    return;
                }else {
                    if (!queryList.empty()){
                        t = queryList.count();
                        emit totaleOp(c + 1, t);
                
                        ferma = false;
                
                        for (int i = c; i < t; i++) {
                            if (ferma)
                                break;
                            QString qrStr = queryList.at(i);
                
                            if (!qrStr.isEmpty()){
                                QSqlQuery qr(qrStr,db);
                

                #ifdef QT_DEBUG
                qDebug() << QString::number(i + 1) + ": " + qrStr;
                #endif

                                if (!qr.exec(qrStr)) {
                
                                    emit errore("Impossibile completare un'operazione sul database: \n" +
                                                db.databaseName() + "\n" +
                                                "Errore: " +
                                                qr.lastError().text() +
                                                "\n" + qrStr);
                                    emit log(qrStr);
                                }
                            }
                            emit totaleOp(i + 1,t);
                        }
                    }
                

                ....sorry for delay but I need to wait 10 minutes between reply

                JonBJ 1 Reply Last reply
                0
                • C cmra

                  baseThreadStart(){
                  if (thPool == nullptr){
                  thPool = new QThreadPool(this);
                  thPool->setExpiryTimeout(-1);
                  }
                  else {
                  thPool->clear();
                  }

                  thPool->start(exQry);
                  }

                  QRunnableDerivedObject::run(){
                  QSqlDatabase db = QSqlDatabase::database(connName);
                  if (!db.isValid()){
                  db = QSqlDatabase::addDatabase(tipoDb,connName);
                  db.setDatabaseName(dbFullPath);
                  }

                  if (!db.open()){
                      emit errore("Database non accessibile\n" +
                                  db.lastError().text());
                      db.removeDatabase(connName);
                      return;
                  }else {
                      if (!queryList.empty()){
                          t = queryList.count();
                          emit totaleOp(c + 1, t);
                  
                          ferma = false;
                  
                          for (int i = c; i < t; i++) {
                              if (ferma)
                                  break;
                              QString qrStr = queryList.at(i);
                  
                              if (!qrStr.isEmpty()){
                                  QSqlQuery qr(qrStr,db);
                  

                  #ifdef QT_DEBUG
                  qDebug() << QString::number(i + 1) + ": " + qrStr;
                  #endif

                                  if (!qr.exec(qrStr)) {
                  
                                      emit errore("Impossibile completare un'operazione sul database: \n" +
                                                  db.databaseName() + "\n" +
                                                  "Errore: " +
                                                  qr.lastError().text() +
                                                  "\n" + qrStr);
                                      emit log(qrStr);
                                  }
                              }
                              emit totaleOp(i + 1,t);
                          }
                      }
                  

                  ....sorry for delay but I need to wait 10 minutes between reply

                  JonBJ Online
                  JonBJ Online
                  JonB
                  wrote on last edited by
                  #8

                  @cmra
                  I'm just throwing this in there: are you sure that where you re-use/re-create the QSqlDatabase in each thread will not run into race condition with other parallel threads doing the same? Just a thought....

                  1 Reply Last reply
                  0
                  • C Offline
                    C Offline
                    cmra
                    wrote on last edited by
                    #9

                    Yes I'm sure because I use a different thread only to import data from csv into db and therefore I have to run too much query.
                    So I just do one import at time.

                    JonBJ 1 Reply Last reply
                    0
                    • C cmra

                      Yes I'm sure because I use a different thread only to import data from csv into db and therefore I have to run too much query.
                      So I just do one import at time.

                      JonBJ Online
                      JonBJ Online
                      JonB
                      wrote on last edited by
                      #10

                      @cmra
                      OIC.
                      OK, so it's many queries in just one thread. What about putting in some kind of processEvents() call within your loop, to see whether if you give Qt a chance to clear up that causes the error to go away?

                      1 Reply Last reply
                      0
                      • SGaistS Offline
                        SGaistS Offline
                        SGaist
                        Lifetime Qt Champion
                        wrote on last edited by
                        #11

                        @cmra said in Too much query executed with QSqlQuery cause fatal signal if executed in different thread:

                        connName

                        Where is it defined ? What does it contain ?

                        Interested in AI ? www.idiap.ch
                        Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                        1 Reply Last reply
                        0
                        • C Offline
                          C Offline
                          cmra
                          wrote on last edited by
                          #12

                          The processEvents() calls does not change nothing (I call it at line number 300 and the error happen at line 315 in release build). The error happen again.
                          connName is a global QString variable defined in the QRunnableDerivedObject and it's value is "async_conn", but this connection name is used only with one QSqlDatabase at start of importing process, no more.

                          1 Reply Last reply
                          0
                          • SGaistS Offline
                            SGaistS Offline
                            SGaist
                            Lifetime Qt Champion
                            wrote on last edited by
                            #13

                            So there's your problem: you are creating connections with the exact same name in all of your threads. You should make the name unique per thread. For example add the thread id to the connection string.

                            Interested in AI ? www.idiap.ch
                            Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                            C 1 Reply Last reply
                            1
                            • C Offline
                              C Offline
                              cmra
                              wrote on last edited by
                              #14

                              No it doesn't...because as I explained before I have just one thread and so just one connection.. I create only one thread fot importing data and no other. The connections in the main app don't use thread to exec query and they have different names.
                              I don't beleave that QThreadPool create more than one thread itself.

                              1 Reply Last reply
                              0
                              • C Offline
                                C Offline
                                cmra
                                wrote on last edited by
                                #15

                                I'm developing for and running the app on an Android Device.

                                1 Reply Last reply
                                0
                                • SGaistS SGaist

                                  So there's your problem: you are creating connections with the exact same name in all of your threads. You should make the name unique per thread. For example add the thread id to the connection string.

                                  C Offline
                                  C Offline
                                  cmra
                                  wrote on last edited by
                                  #16

                                  @SGaist I tried also to add to the connName variable the thread Id by QThread::currentThreadId() so the new name is (for example): async_connca6b1920 but this don't solve my problem.

                                  1 Reply Last reply
                                  0
                                  • C Offline
                                    C Offline
                                    cmra
                                    wrote on last edited by
                                    #17

                                    I tried also to re install qt - android sdk and ndk- changing testing phone...but nothing of these solutions solve the problem...

                                    1 Reply Last reply
                                    0
                                    • SGaistS Offline
                                      SGaistS Offline
                                      SGaist
                                      Lifetime Qt Champion
                                      wrote on last edited by
                                      #18

                                      To be sure to understand you correctly, you are trying to execute 33000 queries on an Android phone ? What kind of device is that ?

                                      Interested in AI ? www.idiap.ch
                                      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                      1 Reply Last reply
                                      1
                                      • C Offline
                                        C Offline
                                        cmra
                                        wrote on last edited by cmra
                                        #19

                                        Yes it is right....I tried on Samsung S6 and Samsung j6. But I already execute with success all the queries in the past, but now I can't find what wrong...if depend by an update of android sdk or ndk or I made a mistake in some place (the code is so simple that I exclude that).
                                        So I ask for help to you because may be some errors that I can't see.

                                        1 Reply Last reply
                                        0
                                        • C Offline
                                          C Offline
                                          cmra
                                          wrote on last edited by cmra
                                          #20

                                          And I think that the problem is not the high number of queries, because it fails after only 300... 300 is not too much.

                                          1 Reply Last reply
                                          0

                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Users
                                          • Groups
                                          • Search
                                          • Get Qt Extensions
                                          • Unsolved