From 10:00 CET Friday 22nd November we will adjust how the server works to deal with some recently reported problems. Therefore there may be a load problem, if you experience more problems than usual trying to access the forum then please PM AndyS or any of the moderators so they can inform me.


Too much query executed with QSqlQuery cause fatal signal if executed in different thread



  • If I execute a large number of query stored in qstringlist with qsqlquery in new thread, the application stop execution with this message:

    " F libc : Fatal signal 7 (SIGBUS), code 1, fault addr 0x1e in tid 855 (qtMainLoopThrea)"

    while execution of queries on database (after about 300 queries in release mode and after about 700 queries in debug mode).
    The database is qsqlite and there is just one connection opened.
    Please help me!


  • Lifetime Qt Champion

    Hi,

    AFAIR, the correct way to do that is to open a new connection to the database per thread. On a side note, are you really creating 300 threads in parallel to execute queries ? A QThreadPool might be of interest in this case.



  • It's not an SQL error in the query, it seam a memory leak problem or an other exception, but I can't discover what it is.


  • Lifetime Qt Champion

    At no point did I say the query was the problem. What I wrote is that you should open one connection to the database per thread and not use the same connection from all the threads.



  • Yeah I already use QThreadPool to exec the queries in the new thread and already use a new connection in the new thread, but I don't create 300 thread but i create a new one thread and inside this I would run 33000 queries, but the app stop after about 300.


  • Lifetime Qt Champion

    Can you show how you do that ?

    Out of curiosity, why use SQLite for what seems to be a pretty intensive database use ? Something like PostgreSQL might be a better fit.



  • baseThreadStart(){
    if (thPool == nullptr){
    thPool = new QThreadPool(this);
    thPool->setExpiryTimeout(-1);
    }
    else {
    thPool->clear();
    }

    thPool->start(exQry);
    }

    QRunnableDerivedObject::run(){
    QSqlDatabase db = QSqlDatabase::database(connName);
    if (!db.isValid()){
    db = QSqlDatabase::addDatabase(tipoDb,connName);
    db.setDatabaseName(dbFullPath);
    }

    if (!db.open()){
        emit errore("Database non accessibile\n" +
                    db.lastError().text());
        db.removeDatabase(connName);
        return;
    }else {
        if (!queryList.empty()){
            t = queryList.count();
            emit totaleOp(c + 1, t);
    
            ferma = false;
    
            for (int i = c; i < t; i++) {
                if (ferma)
                    break;
                QString qrStr = queryList.at(i);
    
                if (!qrStr.isEmpty()){
                    QSqlQuery qr(qrStr,db);
    

    #ifdef QT_DEBUG
    qDebug() << QString::number(i + 1) + ": " + qrStr;
    #endif

                    if (!qr.exec(qrStr)) {
    
                        emit errore("Impossibile completare un'operazione sul database: \n" +
                                    db.databaseName() + "\n" +
                                    "Errore: " +
                                    qr.lastError().text() +
                                    "\n" + qrStr);
                        emit log(qrStr);
                    }
                }
                emit totaleOp(i + 1,t);
            }
        }
    

    ....sorry for delay but I need to wait 10 minutes between reply



  • @cmra
    I'm just throwing this in there: are you sure that where you re-use/re-create the QSqlDatabase in each thread will not run into race condition with other parallel threads doing the same? Just a thought....



  • Yes I'm sure because I use a different thread only to import data from csv into db and therefore I have to run too much query.
    So I just do one import at time.



  • @cmra
    OIC.
    OK, so it's many queries in just one thread. What about putting in some kind of processEvents() call within your loop, to see whether if you give Qt a chance to clear up that causes the error to go away?


  • Lifetime Qt Champion

    @cmra said in Too much query executed with QSqlQuery cause fatal signal if executed in different thread:

    connName

    Where is it defined ? What does it contain ?



  • The processEvents() calls does not change nothing (I call it at line number 300 and the error happen at line 315 in release build). The error happen again.
    connName is a global QString variable defined in the QRunnableDerivedObject and it's value is "async_conn", but this connection name is used only with one QSqlDatabase at start of importing process, no more.


  • Lifetime Qt Champion

    So there's your problem: you are creating connections with the exact same name in all of your threads. You should make the name unique per thread. For example add the thread id to the connection string.



  • No it doesn't...because as I explained before I have just one thread and so just one connection.. I create only one thread fot importing data and no other. The connections in the main app don't use thread to exec query and they have different names.
    I don't beleave that QThreadPool create more than one thread itself.



  • I'm developing for and running the app on an Android Device.



  • @SGaist I tried also to add to the connName variable the thread Id by QThread::currentThreadId() so the new name is (for example): async_connca6b1920 but this don't solve my problem.



  • I tried also to re install qt - android sdk and ndk- changing testing phone...but nothing of these solutions solve the problem...


  • Lifetime Qt Champion

    To be sure to understand you correctly, you are trying to execute 33000 queries on an Android phone ? What kind of device is that ?



  • Yes it is right....I tried on Samsung S6 and Samsung j6. But I already execute with success all the queries in the past, but now I can't find what wrong...if depend by an update of android sdk or ndk or I made a mistake in some place (the code is so simple that I exclude that).
    So I ask for help to you because may be some errors that I can't see.



  • And I think that the problem is not the high number of queries, because it fails after only 300... 300 is not too much.



  • ....and FINALLY I FOUND THE SOLUTIONS that wasn't in this part of code (this is correct)...
    I use reference & to a global variable in the base app and assign it to a pointer * = &var.. When declare as pointer also the global variable all works good.
    Thanks too much all people help me, but the problem was not here. I'm sorry to busy your time...
    Thanks again.


  • Lifetime Qt Champion

    Glad you found out !

    Global variable ?

    They should be avoided as much as possible. What is it and what are you doing with it ?



  • The global variables are custom classes to manage the tables of database derived by QObject class. I write inside them functions to read and write data and the tables structure.



  • The strange thing is that I don't use these variables in the new thread but I use only the QStringList passed.
    But the error happen only in the new thread during the import otherwise the app seam to work right.
    I don't know why.
    To develop the app I use qml and I use "qmlRegisterType()" to use c++ object.


  • Lifetime Qt Champion

    If you really need that kind of global object (which you likely don't), then you should rather implement them as singletons with proper access protection.



  • It's ok...thanks again for your time!


Log in to reply