Too much query executed with QSqlQuery cause fatal signal if executed in different thread

cmra

It's not an SQL error in the query, it seam a memory leak problem or an other exception, but I can't discover what it is.

SGaist

At no point did I say the query was the problem. What I wrote is that you should open one connection to the database per thread and not use the same connection from all the threads.

cmra

Yeah I already use QThreadPool to exec the queries in the new thread and already use a new connection in the new thread, but I don't create 300 thread but i create a new one thread and inside this I would run 33000 queries, but the app stop after about 300.

SGaist

Can you show how you do that ?

Out of curiosity, why use SQLite for what seems to be a pretty intensive database use ? Something like PostgreSQL might be a better fit.

cmra

baseThreadStart(){
if (thPool == nullptr){
thPool = new QThreadPool(this);
thPool->setExpiryTimeout(-1);
}
else {
thPool->clear();
}

thPool->start(exQry);
}

QRunnableDerivedObject::run(){
QSqlDatabase db = QSqlDatabase::database(connName);
if (!db.isValid()){
db = QSqlDatabase::addDatabase(tipoDb,connName);
db.setDatabaseName(dbFullPath);
}

if (!db.open()){
    emit errore("Database non accessibile\n" +
                db.lastError().text());
    db.removeDatabase(connName);
    return;
}else {
    if (!queryList.empty()){
        t = queryList.count();
        emit totaleOp(c + 1, t);

        ferma = false;

        for (int i = c; i < t; i++) {
            if (ferma)
                break;
            QString qrStr = queryList.at(i);

            if (!qrStr.isEmpty()){
                QSqlQuery qr(qrStr,db);

#ifdef QT_DEBUG
qDebug() << QString::number(i + 1) + ": " + qrStr;
#endif

                if (!qr.exec(qrStr)) {

                    emit errore("Impossibile completare un'operazione sul database: \n" +
                                db.databaseName() + "\n" +
                                "Errore: " +
                                qr.lastError().text() +
                                "\n" + qrStr);
                    emit log(qrStr);
                }
            }
            emit totaleOp(i + 1,t);
        }
    }

....sorry for delay but I need to wait 10 minutes between reply

JonB

@cmra
I'm just throwing this in there: are you sure that where you re-use/re-create the QSqlDatabase in each thread will not run into race condition with other parallel threads doing the same? Just a thought....

cmra

Yes I'm sure because I use a different thread only to import data from csv into db and therefore I have to run too much query.
So I just do one import at time.

JonB

@cmra
OIC.
OK, so it's many queries in just one thread. What about putting in some kind of processEvents() call within your loop, to see whether if you give Qt a chance to clear up that causes the error to go away?

SGaist

@cmra said in Too much query executed with QSqlQuery cause fatal signal if executed in different thread:

connName

Where is it defined ? What does it contain ?

cmra

The processEvents() calls does not change nothing (I call it at line number 300 and the error happen at line 315 in release build). The error happen again.
connName is a global QString variable defined in the QRunnableDerivedObject and it's value is "async_conn", but this connection name is used only with one QSqlDatabase at start of importing process, no more.

SGaist

So there's your problem: you are creating connections with the exact same name in all of your threads. You should make the name unique per thread. For example add the thread id to the connection string.

cmra

No it doesn't...because as I explained before I have just one thread and so just one connection.. I create only one thread fot importing data and no other. The connections in the main app don't use thread to exec query and they have different names.
I don't beleave that QThreadPool create more than one thread itself.

cmra

I'm developing for and running the app on an Android Device.

cmra

@SGaist I tried also to add to the connName variable the thread Id by QThread::currentThreadId() so the new name is (for example): async_connca6b1920 but this don't solve my problem.

cmra

I tried also to re install qt - android sdk and ndk- changing testing phone...but nothing of these solutions solve the problem...

SGaist

To be sure to understand you correctly, you are trying to execute 33000 queries on an Android phone ? What kind of device is that ?

cmra

Yes it is right....I tried on Samsung S6 and Samsung j6. But I already execute with success all the queries in the past, but now I can't find what wrong...if depend by an update of android sdk or ndk or I made a mistake in some place (the code is so simple that I exclude that).
So I ask for help to you because may be some errors that I can't see.

cmra

And I think that the problem is not the high number of queries, because it fails after only 300... 300 is not too much.

cmra

....and FINALLY I FOUND THE SOLUTIONS that wasn't in this part of code (this is correct)...
I use reference & to a global variable in the base app and assign it to a pointer * = &var.. When declare as pointer also the global variable all works good.
Thanks too much all people help me, but the problem was not here. I'm sorry to busy your time...
Thanks again.

SGaist

Glad you found out !

Global variable ?

They should be avoided as much as possible. What is it and what are you doing with it ?