[Solved] QSqlDatabase memory leak.

[Davita]

Hi guys. I just found a strange behavior in QtSql module. I think it's a bug causing memory leak, but I wanted to be sure about it before reporting. I'm using Qt version 4.7.3, Sqlite driver and Windows 7 x64 edition. I found that memory leak occurs in two situation.

1. When I call QSqlDatabase::close(). Don't know why but if I call close explicitly on an instance of QSqlDatabase, a memory leak occurs. Well, I can live without it by not calling close and letting the destructor do the cleanup, but still, it's weird.
2. When an instance of QSqlDatabase is open and we try to acquire another instance of QSqlDatabase with same name (by using QSqlDatabase::database) and query db with the newly created instance, I get a memory leak.
   By memory leak, I mean that memory consumption goes up and stays. I can see in task manager that I hit several hundreds of MBs, even after the loop is finished, memory consumption doesn't drop down.

@ for(int i = 0; i < 1000; i++) {
int index = 0, limit = 10000;
QSqlDatabase db = createDatabase();
db.open();

    QSqlQuery query(db);
    query.prepare("SELECT * FROM History ORDER BY CreateDate DESC LIMIT :Index, :Limit");

    query.bindValue(":Index", index);
    query.bindValue(":Limit", limit);
    if(query.exec() && query.first()) {
        do
        {
            QSqlRecord record = query.record();
            QSharedPointer<History> history(readHistory(record, query));
            getApplication(query.value(record.indexOf("ApplicationId")).toUInt()); // This causes memory leak, because it creates a new instance of QSqlDatabase. Remove this line and memory leak is gone.
        }
        while(query.next());
    }
}

@

@void getApplication(uint id)
{
QSqlDatabase db = createDatabase();
db.open();

QSqlQuery query(db);
query.prepare("SELECT a.*, IFNULL(r.id, 0) AS HasRules FROM Applications AS a "
              "LEFT JOIN Rules r ON r.ApplicationId = a.id WHERE a.Id = :Id "
              "GROUP BY a.Id");
query.bindValue(":Id", id);
if(query.exec() && query.first()) {
    QSqlRecord record = query.record();
    //Do some stuff
}

}
@

@QSqlDatabase createDatabase()
{
QSqlDatabase db = QSqlDatabase::contains(CONN_NAME) ?
QSqlDatabase::database(CONN_NAME) :
QSqlDatabase::addDatabase(SQLITE_DRIVER_NAME, CONN_NAME);

db.setDatabaseName("C:/Temp/qtest/debug/data.dat");
return db;

}
@

http://dl.dropbox.com/u/3055964/qtest.zip <-- this is a ready to run project with source code, it tests both memory leak scenarios. Before running it, make sure to update data.dat file's path in createDatabase function.

How do you think guys, am I doing something wrong or it's really a memory leak?
Thank you very much, I appreciate your help :)

[Davita]

no one? :-(

[lgeyer]

QSqlDatabase is not known to leak memory, but various underlying drivers are. So if you run into memory leaks the native driver used might be the problem - valgrind or alike will help here.

And as far as I can judge from the code I've seen your application design tends to aid and abet such situations.

You have to be aware of the fact that a call to QSqlDatabase::open() on an already opened database causes (the QSqliteDriver) to close and re-open the database connection. Closing the database causes (besides perormance implications) in further consequence ... freeing any resources acquired, and invalidating any existing QSqlQuery objects that are used with the database. This will also affect copies of this QSqlDatabase object .... So in fact a call to getApplication() will invalidate the query you are currently <code>do ... while</code>'ing over as it calls QSqlDatabase::open() which might lead to the memory leak your are experiencing.

• Add, initialize and open your QSqlDatabase just once and pass the QSqlDatabase (value) object around. Use the default connection whenever possible as every call to QSqlDatabase::contains() or QSqlDatabase::database() causes a dictionary lookup (and additional thread-safety mechanisms) which just wastes resources if there is just one connection.
• Never directly or indirectly (through QSqlDatabase::open()) close your database as long as there are active queries. Close your database once you are done with all of your operations - there is no harm in keeping the connection alive for the whole application lifetime. SQLite supports concurrent access out of the box if this is your concern.
• Keep your database code as simple and straight as possible. Native drivers tend to leak memory when things get nested and dirty.

[Davita]

Thank you very much Lukas, I appreciate your help.
I didn't know different instance of QSqlDatabase shared same resources internally. Now I call open just once and nothing leaks anymore :)

[fluca1978]

Seems almost correct to me. The only doubt I've got, digging the source code for QSqlDatabase is that:

bq. Sets the connection's database name to \a name. To have effect,
the database name must be set \e{before} the connection is
\l{open()} {opened}. Alternatively, you can close() the
connection, set the database name, and call open() again.

while in your createDatabase() method you set the database name after it has been opened (default argument in database() ).
Things I will try before issuing a bug report is to check what QSqlDatabase::contains(CONN_NAME) returns and what happens if you remove the use of QSqlDatabase::database() from within a query loop (this is something that puzzles me, but I did not find nothing that catch my attention in the source code).

Ops, didn't see the Lukas' post, much more correct than mine!

[lgeyer]

[quote author="Davita" date="1325664572"]Thank you very much Lukas, I appreciate your help.
I didn't know different instance of QSqlDatabase shared same resources internally. Now I call open just once and nothing leaks anymore :)[/quote]

You're welcome. Docnote to QSqlDatabase added.