Insertings rows into QSQlite database eats up all RAM Memory

[guru007]

I am using QSQlite database for analyzer application. Goal, is to parse traffic data of some protocol (say PCIe) and show it to User details of each packet. For Dataset of 400 MB, row count reach 60 million.
Currently, there are 8 columns and each column is of type TEXT.
When Application starts insertions into database, RAM Memory usage increases significantly.
Using below Insert function:

void database::insertData(const QString &a_offset,const QString &a_byte3,const QString &a_byte2,
                          const QString &a_byte1,const QString &a_byte0,const QString &a_dataType,
                          const QString &a_status,const QString &a_location)
{
static QSqlQuery insertQuery;
    bool ok = false;

  QueryString = "insert into arincData values('"+a_offset+"'," \
                                               "'"+a_byte3+"'," \
                                               "'"+a_byte2+"'," \
                                               "'"+a_byte1+"'," \
                                               "'"+a_byte0+"'," \
                                               "'"+a_dataType+"'," \
                                               "'"+a_status+"'," \
                                               "'"+a_location+"')";

    ok = insertQuery.exec(QueryString);

}

[JonB]

@guru007
A (Q)SQLite operation will consume a certain, unspecified amount of memory. Your description states "increases significantly" while your title states "eats up all RAM Memory", these are not the same thing.

Because your QSqlQuery is static it will hang onto whatever resources. Start by removing the static. Even then it is likely/possible that some memory will be consumed which may or may not be returned to the OS for re-use, so how are you measuring memory usage and when?

After that test, if you insist on maintaining a static then at least call QSqlSquery::clear(), though I don't know how much difference that will make.

[Christian Ehrlicher]

Clear() is not needed and static is plain wrong. Also use a prepared query and bind the values instead creating a sql out of your strings to avoid sql injection problems.

[JonB]

@Christian-Ehrlicher
Agreed static is wrong (apart from anything else it will hold onto the QSqlDatabase, e.g. at close down). I only suggested clear() if user insists because of

Clears the result set and releases any resources held by the query. Sets the query state to inactive. You should rarely if ever need to call this function.

[Christian Ehrlicher]

@JonB The QSqlQuery is on the stack when it's not static so... And the clear also happens on each new exec/prepare.