Speed issue on Qt 6.2.0 vs Qt 6.1.3

hanwil1953

I have a Project which reads about 5000 records from a sqlite database on startup by QSqlQuery. When compiled with Qt6.1.3 this takes about 1-3 secs, when compiled with 6.2.0 it takes about 7-25 secs. There is the same difference on Windows and Android. Has anybody experienced a similar effect between 6.1 and 6.2? Or maybe somebody knows what has to be changed in the project for 6.2.

Christian Ehrlicher

The Qt sqlite plugin has exactly three commits between 6.1.3 and 6.2.0 but I don't see why they should influence the performance in any way:

c2c4266c8c SQLite plugin: use QString::unicode(), not utf16()
47a1e10d63 SQLite driver: fix crash when binding a QByteArray/QString
6720519af7 SQLite: Handle tables and fields with a dot in the name correctly

The QtSql module has no real change between those two versions:

a76017b565 QSqlTableModel::orderByClause(): Quote the table name
14f9f00fdb QSqlQuery: make it a move only type
872f3fffbf QSqlError: protect against self-assignment
563dc357fb Fix misguided conditional, simplify code

I would guess it's somewhere else in your code.

hanwil1953

Thanks for pointing me to the right direction. In fact it wasn't the SQLite reader which is slower in 6.2 but the conversion of 3 fields of Strings to QDateTime (QDatetime datetime = QDatetTime.fromString(dt, "yyyy-MM-dd"). I wrote my own simple conversion routine

QDateTime Movie::StringToDateTime(QString dt)
{
int year = 2000;
int month = 1;
int day = 1;;

year = dt.mid(0,4).toInt();
month = dt.mid(5,2).toInt();
month = dt.mid(8,2).toInt();
QDate *date = new QDate(year, month, day);
QDateTime dtim = QDateTime::currentDateTime();
dtim.setDate(*date);
delete date;
return dtim;
}

and the speed is now the same as in 6.1.3.

JonB

@hanwil1953
If you are saying that calls to QDatetime datetime = QDatetTime.fromString(dt, "yyyy-MM-dd") in 6.2 makes your code run 7x slower than e.g. at 6.1.3 that in itself sounds like it is worthy of addressing....

hanwil1953

Took some time, but I made some detailed measurements. Here the results for reading 5443 record from the database just type conversion, no further processing.

Reading without date conversion: 6.1.3 - 1080ms 6.2.0 - 1000ms
Reading with simple dateconversion: 6..1.3 - 1100ms 6.2.0 - 1300ms (bit slower an 6.2, but neglectable)
Reading with QDateTime.fromString: 6.1.3 - 1120ms 6.2.0 - 3040ms (almost 3x slower)

So in 6.1.3 QDateTime.fromString takes almost no time, in 6.2.0 it is responsible for about 2000ms of the reading time.
One Date conversion with my simple routine takes about 4us in 6.1.3 and 10us in 6.2.0
The conversion with QDateTime.fromString takes about 4us in 6.1.3 and 100 us in 6.2.0

The numers are not very accurate, because the timing varies between measurements somewhat. But one can see that the conversion with QDateTime.fromString is much slower in 6.2.0 than in 6.1.3.
I hope this helps others.

JonB

@hanwil1953
You can see 6.2.0 changelog at https://code.qt.io/cgit/qt/qtreleasenotes.git/about/qt/6.2.0/release-note.md. There are several mentions of QDateTime and parsing, often to do with changing to UTC instead of GMT, but others too.

Your timings are so striking that, if you feel like it, you really ought raise a Qt bug for this and see what the devs have to say....

UPDATE
I think your issue may be being discussed in https://bugreports.qt.io/browse/QTBUG-97489 (though that says "6.2 -> 6.3.0"). This is very much a current issue. I note it says

This resulted from two bugs coming together since QDateTime::fromString starting using QVarLengthArray in https://codereview.qt-project.org/c/qt/qtbase/+/364497

Maybe you will see improvement of your case at 6.3, I don't know.

Robert Loehning

@hanwil1953
Hi, I quickly tried passing the same date 10,000 times through QDateTime::fromString() and didn't notice a significant deceleration between Qt 6.1.3 and Qt 6.2.0.

Are the strings you're passing to QDateTime::fromString() all well-formed date strings or are some of them special in any way? Can you maybe narrow down the problem to some inputs which are extremely slow?

@JonB
Thanks for informing us about this problem. I don't think it is QTBUG-97489, though, since that can't be reproduced in Qt 6.2.0, yet.

Robert Loehning

@hanwil1953
Do you face the same problem if you replace

QDateTime datetime = QDateTime::fromString(dt, "yyyy-MM-dd");

with

QDateTime datetime = QDateTime::fromString(dt, Qt::ISODate);

or even

QDateTime datetime(QDate::fromString(dt, Qt::ISODate), QTime(0,0));