Necromancy, sorry, but there was a point made a while back that kind of zipped by without comment, and I wanted to circle back to it:
@Christian-Ehrlicher said in Rust file parsing significantly faster than Qt/C++ file parsing. Solutions for Qt implementation wanted. File size: 68.5 MB:
The problem with QDateTime is that for every call the internal format parser (QDateTimeParser, private class) is re-created and needs to re-evaluate the string. This takes a lot of time.
Given how expensive it is to re-create that parser for every call, I wonder if Qt would consider making the parser public. instantiable, and reusable, so that when parsing a lot of same-formatted date strings, it could be created just once and applied to all of them? Kind of like Python's re.compile() for regular expressions.
e.g. something like:
QDateTimeStringParser* dateParser = new QDateTimeStringParser(
"yyyyMMdd HHmmss zzz0000"
);
QElapsedTimer* parseTimer1 = new QElapsedTimer();
parseTimer1->start();
for (int ii = 0; ii < allData.size() - 1; ii++)
{
QByteArrayList data = allData.at(ii).split(';');
t.dt = dateParser->parse(data.at(0));
t.last = data.at(1).toDouble();
t.bid = data.at(2).toDouble();
t.ask = data.at(3).toDouble();
t.volume = data.at(4).toInt();
instr.tickList.append(t);
}
qDebug().noquote() << QString("Qt parse time: %1ms")
.arg(parseTimer1->elapsed());
dateParser->deleteLater();
I wonder if that could make it significantly faster? parse() would have to be reentrant and stateless, of course (beyond the stored, immutable format string), which could still be tricky for a complex parser.
