QDateTime::fromString() returns invalid Datetime

Wuzi

Hello,

I'm using QDateTime::fromString() and I've seen that in some cases the function returns an invalid DateTime for an unknown reason. Is this a bug? I'm using Qt5.15.2 under linux
To evaluate the content of the variable, I used the debugger directly

#include <QDateTime>

int main(int argc, char *argv[])
{
	auto dt0 = QDateTime::fromString("2016-03-26T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz"); // valid
	auto dt1 = QDateTime::fromString("2017-03-26T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz"); // invalid
	auto dt2 = QDateTime::fromString("2018-03-26T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz"); // valid
	auto dt3 = QDateTime::fromString("2019-03-27T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz"); // valid

	return 0;
}

Christian Ehrlicher

Because 2017-03-26T02:14:34.000 simply does not exists in europe.

Wuzi

@Christian-Ehrlicher Thank you for your response. But if this date is UTC it is valid or not?

Christian Ehrlicher

@Wuzi said in QDateTime::fromString() returns invalid Datetime:

But if this date is UTC it is valid or not?

But where do you pass this information? QDateTime::fromString() is using the current locale. Use QLocale::toDateTime() instead

Wuzi

It was just in general. How can I use toDateTime() to parse UTC time?
I tried, but with no success:

int main(int argc, char *argv[])
{
	QLocale l = QLocale(QLocale::Language::AnyLanguage, QLocale::Territory::World);
	auto dt0 = l.toDateTime("2016-03-26T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz");
	auto dt1 = l.toDateTime("2017-03-26T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz");
	auto dt2 = l.toDateTime("2018-03-26T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz");
	auto dt3 = l.toDateTime("2019-03-27T02:14:34.000", "yyyy-MM-ddThh:mm:ss.zzz");

	return 0;
}

One possible solution which worked, was adding UTC to the string and the "t" format.

#include <QDateTime>

int main(int argc, char *argv[])
{
	auto dt0 = QDateTime::fromString("2016-03-26T02:14:34.000 UTC", "yyyy-MM-ddThh:mm:ss.zzz t");
	auto dt1 = QDateTime::fromString("2017-03-26T02:14:34.000 UTC", "yyyy-MM-ddThh:mm:ss.zzz t");
	auto dt2 = QDateTime::fromString("2018-03-26T02:14:34.000 UTC", "yyyy-MM-ddThh:mm:ss.zzz t");
	auto dt3 = QDateTime::fromString("2019-03-27T02:14:34.000 UTC", "yyyy-MM-ddThh:mm:ss.zzz t");

	return 0;
}

ChrisW67

Or let Qt parse the date as local and the tell the object to consider it UTC:

qDebug() << QTimeZone::systemTimeZoneId();

QDateTime dt = QDateTime::fromString("2016-03-26T02:14:34.000", Qt::ISODateWithMs);
qDebug() << dt << dt.isValid();  // << Local time
dt.setTimeSpec(Qt::UTC);
qDebug() << dt << dt.isValid(); // << UTC, note time is not adjusted

dt = QDateTime::fromString("2017-03-26T02:14:34.000", Qt::ISODateWithMs);
qDebug() << dt << dt.isValid();
dt.setTimeSpec(Qt::UTC);
qDebug() << dt << dt.isValid();

Output for my local time (UTC+10 no DST at all)

"Australia/Brisbane"
QDateTime(2016-03-26 02:14:34.000 AEST Qt::LocalTime) true
QDateTime(2016-03-26 02:14:34.000 UTC Qt::UTC) true
QDateTime(2017-03-26 02:14:34.000 AEST Qt::LocalTime) true
QDateTime(2017-03-26 02:14:34.000 UTC Qt::UTC) true

and in Europe (UTC+1 and DST)

"Europe/Paris"
QDateTime(2016-03-26 02:14:34.000 CET Qt::LocalTime) true
QDateTime(2016-03-26 02:14:34.000 UTC Qt::UTC) true
QDateTime(Invalid) false
QDateTime(2017-03-26 02:14:34.000 UTC Qt::UTC) true

kshegunov

@ChrisW67 said in QDateTime::fromString() returns invalid Datetime:

Or let Qt parse the date as local and the tell the object to consider it UTC:

Well, no. The date is still invalid - it doesn't (always) exist in "local time" no matter whether you force it after that.
The only correct solution is to provide the timezone information in the string, which is why ISO dates also allow it (and UTC is so prevalent that it has the shortest of formats - notice the traling 'Z'):

QDateTime date = QDateTime::fromString("2017-03-26T02:14:34.000Z", Qt::ISODateWithMs);

ChrisW67

@kshegunov said in QDateTime::fromString() returns invalid Datetime:

Well, no. The date is still invalid - it doesn't (always) exist in "local time" no matter whether you force it after that.

You are correct that some strings representing a date/time are always invalid in some time zones, and not in others. My observation relates to the way QDateTime behaves, not the way time zones behave.

If you look at my example for Paris you will see the 2017 date is initially parsed, loaded and treated as Invalid (correctly). However, telling the QDateTime object containing the invalid time for Paris that it actually holds a UTC date/time generates the expected, valid result. The date/time components have been retained and the Invalid state is determined in light of the TimeSpec in use (LocalTime vs UTC). As the documentation for QDateTime::setTimeSpec() changing this results in a different point in time (in this case a valid one).

Note, that this is distinctly not the same as converting the same point in time time between time zones. Attempting to determine the corresponding UTC time for an Invalid Paris time, in this case by adjusting by an undefined 1 or 2 hours, should also be expected to be Invalid or exhibit undefined behaviour. Interestingly, Qt loaded the original date with an offsetFromUtc() == 0 (there being no valid offset corresponding to the date string in Paris) and behaves as such when asked for toUTC().

I work in aviation where everything is UTC/Zulu time by default, and local time by exception, and I agree with you that unambiguous date/time representations are important. Appending "Z" or "+00:00" to incoming unqualified date strings is an option, but having a source that issues unambiguous data from the start is definitely preferable.

JonB

@ChrisW67 said in QDateTime::fromString() returns invalid Datetime:

I work in aviation where everything is UTC/Zulu time by default

Hallelujah! I too have worked where multiple timezones are involved, and it's a pain. Why, oh why, can't the whole world agree to just use UTC everywhere, no time zones, it would make programs (and appointments) so much easier....?

Wuzi

Ok thank you. So I will add the trailing Z to the string so that it is correct. Thank you all!

ChrisW67

@JonB said in QDateTime::fromString() returns invalid Datetime:

Hallelujah! I too have worked where multiple timezones are involved, and it's a pain.

It is not all roses though :( Times for passenger schedules are local, flight plans are all Zulu. In aviation runway lengths and widths are in metres, altitudes in feet, sometimes metres, referenced to the actual barometric pressure at the aerodrome or in the area (in hectopascals or inches of Mercury), or a flight level (hundreds of feet) relative to a standard pressure depending on the region and height at which you are flying. Airspeed is in knots or a Mach number, distances in nautical miles. Bearings are in degrees, typically magnetic, but North is sometimes 0 sometimes 360. Aircraft weight is given in pounds or kilogrammes/tonnes, engine thrust in pound force or newtons, fuel in pound, litres, kg/tonnes and sometime even gallons (US or imperial). Terrain elevation might be given in feet or metres, but safety clearances (above or laterally) are in metres for terminal procedure design and feet in general elsewhere. Temperatures are in Celsius mostly, Fahrenheit sometimes. Weather reports at aerodromes give horizontal visibility in kilometres or statute miles. Latitudes/longitudes are sometimes presented in decimal, sometimes sexagesimal with variable precision/spacing/formatting. Aircraft structures are variously metric or imperial.

JonB

@ChrisW67
OMG!!

But my desire is that there is no "local time" anywhere, just everyone uses UTC always. Miles simpler....

The crazy things about all those different units you mention is that --- once they have to proper code to measure/convert --- computers really don't have any problems dealing with them all effortlessly. Which is why pilots' days --- like many other professions --- are numbered... :)

kshegunov

@ChrisW67 said in QDateTime::fromString() returns invalid Datetime:

You are correct that some strings representing a date/time are always invalid in some time zones, and not in others. My observation relates to the way QDateTime behaves, not the way time zones behave.

It's worse than that. Some points are ambiguous in local time with DST in effect, and some plainly don't exist at all. Not to mention you're suggesting a hidden use case for QDateTime, which may or may not work due to an implementation detail. Sorry, it simply is an incorrect solution.

Christian M.

Hi @ChrisW67 , @Wuzi @kshegunov , I had this problem yesterday and finally found what it seems to be an easy solution. For example, in Melbourne Australia the Daylight Saving Time starts at "02/10/2022 02:00:00" so clocks are turned forward 1 hour to "02/10/2022 03:00:00". In other words, if my system time zone is ‘Australia/Melbourne’, the following line will return an invalid QDateTime because it doesn't exist:

QDateTime ts = QDateTime::fromString("02/10/2022 02:00:00", "dd/MM/yyyy hh:mm:ss"); //invalid

To get a valid QDateTime for example in UTC, you can instead use the following approach:

QCalendar cal(QCalendar::System::Julian);
QDateTime ts = QDateTime::fromString("02/10/2022 02:00:00", "dd/MM/yyyy hh:mm:ss", cal); //valid
ts.setTimeSpec(Qt::UTC); // Alternatively you can use: ts.setTimeZone(QTimeZone(QByteArray("UTC")));
qDebug()<<ts.toString("dd/MM/yyyy hh:mm:ss");

JonB

@Christian-M
Hello and welcome.

I do not understand. The whole point is that is that from 02:00 to 02:59 on that day there IS no "valid time". It does not exist. In local time, which is what you are asking about. At best it is ambiguous: does 02:30 represent 30 minutes after the clocks changed from 02:00 or 30 minutes before they changed to 03:00? So I don't know what you mean by "To get a valid QDateTime for example in UTC" when the local time is simply not valid. If you want to treat 02:30 as a UTC time that is fine, but is quite a different time from local time.

Christian M.

Hi @JonB , I found this problem when I was trying to read timestamps from an csv file written by a device using standard time instead of DST (i.e., a time zone different from my laptop's local time ). When I tried to parser that specific timestamp on my laptop using QDateTime::fromString I was getting an invalid timestamp because QDateTime::fromString assumes that the input string is in the same time zone of my laptop. As I mentioned before "02/10/2022 02:00:00" doesn't exist in Australia/Melbourne time zone because of DST's ambiguity. However, the timestamp makes sense in because it was recorded on a different time zone, not in my local time zone.**
**Anyway, another solution in my case could be to force my users to switch the time zone of their machines every time they need to use my app. That’s not nice!!

I think there should be another version of QDateTime::fromString with a third argument that allows developers to specify the time zone of the string they want to parser. Ideally it would be nice to have something like this although it doesn’t exist yet:

static QDateTime QDateTime::fromString(const QString &string, const QString &format, const QTimeZone &timeZone); // this method doesn't exist yet

JonB

@Christian-M
OK, so the only correct way to do this is: convert from foreign TZ (where 02:00--03:00 is valid) to UTC, convert from UTC to your local TZ (no ambiguity, a given UTC only maps to one local TZ time). It never was correct to even try to interpret that foreign TZ time as a local time. You might as well interpret that foreign time as though it were UTC, since you don't know anyway, then at least it won't cause an error.

QDateTime QDateTime::fromString(const QString &string, const QString &format, const QTimeZone &timeZone);

Indeed! Can one not achieve this by either adding some TZ information to the string or is void QDateTime::setTimeZone(const QTimeZone &toZone) just what this is for? I might have a play later if I have some time....

P.S.
I see that @Christian-Ehrlicher said earlier

But where do you pass this information? QDateTime::fromString() is using the current locale. Use QLocale::toDateTime() instead

Maybe that will allow conversion from a foreign local time?

Christian M.

Hi @JonB . You don’t necessarily need to convert from the foreign time zone (where 02:00--03:00 is valid) to UTC and then to your local time zone. In fact, you can directly convert from the foreign time zone to your local time zone without any intermediate conversion to UTC. Let’s say that the foreign time zone is UTC+3 and my local time zone is ‘Melbourne/Australia’ (with DST). You don’t need to convert from UTC+3 to UTC and then to ‘Melbourne/Australia’. You can just convert from UTC+3 to ‘Melbourne/Australia’ as follows:

QCalendar cal(QCalendar::System::Julian);
QDateTime foreingTz = QDateTime::fromString("02/10/2022 02:00:00", "dd/MM/yyyy hh:mm:ss", cal); //valid
foreingTz.setTimeZone(QTimeZone(QByteArray("UTC+3")));
QDateTime localTz = foreingTz.toLocalTime();

qDebug()<<foreingTz; // 2022-10-15 02:00:00.000 UTC+03:00
qDebug()<<localTz; // 2022-10-15 10:00:00.000 AUS Eastern Daylight Time

As you can see, it is not strictly necessary to convert from the foreign time zone to UTC and then to your local time zone. However, if you are working with databases, I think it would be a good practice store to convert all your timestamps to UTC before storing them in the database. In this way, when your remote clients access the database from different time zones, the timestamps could be converted to their specific time zones on the fly. For example, just before showing the values on the screen. This approach seems to work for me when I must deal with timestamps from several time zones.

JonB

@Christian-M
Firstly, your post has literally crossed with my adding a P.S. to my reply above.

foreingTz.setTimeZone(QTimeZone(QByteArray("UTC+3")));

Are you sure this approach is correct (I don't think so)? Ignoring for the moment that you may know the foreign timezone name but not its UTC offset. Let's say this foreign TZ has its own daylight savings rules, and let's say that this foreign local time does (or does not, you don't know) lie in its DST. Then, correct me if I am wrong, I cannot see how from UTC+3 datetime code could possibly know whether DST is or is not in effect on the stated time. In effect, you need to know whether to pass, say, UTC+3 vs UTC+4 for that foreign local time at its different times of year, and you do not know that. In which case, it cannot convert correctly.....???

[P.S. I assume your input string does not itself come with any UTC+... suffix on it. Obviously if it does then the whole thing is trivial, since one can get to UTC from that and then UTC to your local. I assume you mean the time string has no UTC information on it, just that you know it comes from e.g. Europe in local time and are wanting to convert to Melbourne local time.]

Christian M.

@JonB It works, you just need to specify the IANA id of your foreign time zone as follows:

foreingTz.setTimeZone(QTimeZone(QByteArray("HERE YOU PUT YOUR IANA ID")));

The only requirement so far is that your timestamp strings must have been taken from a time zone where they actually exist. You can consult your IANA time zone IDs using the method:

static QList<QByteArray> QTimeZone::availableTimeZoneIds()

By the way, the offset of UTC+3 is always the same. it doesn't have DST effect as it is standard time. DST is just a correction added by some countries in their local areas so they can take some commercial advantage from that. UTC+X is standard and doesn't have DST. Please have a look at the documentation here: https://doc.qt.io/qt-6/qtimezone.html#details
and specifically here: https://doc.qt.io/qt-6/qtimezone.html#time-zone-offsets :

... The total offset is comprised of two component parts, the standard time offset and the daylight-saving time offset. The standard time offset is the number of seconds to add to UTC to obtain standard time in the time zone. The daylight-saving time offset is the number of seconds to add to the standard time offset to obtain daylight-saving time ...

I hope that can help to clarify your confusion.

Solved QDateTime::fromString() returns invalid Datetime