Dealing with QDateTime and January 1 1970 at 00:00:00

wavelike

My application reads CSV files that contain date-time strings. Sometimes, one of the dates happens to be January 1st of 1970. Sometimes, its corresponding time happens to be 00:00:00, which generates invalid QDateTime which, in turn, stops the whole application in its tracks as it cannot complete data import. I use the static function QString::fromString(inputString, formatString), if that is of any importance. It happens on all three of my machines, two Linux and one Windows, all running different versions of Qt.

Is getting invalid QDateTime for that specific moment in time normal & expected? I know it's the beginning of time in Unix, but I wouldn't expect to see invalid QDateTime.... Much, much more importantly, what is the best way to go around this problem?

Thanks.

JonB

@wavelike said in Dealing with QDateTime and January 1 1970 at 00:00:00:

static function QString::fromString(inputString, formatString)

No such function. QDate::fromString(inputString, formatString)? QDateTime::fromString(inputString, formatString)? Which one? It takes a formatString, how do we know what you are passing here? You are asking the question, so please supply whatever you are actually using correctly.

You claim it "generates invalid QDateTime". Are you sure you don't mean: the QDateTime returned is valid, but code then tries to convert that to, say, a time_t, and (because of local time conversion?) that falls before January 1 1970 at 00:00:00 and so is not in range?

Forget your CSV file. Try a literal inputString, as per whatever you would get out of the file. Show us lines you try and the output, what works and what does not.

sierdzio

Sounds like a bug. QDateTime should be able to handle this point in time without problems.

Perhaps the comment from Thiago here will help?

JonB

@sierdzio said in Dealing with QDateTime and January 1 1970 at 00:00:00:

Sounds like a bug.

It sounds more like a "user error" than a Qt bug to me! As per the comment you reference :) As I wrote, we need to see what code the OP is actually using, against what input, with what output.

wavelike

@JonB

Sorry, I should have written it differently, but forgot there is more than one function under the same name and that there is an optional argument (calendar).
QDateTime QDateTime::fromString(const QString &string, const QString &format, QCalendar cal)

Maybe more important piece of information: When I open the CSV file and modify the datetime string by adding just one second to 01-01-1970 00:00:00, it reads the field properly. Previous lines of the same file contain datetime from the '50s and '60s...
As I said, I have no problems with any other combination, at least as far as I noticed i.e. every time there was a problem it was the combination of January 1 1970 and zero time (00:00:00).

As per your question "Are you sure you don't mean: the QDateTime returned is valid, but code then tries to convert that to, say, a time_t, and (because of local time conversion?) that falls before January 1 1970 at 00:00:00 and so is not in range?" I can't say I definitely know what you mean by that, but I suspect that that is not a problem because it is just one instance of time that is problematic. All I do is read a field from the file and use it with the QDateTime::fromString function. It fails with just one combination of date and time, for every provided format.

JonB

@wavelike
Like I wrote:

Forget your CSV file. Try a literal inputString, as per whatever you would get out of the file. Show us lines you try and the output, what works and what does not.

For example, I try this (Python/PySide2, but irrelevant, same will work from C++), and get correct result:

>>> from PySide2.QtCore import QDateTime
>>> print(QDateTime.fromString("01/01/1970 00:00:00", "dd/MM/yyyy hh:mm:ss"))
PySide2.QtCore.QDateTime(1970, 1, 1, 0, 0, 0, 0, 0)
>>> print(QDateTime.fromString("31/12/1969 00:00:00", "dd/MM/yyyy hh:mm:ss"))
PySide2.QtCore.QDateTime(1969, 12, 31, 0, 0, 0, 0, 0)
>>>

Note that I am showing that even a datetime prior to 1/1/1970 still works too.

Show me where you get back "an invalid QDateTime"?

J.Hilk

@JonB said in Dealing with QDateTime and January 1 1970 at 00:00:00:

Like I wrote:

Forget your CSV file. Try a literal inputString, as per whatever you would get out of the file. Show us lines you try and the output, what works and what does not.

For example, I try this (Python/PySide2), and get correct result:

from PySide2.QtCore import QDateTime
QDateTime.fromString("01/01/1970 00:00:00", "dd/MM/yyyy hh:mm:ss")
PySide2.QtCore.QDateTime(1970, 1, 1, 0, 0, 0, 0, 0)

I can confirm

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    QString dtStr("01-01-1970 00:00:00");

    QDateTime dt = QDateTime::fromString(dtStr,"MM-dd-yyyy hh:mm:ss");

    qDebug() << dt;

    return a.exec();
}

results in a valid datetime

QDateTime(1970-01-01 00:00:00.000 CET Qt::LocalTime)

wavelike

@J-Hilk said in Dealing with QDateTime and January 1 1970 at 00:00:00:

@JonB said in Dealing with QDateTime and January 1 1970 at 00:00:00:

Like I wrote:

Forget your CSV file. Try a literal inputString, as per whatever you would get out of the file. Show us lines you try and the output, what works and what does not.

For example, I try this (Python/PySide2), and get correct result:

from PySide2.QtCore import QDateTime
QDateTime.fromString("01/01/1970 00:00:00", "dd/MM/yyyy hh:mm:ss")
PySide2.QtCore.QDateTime(1970, 1, 1, 0, 0, 0, 0, 0)

I can confirm
int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    QString dtStr("01-01-1970 00:00:00");

    QDateTime dt = QDateTime::fromString(dtStr,"MM-dd-yyyy hh:mm:ss");

    qDebug() << dt;

    return a.exec();
}
results in a valid datetime

QDateTime(1970-01-01 00:00:00.000 CET Qt::LocalTime)

Ok, I confirm this too.

I think I know where the problem is. I thought I had that covered, but now I will have to go back to the code and take a look. Anyway, here goes the same code as above, with addition.

 #include <QApplication>
 #include <QString>
 #include <QDateTime>
 #include <QDebug>
   
int main(int argc, char *argv[])
{
   QApplication a(argc,argv);

   QString dtFormat("MM-dd-yy hh:mm:ss");

   QString dt1Str("01-01-70 00:00:00");
   QString dt2Str("01-01-70 00:00:01");

   QDateTime dt1 = QDateTime::fromString(dt1Str,dtFormat);
   QDateTime dt2 = QDateTime::fromString(dt2Str,dtFormat);

   qDebug()<<dt1<<dt2<<"\n";
   qDebug()<<"dt1 is valid? "<<dt1.isValid()<<"\n";
   qDebug()<<"dt1 is equal to QDateTime()? "<<(dt1 == QDateTime())<<"\n";
   qDebug()<<"dt2 is valid? "<<dt2.isValid()<<"\n";
   qDebug()<<"dt2 is equal to QDateTime()? "<<(dt2 == QDateTime())<<"\n";

   return a.exec();
}

The output is true, true, true, false. Now I have to dive into the code and see where this is checked and how exactly it's done, but hopefully that is the culprit.

Thanks everyone.

P.S. From documentation:

QDateTime::QDateTime()
Constructs a null datetime (i.e. null date and null time). A null datetime is invalid, since the date is invalid.

So, dt1 above is a null datetime, because it is equal to QDateTime(). But dt1.isValid() returns true. I am very, very tired these days, so this is probably obvious, but... uh?

KroMignon

@wavelike said in Dealing with QDateTime and January 1 1970 at 00:00:00:

So, dt1 above is a null datetime, because it is equal to QDateTime(). But dt1.isValid() returns true. I am very, very tired these days, so this is probably obvious, but... uh?

Hmm, yes this very confusing. But even if QDateTime() is an invalid instance, using QDateTime() to check if a QDateTime instance is invalid is a bad usage!
To check a QDateTime instance validity, you have to use QDateTime::isValid().
I suppose that operator==() compare the unixtimestamp, and per default this value is probably '0' and your dt1 is unix time 0!

JonB

@wavelike
I agree with your finding:

>>> QDateTime.fromString("01/01/70 00:00:00", "dd/MM/yy hh:mm:ss")
PySide2.QtCore.QDateTime(1970, 1, 1, 0, 0, 0, 0, 0)
>>> QDateTime.fromString("01/01/70 00:00:01", "dd/MM/yy hh:mm:ss")
PySide2.QtCore.QDateTime(1970, 1, 1, 0, 0, 1, 0, 0)
>>> QDateTime()
PySide2.QtCore.QDateTime(0, 0, 0, -1, -1, -1, -1, 0)
>>> 
>>> QDateTime.fromString("01/01/70 00:00:00", "dd/MM/yy hh:mm:ss") == QDateTime()
True
>>> QDateTime.fromString("01/01/70 00:00:01", "dd/MM/yy hh:mm:ss") == QDateTime()
False

I agree this is strange! Although isValid() returns True for both, it seems comparing (01/01/70 00:00:00, only this) against QDateTime() for invalidity is not a good idea! You must stick with isValid().

Only a Qt expert (not me) examining the above can explain behaviour.... ?

EDIT @KroMignon's reply crossed with mine. Somewhere his suspicion that

I suppose that operator==() compare the unixtimestamp, and per default this value is probably '0' and your dt1 is unix time 0!

Which exact Qt version are you using?

https://doc.qt.io/qt-5/qdatetime.html#operator-eq-eq

bool QDateTime::operator==(const QDateTime &other) const

Returns true if this datetime is equal to the other datetime; otherwise returns false.

Since 5.14, all invalid datetimes are equal to one another and differ from all other datetimes.

will doubtless indeed be the cause!

I think what you have found is a "bug", given the above! Your valid 1970 datetime does not differ from an invalid datetime....

I suspect Qt has always behaved like this, but I think this is indeed a nasty "gotcha" and worthy of being reported, I can see how it could bite others on the butt unexpectedly....

wavelike

@KroMignon , @JonB
I agree that using isValid() seems like a much better programming practice. However, documentation is not always perfectly clear and can lead to wrong conclusions if not read thoroughly (which we usually don't do when facing deadlines...).

On Linux, I am using Qt version 5.13.0, while on Windows it's 5.9.2.

wavelike

@JonB said in Dealing with QDateTime and January 1 1970 at 00:00:00:

...

https://doc.qt.io/qt-5/qdatetime.html#operator-eq-eq

bool QDateTime::operator==(const QDateTime &other) const

Returns true if this datetime is equal to the other datetime; otherwise returns false.

Since 5.14, all invalid datetimes are equal to one another and differ from all other datetimes.

will doubtless indeed be the cause!

I think what you have found is a "bug", given the above! Your valid 1970 datetime does not differ from an invalid datetime....

I suspect Qt has always behaved like this, but I think this is indeed a nasty "gotcha" and worthy of being reported, I can see how it could bite others on the butt unexpectedly....

If you, or anyone else, have time and know where to report these kinds of things, please do it. If it is worth reporting, of course. I really don't have time, unfortunately.

Again, thank you for your help. :)

JonB

@wavelike
We are saying that you must change over your code to only use .isValid() for this check, not == QDateTime(). You have no choice as Qt stands.

I am saying to you that this may be worthy of reporting to Qt bug site, because I feel it is clearly "wrong", in terms of documentation/expected behaviour. Whether they will do anything about it if Qt has always behaved like this on that date I cannot say....

EDIT FWIW, I have raised https://bugreports.qt.io/browse/QTBUG-82853

EdwardWelbourne

Up to 5.14, comparison between date-times was tacitly only defined for valid date-times; you needed to check .isValid() before comparing. In fact, it was quite possible to construct diverse invalid date-times, each of which would compare equal to some valid date-time. At 5.14, this is fixed so that all invalid date-times compare less than all valid date-times (and, in particular, equal to none of them).
So either check isValid() first, or upgrade to 5.14 (or later).

EdwardWelbourne

... oh, and: yes, to determine validity, always use isValid() rather than comparison to an invalid value (not just for QDateTime): isValid() is authoritative, comparison might fail for types that may have more than one invalid value.

JonB

@EdwardWelbourne said in Dealing with QDateTime and January 1 1970 at 00:00:00:

Up to 5.14,

I only have 5.12.2, and I think the OP says he has 5.13. In those two

QDateTime.fromString("01/01/1970 00:00:00", "dd/MM/yyyy hh:mm:ss") == QDateTime()  => true

Are you saying this now returns false at 5.14? Thanks.

EDIT On the QTBUG @EdwardWelbourne has stated it is fixed at 5.14.