Byteordermark is not set in *.csv file

hkottmann

Im currently programming with Qt 6.31 an I've recognized that the ByteOrderMark is not set even if I call setGenerateByteOrderMark(true). Here my code:

QTextStream out(&file);
out.setGenerateByteOrderMark(true);

In past version of Qt this was always set correctly

Christian Ehrlicher

Works fine for me with Qt 6.6:

int main(int argc, char* argv[])
{
    QCoreApplication app(argc, argv);

    QFile f("D:\\test.txt");
    f.open(QIODevice::WriteOnly);
    QTextStream ts(&f);
    ts.setGenerateByteOrderMark(true);
    ts << QLatin1String("blub");
    f.close();
}

hkottmann

@Christian-Ehrlicher So it's obviously a bug in Qt 6.31

hkottmann

I've tried Qt 6.42. Same thing here

Christian Ehrlicher

@hkottmann I doubt there is a problem in 6.3/6.4 but fixed in 6.6 without any hint in the bug reporting system. Please try my example and properly look in the output if there is really no bom. How do you check for the bom existance?

hkottmann

Rapidcsv was the evildoer, it removes the BOM. But it's the most efficient library to add the end date of the header of csv file which contains the the data recording of a huge amount of data. So here my solution which I put in a QFuture:

qint64 SmartDCPD::processDataFile(const QString &fileName)
{
    QElapsedTimer t;
    t.start();
    QString tempfile = fileName + ".tmp";
    rapidcsv::Document doc(fileName.toUtf8().constData(), rapidcsv::LabelParams(-1, -1), rapidcsv::SeparatorParams(';'));
    QString stamp = QDateTime::currentDateTime().toString("dd.MM.yyyy hh:mm:ss.zzz");
    doc.SetCell<std::string>(1, 2, stamp.toUtf8().constData());
    doc.Save();
    QFile::rename(fileName, tempfile);
    QFile file(tempfile);
    file.open(QIODevice::ReadOnly);
    QTextStream str(&file);
    QFile f(fileName);
    f.open(QIODevice::WriteOnly);
    QTextStream ts(&f);
    ts.setGenerateByteOrderMark(true);
    ts << str.readAll();
    f.close();
    file.close();
    QFile::remove(tempfile);
    return t.nsecsElapsed();
}

JonB

@hkottmann said in Byteordermark is not set in *.csv file:

Rapidcsv was the evildoer

But how did this differ between Qt versions, 5 vs 6?