QDataStream bad_alloc corrupt file

maxp

Hi, when opening a file with QFile (QIODevice::ReadOnly) and using QDataStream to read in the contents of a file that I wrote with QFile and QDataStream everything works as it should. But when I delete a part of the content and try to read it I get a std::bad_alloc. How should I use QDataStream to be robust against corrupt files? I cant find an example that does this. Also using status QDataStream::ReadPastEnd and QDataStream::ReadCorruptData don't appear to help me. The status is always QDataStream::Ok.

So my question is, can someone show me an example of how to use QDataStream to be robust against reading files containing adapted data.

struct Data
{
    uint32_t port;
    QHostAddress ip;
    QString category;
};
QDataStream& operator<<(QDataStream& stream, Data const& data)
{
    stream << data.port << data.ip << data.category;
    return stream;
}
QDataStream& operator>>(QDataStream& stream, Data& data) {
    stream >> data.port;
    stream >> data.ip;
    stream >> data.category;
    return stream;
}

QList<Data> data;

QFile file("data.txt");
if (!file.open(QIODevice::ReadOnly)) { return; }
QDataStream stream(&file);
stream.setVersion(QDataStream::Qt_5_12);
stream >> data; // std::bad_alloc happens here
file.close();

Christian Ehrlicher

And how do you write it out to data.txt (why txt - it's a binary file, no text)

maxp

Ah yes, actually it was "data.ini", but I have made a small example here and the fact that it has extension txt does not really change the working right?

Christian Ehrlicher

@maxp said in QDataStream bad_alloc corrupt file:

it has extension txt does not really change the working right?

Correct, but as I already said the part on how you save the file is missing.

maxp

Sorry, I was trying to immediately submit the second text, but I have to wait 600 seconds apparently :(.
I write like this to the file, but reading that is no problem. It is only a problem when I open it for example with notepad and delete a part such that the content is corrupt. I want to be able to detect that instead of having the application throw a bad alloc exception or should I just try catch this and is that robust enough?

// Write to devices file
QFile file("data.ini");
if (!file.open(QIODevice::WriteOnly)) return;

// Small data example
Data data{.port=3300, .ip=QHostAddress(), .category="text"};
QList<DeviceData> data_list;
data_list.append(data);

QDataStream stream(&file);
stream.setVersion(QDataStream::Qt_5_12);
stream << data_list;
file.close();

Christian Ehrlicher

You must not edit a file generated with QDataStream with a text editor. Either add a try/catch block or add a checksum to your file which can be used to check the integrity

KroMignon

@maxp said in QDataStream bad_alloc corrupt file:

It is only a problem when I open it for example with notepad and delete a part such that the content is corrupt

Yes of course this is a problem!
Qt is open source, so you could look yourself in QDataStream to find out why this is not possible.
When you are serializing an QList, then QDataStream firs writes the QList size, and then each item in the list.

So hacking the output with a text editor will generate a corrupted/unreadable file.

What did you expect?

maxp

Yes indeed, but I just want to make my program not crash in case I have a corrupt file, or a user manipulated it.

maxp

So at least the try catch / checksum should be added as Christian said.

I was wondering why the status was not QDataStream::ReadCorruptData in this case and when it is the case. But I can check it further in the open source code than.

KroMignon

@maxp said in QDataStream bad_alloc corrupt file:

Yes indeed, but I just want to make my program not crash in case I have a corrupt file, or a user manipulated it.

Then you have to deal with QDataStream::startTransaction() and QDataStream::commitTransaction().
For example:

QList<DeviceData> data_list;
QDataStream stream(&file);
stream.setVersion(QDataStream::Qt_5_12);
stream.startTransaction();
stream >> data_list;
if(!stream.commitTransaction())
{
   qDebug() << "ERROR!!!";
}

maxp

Thanks for the information, I'm going to look further into the transaction.

Christian Ehrlicher

I would guess due to the modification with the text editor the length indicator of the QList is a very large number which results in a bad_alloc. Therefore the stream can not return with ReadCorruptData

JonB

@maxp
As the others have said for your immediate issue.

At a conceptual level: I don't know if this is just a "one-off" where you are somehow wanting to recover a corrupt file. But if you have any intention of wanting to be able to edit these files, for recovery or other purposes, think about changing over to a non-binary, text-friendly format like JSON instead of QDataStream.

maxp

Well, recovering the file is not necessary and it does not have to be editable from outside the program.
My initial plan was to read the file, not crash and show an error message/log if there is some kind of problem and just carry on with the program.
Then it became: try to read the file, catch the bad_alloc to not crash and warn the user to check the file or delete it. (whatever i change in the file seems to lead to the bad_alloc)
I shall have to dig deeper in this problem and how the QDataStream/transaction/etc works :).

JonB

@maxp said in QDataStream bad_alloc corrupt file:

bad_alloc

Read, say, https://stackoverflow.com/questions/9456728/how-to-deal-with-bad-alloc-in-c for a typical discussion. You can catch it, it's potentially fraught, safest is to exit afterward. Up to you.

It's just as possible that a different piece of "bad" data will cause some quite different error.

But in general expecting a program to deal satisfactorily with a binary data stream which is simply "incorrect" and not "hang/crash" is pretty optimistic. You're not going to be able to cover all cases as you would like.

maxp

Ok thanks, so exiting after the caught bad_alloc is already good. For this program that is enough. But when I would want to read a file in a program which should not just quit after reading a corrupt one I should use something else than.