Reading large ASCII file with QFile
-
@HB76
HiSo the file reader is running in thread?
Try to disable the emit loadingValueChanged for a test and see if it still crashes.
You might be spamming the event loop which can crash.
else i see no reason to crash even if file was 1 TB as we just read it line by line. (and not store each line)
-
-
@HB76 said in Reading large ASCII file with QFile:
Is it possible that I run out of RAM ?
How large is one line ?
Also how do you split it to values ?Yes, must be something like that then.
is it a 32 bit or 64 bit app ?
also if you run with debugger on, what line makes it crash ? -
The longest line can have more than 30'000 characters.
Each line contain a keyword at the beginning which is separated from data thanks to the "/" char.
After that, data are separated with coma.
So I use the split method to get a QStringList of data that I convert in values by using ToInt() method of QString class.My compiler version in 32 bits. Do you think it can have an impact of the reading ?
-
Hi,
Since you have a 32bit application you can't use more than 4GB of RAM. Is that the case with your application ?
-
@HB76 said in Reading large ASCII file with QFile:
My code works fine with smaller file (<500 Mb) but when I try to read large files, the program crash.
Please provide a backtrace of the crash.
-
The exception returned is std::bad_alloc.
I should try with 64 bits version, but I need to add the 64 bits MinGW compiler since I don't have it configured in Qt. -
@HB76
Just install the Qt 64 bit MinGW Qt binary from the maintenance tool and you are good to go.Also, have a look at
https://doc.qt.io/qt-5/qstringref.html
which might provide a huge boost since it will not copy the strings and all you want is to call toInt on it. -
@HB76 said in Reading large ASCII file with QFile:
The exception returned is std::bad_alloc.
This doesn't tell me where the memory was not enough. But you have your answer anyway, you don't have enough memory.
