How to detect memory errors in Qt based gui

JohanSolo

A QString loaded from a 1.5 GB text file needs about 3 GB of memory, as the characters are stored in UTF-16, see the doc. On 32-bits system this almost exceeds the max addressable memory... Depending on how you allocate the QString instance, the consumed memory is far above what is allowed for an application. I therefore think it's normal you have troubles doing this. I would find surprising if you could do this without problems.

On a side note, I think you were told several times not to try to read such a huge file in one go...

VRonin

I'd use QFile instead of fopen()
make sure setText signature is setText(const QString&) and not setText(QString)
can you minimize the memory duplication replacing setText(ts.readAll()); with something like while(!ts.atEnd()) append(ts.readLine());

jsulm

I'm wondering what the use-case is which requires to load 1.5GB text file?
For such amount of data memory mapped file I/O would be better.

JohanSolo

@VRonin said:

make sure setText signature is setText(const QString&) and not setText(QString)

IIRC the implicit sharing of QString should minimise the impact of having a copy (the 3 GB of memory are not changed).

VRonin

@JohanSolo true but I don't know the content of setText, if you use setText(QString) and then use something as easy as append a new line at the end it triggers a huge copy that const QString& (or QString&) would prevent

Qt Enthusiast

@JohanSolo
On a side note, I think you were told several times not to try to read such a huge file in one go...
Hi all
I have already made the change in my code after the discussion on forum as follows

// BUFFER_SIZE is IMB
if (FILE* fp = fopen(ofilename.toLatin1(), "r")) {
QTextStream ts(fp, QIODevice::ReadOnly);
while (!ts.atEnd()) {
QString s = ts.read(BUFFER_SIZE);
append(s);
}
}

My problem of QString reading a file of 1.5 BG in one go is gone
I am asking for such tool that detects memory errors so as to verify even after my change there are no memory errors . This is just to validate the my changes

jsulm

@Qt-Enthusiast What do you mean by "memory errors"?
Do you mean memory leaks? If so then you can use Valgrind as already suggested.

Qt Enthusiast

I am try to run valigrind in my tool and will contact you in case of any errors

Qt Enthusiast

One more things even after this change , for 1.5 GB file if I move the scrollbar in one go to the end , the GUI hangs for some time . Any reason why this is so . I assume I will see the same behavior with setText

jsulm

I think the Qt editor component is not designed for such huge files, that's probably why it is slow. Who wants to write 1.5GB text files and how long will it take?