Unsolved Saving data in a file is too slow
-
@rahuman You could open the file once instead of opening it every time you want to write to it.
-
@jsulm
yes,I tried that also.But only 1 mb is increasing. -
Hi,
What is the data volume ?
What is the speed you expect ?
What is your hard drive speed ? -
@SGaist
Data volume : Using E(drive),Available space :350GB
My required data size is 10GB to be download.
Speed: Required 40 Mbyte/Sec
Hard drive speed:7200 rpm(Toshiba 1 TB (DT01ACA100))Note:While doing Usb data transmission also i faced the same issue.But in python data is saving with the required speed.
-
@rahuman Did you check whether data transfer continues after 1MB?
-
@jsulm
Yes,Continously i can recieve data, but with less speed.
For ex: 1 Min it was reading 10Mb data -
@rahuman And did you check that you are actually sending at 40MB/sec?
-
@jsulm
No, i didn't check of sending speed .As of now i am just reading data. -
@rahuman said in Saving data in a file is too slow:
@jsulm
yes,I tried that also.But only 1 mb is increasing.I do not know what "But only 1 mb is increasing." might mean.
Please make 100% clear whether you are presently calling
QFile::open()
every time you get signalreadyRead()
/callreadAll()
, or not. Continually re-opening a file each time you want to append to it, for "small" chunks, is not optimal. -
1 Mb increasing means previously written like this.
QByteArray RxBuffer = socket->readAll();
qDebug() <<"Reading data......" <<RxBuffer;
QFile file ("C:/123.bin");
if(file.open(QIODevice::WriteOnly | QIODevice::Append | QIODevice::Text))
{
QTextStream out(&file);
out <<RxBuffer;
}And changed like
QByteArray RxBuffer = socket->readAll();
qDebug() <<"Reading data......" <<RxBuffer;
QFile file ("C:/123.bin");
file.open(QIODevice::WriteOnly | QIODevice::Append | QIODevice::Text)
QTextStream out(&file);
out <<RxBuffer; -
@rahuman
This says nothing about 1MB, one way or the other.If the second "changed like" is supposed to address "Continually re-opening a file each time you want to append to it", it does not.
-
Why do you use a QTextStream which converts your QByteArray into a QString and then back to a QByteArray (when it's written to disk). Simply using QFile::write() should be enough here.
-
If you learn nothing else, understand that file IO speeds are not guaranteed. If is dependent upon the OS level IO your system supports, availaiblity of buffer/cache, and physical write speeds of the media. Waht happens when you compare the write speeds of rotational disk to an SSD and then to a ramdisk?...and like others said...don't open/close the file on every transaction