QFile save for usb drive
-
Hi,
I used the following snippnet to save myData to a file. It works well when I save it to my hard disk c:. but when I try to save it to usb drive, it is extremly slow. Is there any way to improve this performance?QFile file( myfile ); if( !file.open( QIODevice::ReadWrite ) ) return false; QDataStream stream( &file ); stream.setVersion( QDataStream::Qt_4_2 ); qDebug()<<" takes long for usb drive"; stream >>myData; file.close();
-
-
Hi,
What USB device is it ?
What generation of USB do you have available ?
What is the size of the data that you are trying to write ? -
The USB device is a normal standard usb2.0 stick drive and I am trying to save 200KB on it. Compare to internal hard drive, it is tool slow.
I also found that QDir::entryList() is very slow for the directory in usb drive. Is there any workaround at least for windows?
QStringList dir_list = qdir.entryList( QDir::NoDotAndDotDot | QDir::NoSymLinks | QDir::Hidden |QDir::AllDirs);
QStringList fn_list = qdir.entryList( QDir::NoDotAndDotDot | QDir::NoSymLinks | QDir::Hidden | QDir::Files); -
Hi
Usb 2.0 is really slow compared to sata III (and II) that is typically used for hard drives.
(hence usb 3 was invented)https://superuser.com/questions/317217/whats-the-maximum-typical-speed-possible-with-a-usb2-0-drive
So have you tried to copy a large file to the usb stick and watch the MB/s ?
If you land between 30-40 MB/s that it not the stick itself.When you say "too slow" , does it takes seconds or how slow ?
Also make sure there is no scanner examining the file in the background.
You can also play around with this
#include <stdio.h> #include <chrono> // for high_resolution_clock void MainWindow::on_pushButton_released() { const int FILE_SIZE = 200; //size in KB const int BUFFER_SIZE = 1024; char buffer [BUFFER_SIZE + 1]; int i; auto start = std::chrono::high_resolution_clock::now(); for(i = 0; i < BUFFER_SIZE; i++) { buffer[i] = (char)(i % 8 + 'a'); } buffer[BUFFER_SIZE] = '\0'; FILE* pFile = fopen ("g:\\somefile.txt", "w"); for (i = 0; i < FILE_SIZE; i++) { fprintf(pFile, buffer); } fclose(pFile); auto finish = std::chrono::high_resolution_clock::now(); std::chrono::duration<double> elapsed = finish - start; qDebug() << "Elapsed time: " << elapsed.count(); }
I get around 0.2 - 0.3 for saving 200KB file on my usb 2 port with usb 2 device.
Note that timing file writes using chrono is imprecise but could give hints. -
I have no problem when I copy file from usb to internal hard disk or internal hard disk to usb stick. Compare to the speed for copying file by windows explorer, copying file by QFile::copy() is not slow. It is slow when I save file to usb stick by QFile/QDataStream. Also when I browse directories, qdir.entryList() shows poor performance compare to explorer or other windows programs. For example if I enter a directory with 10000 files, it takes more than one minute to finish qdir.entryList().
-
@samdol
ok. so we assume device is ok.
Did you try test program and compared elapse times?
0.3 is not slow as such.
Could be interesting to see where you land.What is myData ?
One big array ?update:
triedFor /L %i in (1,1,10000) do fsutil file createnew B%i.tmp 65536
gives me 10.000 files on usb stick.
(unmount / mount before test to avoid cashing)#include <QDir> void MainWindow::on_pushButton_2_released() { auto start = std::chrono::high_resolution_clock::now(); QDir qdir("g:/"); QStringList fn_list = qdir.entryList( QDir::NoDotAndDotDot | QDir::NoSymLinks | QDir::Hidden | QDir::Files); auto finish = std::chrono::high_resolution_clock::now(); std::chrono::duration<double> elapsed = finish - start; qDebug() << "Elapsed time: " << elapsed.count(); }
Elapsed time: 1.54938
So under 2 secs.
This is win 10. Qt 5.7. -
@mrjj
Thank you for your example.
When I run on_pushButton_released() first time, I got
Elapsed time: 16.3639
but when I run it second time and later
Elapsed time: 0.280801
When I run on_pushButton_2_released() for directory with 10000 files on usb, the result is
Elapsed time: 53.6064
quite slow compare to your result. I used windows7 and Qt 5.6.2
Does it mean qdir.entryList() does not like my usb drive
or should I upgrade to Qt 5.7? -
Hi
Could you try with clean / new usb stick ?Those numbers are pretty high for entryList.
Are those files in sub folders?
Its hard to say if it is device or driver issue or something with windows.
or if it simply is that slow in some cases.If time permits simple install 5.7 along side 5.6 and test if it changes anything.
I will also test on a win 7 tomorrow just to be sure.
-
@mrjj
As you suggested, it seems that could be a problem of Qt 5.6.2.
After long wait, finally I could try to compile Qt 5.7.1 in my machine today.
I have downloaded qt-everywhere 5.7.1 and tried to compile in windows 7 platform.
I did
configure -prefix C:\Qt\5.7.1 -static -release -platform win32-g++ -no-compile-examples -opengl desktop
and then
mingw32-make
After several hours(Slow machine), I met the following error message
In file included from release\statemachine.cpp:7:0:
release\statemachine.h:10:30: fatal error: QScxmlStateMachine: No such file or directory
#include <QScxmlStateMachine>
^
compilation terminated.
Makefile.Release:135: recipe for target '.obj/release/statemachine.o' failed.Do I have to add some extra options to configure to solve this problem?
-
Hi,
Since you are upgrading, why not 5.9.1 ?
-
Hi
Can't you just grap it directly then ?
https://download.qt.io/official_releases/qt/5.9/5.9.1/Or something Im missing?
-
@mrjj
I could download enterprise version before 5.9 but since my license expired, I could not gain it. By the way, I could compile Qt 5.6.2 with that configure options but Qt 5.7.1 always spews error when I tried to compile with the same options. I could not understand the method to compile for Qt 5.6.2 did not apply to Qt 5.7.1.