Copy of large files and QFileSystemWatcher under windows

stephane78

Hi,
I have already signaled the QtBug 52226 about renaming of subdirectories with QFileSystemWatcher.
I have another problem when I copy a large file (300 Mo or 1 Go) into a directory watched by the QFileSystemWtacher. The problem is that the signal fileChanged is emitted at the beginning of the copy of the file and I need the signal is emitted at the end of the copy of file i.e. when the copy of the file is finished.
How can we detect by programming (Qt or win32) that the copy of a large file (into a directory) is finished?
is it a Qtbug of QFileSystemWatcher that I must report or should I modify the code of QFileSystemWatcher ?
I had already watched before the code of QFileSystemWatcher because of the first bug and I don't understand very well this code.
Could someone help me a little bit ?

SGaist

Hi,

When copying large files, I usually avoid letting QFile do that alone. I usually do the read/write operation by hand to allow for better feedback and GUI responsiveness.

calebhalvy

@SGaist What do you mean do the read/write operation by hand?

SGaist

Yes, I use QFile for the read and write part.

stephane78

@SGaist It is not copy by programming.
I have an app made with QFileSystemWatcher(a file synchronization app) that monitors and watches a directory and when I run the app and that I copy a large file into the watched directory, the signal directoryChanged of QFileSystemWtacher is emitted at the beginning of the copy of the large file and it should be at the end of the copy of the large file ...that's my problem

jsulm

@stephane78 said in Copy of large files and QFileSystemWatcher under windows:

it should be at the end of the copy of the large file

Why do you think it should? I know, you need it at the end, but maybe someone else needs it at the beginning.

VRonin

The only two things I can think of are:

try KDirNotify instead
Afetr you get the "beginning of copy" signal, store the file size, start a QTimer (500ms?) and check the file size again, if they are equal it probably is finished. I know it's not optimal

stephane78

@VRonin Thank you VRonin, but how can I use KDirNotify because I am under windows (the main version of this app must run under windows although there will be perhaps a mac and linux version too) and KDE runs under linux.
for the solution with the timer I will perhaps try it...thanks

VRonin

The KDE API has been rewritten in version 5 to be platform independent. The KIO module (which contains KDirNoitify) runs on all desktop platforms (i.e. not on android) by default

stephane78

@VRonin thanks I didn't know it. and I should compile the KIO module or library and link it with my app ? I have looked at the url and it seems it uses QtDbus. Can I use QtDbus under windows too? and the signals that are emitted seem to be for files and not for directories ? or is it for directories too ?
it seems although more powerful that QfileSystemWatcher, because there are only two signals emitted by QfileSystemWatcher directoryChanged and fileChanged, that's why, I have added code in a class to process all the cases...

stephane78

@VRonin Hi it is the class KDirWatch that does that (link text (Ihave asked to the developper of KIO)
I will try this class KDirWatch....

stephane78

@VRonin, I have watched the code of KDirWatch but I think it doesn't solve my problem, so I will try to use your solution with the timer...;thanks

stephane78

@VRonin, I have a question before testing, does QFile::size() return the actual size of the large file being copied (so not complete) or does it return 0 ?

VRonin

QFile does not know it's being copied so it will return the actual incomplete file size.

One other thing that comes to mind but I did not test is that the copy probably locks the file for writing so attempting QFile::open(QIODevice::WriteOnly) might return false

stephane78

@VRonin OK my problem is solved thanks to you. I have made it with timers.....

VRonin

Can you share your final code so other users can benefit from the solution?

stephane78

@VRonin not all my code because it is a for a Saas software of my company that will be copyrighted (we use Qt-LGPL)...But I can say something important : the condition to test the end of the copy of the large file (in the slot connected to the timeout ) is the equality of previous size and current size and QFile::open(QIODevice::ReadOnly).In my case I use ReadOnly because I use the file and read it after again to use it and it is ok. and it works.a third condition that I have putted in the test is that the current size is not equal 0 because I believe that QFile::size() return 0 so long the file is not finished copied but I am not sure of this condition but if I remove it, I have a visual c++ runtime error that comes .
so you have the three conditions of the test of the end of the copy of the large file and it is the most important.
the left is specific to my software.
but When I detect a new file added thanks to QFileSYstemWatcher I create a new object with the path of the file and in the constructor of this object I create a timer and start it.so for each file added in the directory there is a new timer that will be deleted later after use of this object linked to the file.....
so there are enough hints now to make the things....

stephane78

@VRonin, I have tested it with a 1,85 Gb file because we are limited to 2Gb files....

VRonin

on the file size =0 I think it's my fault as the docs clearly state:

if the device is closed, the size returned will not reflect the actual size of the device.

so before QFile::open() ,QFile::size() is meaningless

stephane78

@VRonin so my test is not really correct, I have now read that too in the documentation. because my test is :
if (currentsize&&currentsize==oldsize&&f.open(QIODevice::ReadOnly))

//(and currentsize=f.size();)