when is a QFile::copy() finished ?
-
Hi all,
does anyone knows if the QFile::copy() returns at the beginning or at the end of the copy process ?
I'm in trouble because my app was copying a large-file on a Windows shared folder using QFile::copy(origin, dest) and then someone (well, it was me... ) has deleted the source file.
I may be wrong but the copy() function did not return any error and the copy process has been silently aborted.
I wonder what's the best way to check whether a copy (local or on a shared resource) is finished or not.
thanks a lot Giampaolo
-
@iw2lsi
I think you can assume it returns at the end of the copy. Of course, underlying OS copy/write might continue beyond the OS call(s), e.g. flushing buffers.I would not assume that e.g. deleting the destination of a copy (or even the source) necessarily has any effect on the operation, would cause an error or any kind of a message.
You should assume the copy has completed as best possible upon return from
copy()
. If you really want you could call something likesync()
at the end to further encourage the OS to complete operations, but I wouldn't suggest it. You do not get a guarantee that e.g. something else has not interfered with the source or the destination while the copy was in progress or immediately after it finished. And nor should you need such just because you have done a copy.Finally the exact behaviour may be OS dependent. Just for example, Linux certainly should allow delete-while-being-copied, the file does not disappear until the last open handle to it is closed. That would not necessarily mean you would even know that the copy has been aborted/terminated, you would just see no destination file in existence after the copy completed, whether early or late.
-
Hi and thanks a lot for your support... I'll do a test with a large file to check the behaviour of QFile::copy(), at least on Windows... as I'm not really sure regarding your sentence "You should assume the copy has completed as best possible upon return from copy()"
Meanwhile I've found that I'm not alone... here:
https://forum.qt.io/post/371501
they suggest to check whether the size of the destination file keep increasing or not... this does not mean that the file copy has been successfully done (what, for example, in case of a network failure ?)... of course the final check could it be to evaluate the MD5 or SHA1 of the two files but so far I don't really need this level of robustness.
thanks again
Giampaolo
-
@iw2lsi said in when is a QFile::copy() finished ?:
has completed as best possible
I mean, given whatever the underlying OS calls implementing the copy do. Which you have to live with. I gave you as an example: let's say process has done a
write()
, does that mean the bytes are in the file? No, they are probably in a memory buffer waiting to be flushed. Doesclose()
guarantee they will be flushed? No, they can still be in memory. (Try pulling the plug on the machine immediately afterwrite()
orclose()
, what state will the file be in after reboot?) What about if the file being written is on a remote drive on a network? So in part it depends on your definition of when your copy is "completed".I am "unconvinced" about looking at the file size as per your referenced post for any kind of "guarantee". For a start it depends on what calls you use: I would not assume that the result returned by, say,
stat()
is necessarily hand in glove in sync withwrite()
s.fstat()
, orfseek()
, might (or might not) be more accurate. And again I would suggest it is OS specific. There may be no harm in trying some of these --- for example, I accept that probably if they do not report the full size for "a while" after aclose()
something may be "amiss" --- but I don't see "guarantees".Reverting to your original question. I would code to call
QFile::copy()
and assume that it has completed (as best you are going to know) when it returns, and that is all I would think your code needs to know or do. I don't know what else you want to do. You could do some extra check about length if you wish. But why do you care? For example: an external process could delete the file (like you say you did). That could happen one millisecond before your copy finishes or one millisecond afterwards. Either way the end user ends up with no destination file in existence. How are you going to distinguish these and why do you care about the first case and not the second? -
@iw2lsi If I remember correctly, QFile::copy is not the right tool to copy large files. It seems to copy everything into buffer and then flush the contents into the destination file all in once.
Maybe the following steps can be taken to copy large files
- check the source file size first and decide if it is big or not(10MB, 20MB or 50MB)
- if it is small, use QFile::copy
- if it is big, use C code to loop through the file and copy parts(find a good size) of the file one after another to the destination. In this way, you are able to give the user an option to cancel at any time.
-
@JoeCFD said in when is a QFile::copy() finished ?:
If I remember correctly, QFile::copy is not the right tool to copy large files. It seems to copy everything into buffer and then flush the contents into the destination file all in once.
Then you should really look at the sources...
https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qfilesystemengine_win.cpp#n1741
-
@Christian-Ehrlicher When I tried it before, this func did not work as I expected. Not sure if it works now. It has been a while.
-
On Windows (local filesystem) you cannot delete a file that is open by another process. And copying should, in a way, keep the file open. Linux behaves differently as described before. So, if your file lives locally on your computer I'd say the copy has been finished when QFile::copy returns (minus some buffering, but that also would keep the file open, I hope).
From my experience I suspect that this is different when the file is on a network drive (still Windows). I think (haven't checked, though) that copying a file on a samba mount just issues a command to the remote server to do the copy. The local view of the network drive can also become out of sync with the file server. There is a lot of room for error (even though it is only for very short periods of time). I have seen some behavior that suggests that QFile::copy can return before the copy is finished when doing the copy on a samba mount.
I'm also not sure what you expect from a copy (finished or not) followed by a delete to happen. Certainly, not allowing to delete a file while it is being copied leaves the system in a reasonable state. But, if you allow to delete a file while being copied it is quite nice if the copy still finishes as it does on Linux.