How to stop file copying

JoeCFD

It is easy to handle if there are a lot of files. What is the best way to stop in the middle of copying in case of a single large file(for example: a few GB). It may not be a good idea to simply kill the thread for QFile::copy.

Christian Ehrlicher

Don't use QFile::copy() then but your own implementation.

VRonin

I remember there was a push to separate QFile as QIODevice from QFile as file system manager but I don't know what happened to it, can't even find the ticket anymore

JoeCFD

@Christian-Ehrlicher good call.
Found something which may be helpful for others as well.
https://stackoverflow.com/questions/10195343/copy-a-file-in-a-sane-safe-and-efficient-way

VRonin

@JoeCFD said in How to stop file copying:

Found something which may be helpful for others as well.

std::filesystem::copy is always an option

@JoeCFD said in How to stop file copying:

QFile::copy can not even copy large size files. A bit disappointed.

You can do it with QFile, instead of using the copy method open a file for reading and a file for writing and move them across in chunks

JoeCFD

@VRonin my os(for production) does not support std::filesystem::copy unluckily and I can not change that. It is kind of lame. It seems I do not have other choices than your suggestion. Thanks.

Christian Ehrlicher

@JoeCFD said in How to stop file copying:

QFile::copy can not even copy large size files.

I don't understand this - why should QFile::copy() can not copy 'large' files ?

JoeCFD

@Christian-Ehrlicher Sorry. It takes long. I tried a large file and somehow the usb light is off. I thought QFile::copy has problem.

JoeCFD

@VRonin I did this with buffer size. However, QFile close() call takes very long. It turns out that the copied data is cached at first. In closing, QFile flushes the data to the disk. Tried to flush after each buffer write and no success. Stop still does not work. May take a look at how QSaveFile works.

Christian Ehrlicher

@JoeCFD said in How to stop file copying:

In closing, QFile flushes the data to the disk.

It's not QFile but the OS.

JoeCFD

@Christian-Ehrlicher True. It seems it is not doable on Linux to make a progress bar or stop in the middle of copying when file size is big. Just tested a transfer of a file with size 260MB with rsync. Caching time is less than 1s. But flushing takes more than 1 minute.

jsulm

@JoeCFD said in How to stop file copying:

It seems it is not doable on Linux to make a progress bar

It is doable if you implement it manually (instead of using QFile::copy()). I mean: read the original file in blocks and write each block to the new file. Between each block you can update the progress bar.

JoeCFD

@jsulm I did it with some source code in QFile::copy(). No help. Flushing happens when close() is called. This is how copy works on Linux.
char block[ 131072 ];
qint64 total_read{ 0 };

            while ( false == source_file.atEnd() ) {
                qint64 read_in = source_file.read( block, sizeof( block ) );
                if ( read_in <= 0 ) {
                    break;
                }

                total_read += read_in;
                if ( read_in != destination_file.write( block, read_in ) ) {
                    break;
                }

                value += read_in / 1024.0;
                progress_dialog.setValue( static_cast< int >( value ) );

                if ( true == progress_dialog.wasCanceled() ) {
                    break;
                }
            }

            source_file.close();
            destination_file.close();

Christian Ehrlicher

@JoeCFD said in How to stop file copying:

This is how copy works on Linux.

On every os where the caching is not explicitly disabled (which is not by default) - that's what a cache is for.

jsulm

@JoeCFD said in How to stop file copying:

Flushing happens when close() is called

I did not say the opposite, I was talking about progress bar...

JonB

@JoeCFD
I don't understand what you are saying. Yes, flushing happens when close() is called. Yes that can take a while. In particular it depends what the device is, and how much buffering is going on in memory. Which you may be able to affect/change, e.g. via some system configuration.

If you are saying you want more time spent during the copying and less time spent at the end of the copying, try using Linux man sync(2). That's what you need to flush. You will probably want to call int syncfs(int fd); on the file descriptor you are copying to. Don't know how/whether you get to that from Qt, have a look through QFile, or you can use level 2 Linux calls to do the whole file copy. The syncfs() call needs to go inside your loop.

You should see a noticeable effect of syncing especially if you use a slow device like a USB stick.

JoeCFD

@jsulm If flushing takes more time than write, progress bar display does not make a lot of sense.

JonB

@JoeCFD
That is true, and there are many apps which are guilty of this. How many of them reach 100% fairly quickly and then sit there for ages? I'm looking at you, Microsoft, in particular!

Other than trying to calculate what the residue will be at the end, and allowing for that during your progress, do try the syncing, I think this will give you more accuracy. P.S. Don't do the sync too frequently. If you write one byte at a time and sync between each your operation will be slow! :) Your block is ~128K, that might be fine.

JoeCFD

@JonB WIll check it out. Thanks.