QFile used in another process

[Daddedebad]

Hey, I have a strange problem with my code. I look for File1 pairID.bin & File2 pairID.bin whenever my QFileSystemWatcher detects a folder change and when those are found together I read the data in them to process in the rest of my program.

This works fine the first 10-20 times I copy-paste those two files. After that I start having issues with opening File1, error code 5: The process cannot access the file because it is being used by another process.

From now on when I restart the program I keep getting this issue instantly. Sometimes it works once or twice but The issue is frequent despite restarting the program.

Here is how I look for the two Files:

1. QFileSystemWatcher calls on function Scanfiles:

void my_user_interface::scanFiles()
{
    QVector<fileData> data = QVector<fileData>(2);
    
    if (findFiles(files))
    {
        readFile(data[0], files[0]);
        readFile(data[1], files[1]);
        if(!files[0].remove())
        {
            qFatal(QString("Error text: %1 Error code: %2").arg(files[0].errorString()).arg(files[0].error()).toUtf8());
        }
        if (!files[1].remove())
        {
            qFatal(QString("Error text: %1 Error code: %2").arg(files[1].errorString()).arg(files[1].error()).toUtf8());
        }

2. FindFiles() is called to look for the pairs of File1 & File2. They must both exist and have the same pairID for the function to return true:

bool findFiles(QFile* files)
{
    QDir dir("Folder");
    QRegExp regExp("\\b(File1)\\b");
    QStringList splitFileName;
    QStringList fileNames;
    dir.setNameFilters(QStringList("*.bin"));
    fileNames = dir.entryList();

    for (int i = 0; i < fileNames.size(); i++)
    {
        if (regExp.indexIn(fileNames[i]) == 0)
        {
            splitFileName = fileNames[i].split(" "); // ex. splitFileName = {"File1", "data.bin"}
            if (splitFileName.size() == 2)
            {
                if (fileNames.contains("File2 " + splitFileName[1]))
                {
                    files[0].setFileName("Folder/File1 " + splitFileName[1]);
                    files[1].setFileName("Folder/File2 " + splitFileName[1]);
                    return true;
                }
            }
        }
    }
    return false;
}

That seems fine, files are found and their directory names are set correctly.

3. Next step is reading one of the files:

readFile(fileData& rfileData, QFile& rFile)
{
    if (!rFile.open(QIODevice::ReadOnly))
        qFatal(QString("Error text: %1 Error code: %2").arg(rFile.errorString()).arg(rFile.error()).toUtf8());
        
    QDataStream* data = new QDataStream(rFile.readAll());
    data->setVersion(QDataStream::Qt_5_15);
    readData(data, rfileData); // Reads datastream and stores into the fileData struct
    delete data;
    

4. After the read, the files are removed as can be seen in step (1.)

It is at step (3.) that the qFatal message is sent that says the file is used in another process.

My program is rather large so I have not shown the entire code, but the code shown above is everything that touches the files I am struggling with. I can try and elaborate with more code if it is needed, but is there perhaps already some obvious mistakes in the code shown that I have made?

I have tried using QLockFiles as well but all I get is a different behaviour where reading the .bin files seems to be reading something completely different... Would that otherwise be a potential solution I should dig deeper into?

What I find weird is that it

1. says the files are used in another process already
2. Works fine the first 10-20 times then when the issue starts It becomes persistent despite restarting the program. Makes me think the process that uses the files are outside the program, somehow I am not releasing the file or whatever process that uses it entirely after removing it?

I am thankful for all the help I can get!

Update:

I decided to add a timed loop which seems to solve the issue. So the process that occupies the file is very short but just enough to cause issues.

The updated code:

readFile::readFile(fileData& rfileData, QFile& rFile)
{
    QTimer timer(this);
    timer.start(1000);

    while(!rFile.open(QIODevice::ReadOnly) && timer.remainingTime() > 0);

    if (!rFile.isOpen() && !rFile.open(QIODevice::ReadOnly))
        qFatal(QString("Error text: %1 Error code: %2").arg(rFile.errorString()).arg(rFile.error()).toUtf8());
        
    QDataStream* data = new QDataStream(rFile.readAll());
    data->setVersion(QDataStream::Qt_5_15);
    readData(data, rfileData); // Reads datastream and stores into the fileData struct
    delete data;

[JonB]

@Daddedebad
The first thing is since this may well be OS-specific behaviour you really need to state what OS you are using....

...And if it's Windows do you have a virus scanner running...?

And since you know those files are changing in that directory (right?), and they are .bin files, and I assume they are created or used by another process (right?), is it possible that intentionally locks them against other processes (i.e. yours) reading them at some times?

[Daddedebad]

@JonB I am using windows 10.

Regarding the binary files, the idea is that another computer creates & puts those file1 pairID.bin and file2 pairID.bin files in there. But currently I am not connected to that program so I simply copy-paste a sample of the two files. So those .bin files are not altered by any other program that I know of at least.

[Daddedebad]

@Daddedebad Also yes there should be a virus scanner running...

[JonB]

@Daddedebad said in QFile used in another process:

@Daddedebad Also yes there should be a virus scanner running...

...So you know what to try next....

[Daddedebad]

@JonB Hey, sorry for the delay.

I needed admin rights for that since this is work-related. While I waited for that I decided to put on the file opening on a timer which seems to work for my issue. I noticed that if I alter the code to read more files at the same time (file1 & File2 & File3 & File4 for example) I no longer had issues as well. So it seems the transfer itself is the process that makes the file busy and that my program opens the files too quickly, or something(?).

I probably should look at how to fix this in the QFileWatcherSystem since it seems too trigger to early but for now I think the issue can be marked as solved.

[JonB]

@Daddedebad
What I was expecting was for you to try disabling the virus scanner for a while and seeing if then there is no issue.

In my experience some virus scanners at least open a file with exclusive access to examine it. that could cause your "file is used in another process". They tend to do this when a file is created or written to. I suppose it could also be from the Windows "copy file" itself, don't know what that does.

As you seem to have discovered for yourself: you may need (a) some sort of delay, e.g. when the QFSW notifies you of change set off a timer for, say, one second to raise the signal at the end which instructs your code to read the file; and/or (b) you may have to retry opening it for read a certain number of times one second apart to allow for this, before giving up.