Incorrect QFile QFileInfo size if file>2Gb on NTFS?

Symbioxy

Hi,

I am developing an app to catalog and search files, currently under Linux only.
This is my first C++/Qt/KDE app.

I have issues to get correct files size, I see negative numbers or zeros in the results.

My testing so far:

same results with QFile or QFileInfo .size
file size looks ok on ext4 drives
file size looks ok on ntfs drives if lower than ~2Gb
file size is incorrect if it is bigger than ~2Gb AND on NTFS drives
in that case, I get negative values if fileSize is an int, or zeros if I declare fileSize as qint64 or long
putting an IO test to read the file does not change the result
I checked the disk under windows to ensure there were no corrupt filesystem info
tried to build a workaround/function to get the size using c++ directly, same results

        //Iterate in the directory to create a list of files
                QStringList fileList;
                QDirIterator iterator(directory, fileTypes, QDir::Files, QDirIterator::Subdirectories);
                while (iterator.hasNext()){
                    
                    //Get file information  (abosulte path, size, datetime)
                    QString filePath = iterator.next();
                    
                    int fileSize;
                    QFileInfo fileInfo(filePath);
                    fileSize = fileInfo.size();
                    //tested: fileSize = get_file_size(filePath.toStdString());
        
                    QDateTime fileDate = fileInfo.lastModified();
                    
                    //add the data to the list, @@ is used as a separator for now
                    fileList << filePath + "@@" + QString::number(fileSize) + "@@" + fileDate.toString("yyyy/MM/dd hh:mm:ss");
                }

workaround I tried but which gives the same incorrect results for files >2Gb/NTFS:

int MainWindow::get_file_size(std::string filename) // path to file
{
    FILE *p_file;
    p_file = fopen(filename.c_str(),"rb");
    fseek(p_file,0,SEEK_END);
    int size = ftell(p_file);
    fclose(p_file);
    return size;
}

I have tested other similar apps to catalog files under linux, one is using wxwidget for instance, and it gets the right file size.
So am I missing something, or is it a limitation of the QFile/QFileInfo capability under Linux ?
Is there a work around other than using other libraries that seem to cope with this case ?

Thank you!

JonB

@Symbioxy
You don't say where you get your Qt from, nor what version it is. Start by confirming that, and is is 32- or 64-bit?

Symbioxy

I'm running Qt 5.15.0, on an openSUSE Tumbleweed 64-bit that I installed from the open source version, Qt Online Installer:
https://www.qt.io/download-open-source?__hstc=&__hssc=&hsCtaTracking=9f6a2170-a938-42df-a8e2-a9f0b1d6cdce|6cb0de4f-9bb5-4778-ab02-bfb62735f3e5

JonB

@Symbioxy
So, being 64-bit, you should find that int is still 32-bit size (qDebug() << sizeof(int);). That is going to start giving negative numbers for > 2GB.

QFileInfo::size() is documented as type qint64, and ftell() returns either a long int, size_t or int64_t. Start by changing your declarations of your int fileSize; & int size, and int MainWindow::get_file_size(), to match correctly, then see where you are?

Symbioxy

Thanks a lot JonB for the help and info, I did not know about int being 32-bit.

I just tested:

using the QFileInfo::size() and declaring qint64 fileSize;
or using the workaround function: long int get_file_size(std::string filename); and long int fileSize;

in both cases I now get only a size of 0 for the files above 2GB.
(it seems only using int fileSize with QFileInfo::size() gives negative numbers)

but I may have jumped into conclusion to fast about the ext4 / ntfs, I think all files above 2Gb have the same issue.
I'll do some more tests to confirm.

Pablo J. Rogina

@Symbioxy said in Incorrect QFile QFileInfo size if file>2Gb on NTFS?:

I now get only a size of 0 for the files above 2GB.

Just in case, per QFileInfo::size() documentation:

If the file does not exist or cannot be fetched, 0 is returned.

So you may want to double check if you indeed can access the file(s) (especially since you stated you're using Linux and trying to read a NTFS file...)

Christian Ehrlicher

@Symbioxy said in Incorrect QFile QFileInfo size if file>2Gb on NTFS?:

I did not know about int being 32-bit.

Wouldn't it be helpful to inform yourself about the basic stuff of C before starting to program high level stuff?

Symbioxy

Thank you all,
Sorry if I poorly interpreted my issue, do not have a solid computing background, mostly learned from try and error...
any good page you'd recommend in this area?
I wish I could correct the title of the post now.

I went through all my declarations and methods to get everything in qint64 instead of int.
I am now retrieving the file sizes properly :)

One last issue: to displayed it in a treeview.
I store this into a text file, and when retrieving it to load the model, I need to convert the QString back into a number.
So far I was using :

                        //Reminder: the double @ separates the filepath, size, and datetime.
                        //Split the string with @@ into a list
                        QRegExp tagExp("@@");
                        QStringList fieldList = line.split(tagExp);
            
                        // Get the values from the list:
                        QString filepath        = fieldList[0];
                         int filesize         = fieldList[1].toint(); 
                         QString filedatetime    = fieldList[2];

now if I change to qint64, what is the correct way to convert the value from a QString to this?
There is no .qint64() in the QString class
.tolonglong() gives me negative values again.

Thanks

Pablo J. Rogina

@Symbioxy said in Incorrect QFile QFileInfo size if file>2Gb on NTFS?:

One last issue: to displayed it in a treeview.

Please just one topic per thread. If you solved the issue with the file size, please mark this post as solved and then open a new entry about the issue with treeview

JonB

@Symbioxy said in Incorrect QFile QFileInfo size if file>2Gb on NTFS?:

I went through all my declarations and methods to get everything in qint64 instead of int.
I am now retrieving the file sizes properly :)

This is good then! :)

Since @Pablo-J-Rogina has asked you to close this topic and create a new one for your new question, I will answer that when you do so :)