How to remove elements from QStringList by comparing each other members.

mrjj

Hi
How big are each file ?
Can they all fit in memory at same time?

What is the data inside ?

Often one can use a checksum
to see if files are the same but it depends on the data as
sometimes a sequence of letters can have the same
checksum as another even we logically would say they are not the same.

Ayush Gupta

@mrjj
I have the code to check if file are same or not.
My problem is I am able to filter name from QStringList. I tried index also.
Suppose there are 6 files in list and I need to compare each file ( for that I have code) which will do the thing.
Then I need to compare the file name if fileName1 > fileName2 then delete fileName2 from list if comparison is equal.

jsulm

@Ayush-Gupta https://doc.qt.io/qt-5/qlist.html#erase

mrjj

Hi
so if
fileName1 and fileName2
is the same.
You must delete the file with the highest number in its name ?

Ayush Gupta

@mrjj yes

mrjj

@Ayush-Gupta

Do you only have files 1 to max 9?

    QString filename1 = "somename1";
    QString filename2 = "somename2";

     int fno1=filename1.right(1).toInt();
     int fno2=filename2.right(1).toInt();
     
     if (fno1 < fno2 ) ....

else you need better extraction.

VRonin

Something like this?

// QVector<QFile> fileList;
// bool equalFiles(const QFile&,const QFile&);
bool foundI;
for(auto i=fileList.begin(), maxJ = fileList.end(), maxI=maxJ-1;i!=maxI;){
    foundI=false;
    for(auto j=i+1;j!=maxJ && !foundI;){
        if(equalFiles(*i,*j)){
            if(i->fileName()>j->fileName())
                j = fileList.erase(j);
            else
                foundI=true;
        }
        else
            ++j;
    }
    if(foundI)
        i=fileList.erase(i);
    else
        ++i;
}

Chris Kawa

@VRonin That's a brute force N! complexity algorithm on file contents. That's horrible.

@Ayush-Gupta Create a [hash]->[filename] map. Read each file in sorted order and calculate their hash (MD5, SHA-1 or whatever works for you). Put the file name in the map using that hash and your resulting map will contain all the filenames with unique contents.

Christian Ehrlicher

@Chris-Kawa Before creating the hash I would compare the filesize (and hope that they all differ :) )

Chris Kawa

@Christian-Ehrlicher Good idea. You could use the size as initial hash value and only do a "full" hash for hash conflict resolution (caching the result of course).

Ayush Gupta

@Chris-Kawa Thanks