[Solved] Duplicate finder

DenisKormalev

If you want to find fully similar files then some hash will help you like md5 or sha1. If you want to find files that are different a bit but quiet similar in general then you should use something more complex like Levenshtein distance (if you compare files with some plain text) or some diff analysis (something like if diff is smaller than 10% of smallest size then they are similar enough).

goetz

Moved to Brainstorm forum, as it has nothing to do with Qt programming as such (initially moved to C++ Gurus, but it's not C++ related either, sorry).

DenisKormalev

Volker, I've thought where it should be moved too, but forgot about brainstorm and left it to someone who will have some ideas. Thanks.

alex.dadaev

Maybe i've misunderstood something, but how can md5 ot sha1 help me with finding dublicates? I thought that its cryptohraphic algorithms for government use :)

goetz

If two hashes are equal then it is very likely that the two files they are computed of bitwise equal content (although not guaranteed as the set of all possible file contents is indefinite, whereas the number of possible hashes is limited and thus there cannot exist an isomorphic relation between them). You can store the hashes in a database and search for duplicates in that. For "similar" files you'll have to go with Levenshtein or other means, as Denis stated.

alex.dadaev

But how can i store file info in a hash ?

DenisKormalev

You can pass it contents through hash function and you will have it hash at the end.

goetz

[quote author="alex.dadaev" date="1295897294"]But how can i store file info in a hash ?[/quote]

You cannot. This "web page":http://lmgtfy.com/?q=hash+function has some explanations for you.

alex.dadaev

Okay :)

alex.dadaev

Is there any way to use QHash methods in QCryptographicHash ?
I'd like to make a comparing table for files that i hash.

tobias.hunger

QHash is a hash table, a datastructure optimized for random access based on a key value.

QCryptographicHash is used to calculate cryptographic hash values from input data. They are completely different things:-)

So, no, you can not use QHash's methods in QCryptographicHash. Just use the result of a QCryptographicHash as a key to a QHash and you should be set. Just make sure to reset the QCryptographic hash whenever you are done with a file, or you will not get the same hash values for the same files (since the second one will still have all the data of the first one "prepended").

goetz

Yes, that's possible. You can use the following function as a start for your project:

@
#define MY_SHA1_BUFFER_SIZE 4096

QString getSha1HashFromFile( const QString &fn )
{
QCryptographicHash ch( QCryptographicHash::Sha1 );
QFile file( fn );
if( !file.open( QIODevice::ReadOnly ) )
return QString();

char buf[MY_SHA1_BUFFER_SIZE];
while( !file.atEnd() ) {
    qint64 read = file.read( buf, MY_SHA1_BUFFER_SIZE );
    ch.addData( buf, read );
}
file.close();
return QString( ch.result.toHex() );

}
@

alex.dadaev

how can i make QString from QFileInfoList ?
is there any possibilities to do that?

giesbert

QFileInfo::path() ???

goetz

No.

QFileInfoList is a typedef for QList<QFileInfo>.

You know how many properties a QFileInfo object describing a single file has, don't you?

If you want a single string from these big bunch of information you will have to construct it yourself.

giesbert

Ok, forgpot to add the iteration by for(...)...

alex.dadaev

but how can i construct a path of a single file?

goetz

Read the docs on "QFileInfo":http://doc.qt.nokia.com/stable/qfileinfo.html - we did it too. Everything you need is documented there. Yes, it takes some 5 minutes to read it all through, but if you're too lazy we can't help you. If you have concrete questions or problems with any of the methods, ask them.

giesbert

If you read the documentation, you would find it ...

@
QFileInfoList list;
for(int i = 0; i < list.size(); ++i)
{
QString filePath = list[i].absoluteFilePath();
{
@

alex.dadaev

i've made it by myself already :) the reason why i ask so dumb questions is because i'm just starting using Qt and programming itself and i just want not to make stupid mistakes.
@QString path[list.size()];for (int i = 0; i < list.size(); ++i) {
QFileInfo fileInfo = list.at(i);
path[i] = fileInfo.path();}@