Delete old dirs

[ODБOï]

Hi,

I'm trying to delete old folders in a given path and only keep 3 (more recent ) folders. I'm not sure about what i did for the moment, can someone suggest an improvement or a better approach please ?

my working dirextory is "C:/Users/Public/VNC/ClientVnc"

#define WORKINGDIR  "C:/Users/Public/VNC/ClientVnc"
...
 void deleteOldDirs2(){
        QDirIterator it(WORKINGDIR);
        QStringList pathList;
        while (it.hasNext())
        {         
            QString name = it.next(); 
            pathList<<name;
        }
        if(pathList.length()>=5){
            for(int i = 0; i<pathList.length() ;i++){
                QDir dir(pathList[i]);
                if(dir.dirName()!="."&&dir.dirName()!=".."&&i<3){
                    qDebug()<< dir.dirName();
                    if(dir.exists())
                        dir.removeRecursively();
                }
            }
        }
    }

this is working as i want but i'm not sure why /how..

maybe because the dir names are the dates of creation followed by the hour (in french..)

jeu. avr. 4 2019 14-35-47
jeu. avr. 4 2019 14-42-33
mer. avr. 3 2019 14-35-47

Should i create QDate objects and compare them to determine witch ones are oldest or my first approach should be ok?

[JonB]

@LeLev

• For choice you should really look be looking at the "creation date" on directories to determine their date order. However you look like Windows, and I'm not sure what that does about creation-date on directories (as opposed to files), you'd have to check. (QFileInfo::birthTime()?)

• Instead, you seem to be relying on the order your directories are delivered back to you from QDirIterator to decide which to delete? I see no docs stating what this order will be, and I would not rely on any particular order. I don't know why this seems to delete what you say you want (may be to do with order directories were originally created).

Anyway, especially since you are recursively deleting hierarchies, you should surely do some check on the name to decide which ones you want to delete? Sort them into date order, parsing the names as necessary, and then delete from that....

P.S.
It may indeed be that Windows delivers back files/directories in the order they were created because they are laid out like that in the "Windows inode" table. And that would actually suit your purposes great for deleting from oldest onward. However, in the absence of firm docs stating so, you are relying on QDirIterator and whatever it calls underneath on Windows. For all I know, it might vary if the FS is FAT vs NTFS, so you would be on shaky ground, up to you....

[ODБOï]

@JonB thank you very much for taking time to provide complete answer

i did like you suggested :

 void deleteOldDirs2(){
        QDirIterator it(WORKINGDIR);
        QStringList pathList;

        QList <QDateTime> dts;

        while (it.hasNext())
        {
            QString name = it.next();
            pathList<<name;
        }
        if(pathList.length()>=5){
            for(int i = 0; i<pathList.length() ;i++){
                QDir dir(pathList[i]);
                if(dir.dirName()!="."&&dir.dirName()!=".."){
                    //ex  : jeu. avr. 4 2019_16-29-47
                    QString tmpDirName = dir.dirName().replace("_"," ").replace("-",":");
                    QString format = "ddd MMM d yyyy hh:mm:ss";
                    QDateTime dt = QDateTime::fromString(tmpDirName,format);
                     if(dt.isValid())
                    dts << dt;
                }
            }
// edit : i add sorting function here
  std::sort(std::begin(dts), std::end(dts));

            for(int j=0; j<dts.length();j++){
                qDebug()<<dts[j].toString();
            }
        }
    }

now in my QList <QDateTime> dts; dates are sorted correctly, sould i re-do some sorting explicitely ?
thx

[JonB]

@LeLev
If I understand you right: I still do not see any sort() call anywhere? So your list is only already correctly sorted because the directories happen to be sorted that way in the order QDirIterator returns them. Which you already knew empirically, but (I think) should not rely on.

I would expect to see some kind of std::sort(dts). dts contains a list of QDateTimes, so whatever it takes to sort with that data in the list. Bearing in mind I'm not a C++-er, looks like you might use qsort() with some comparer function, e.g. see https://stackoverflow.com/questions/46804220/how-to-sort-qlist-according-to-a-certain-order-not-alphabetical for an example. But it might be that nowadays you should use std::sort(), http://www.cplusplus.com/reference/algorithm/sort/, Qt example in https://evileg.com/en/post/243/.

[ODБOï]

@JonB i edited the code. Thank you again.

Last question please : the folders i will delete will contain about 6Go of image files, so should i call app.processevents() in my for loop where deletion is done ?

[JonB]

@LeLev
I'm on holiday so this is my last reply :)

Yes, you might want to call processEvents() if it's taking a long time (size of files may not matter as much as number of files). It's always a bit naughty calling that, and you may have to keep an eye on other areas of code being re-entered if you do. Or, you might decide that the whole deletion is best done in a thread of its own since it's "standalone"?

After your code what you have a is a sorted list of dates. You then need to get back to the directory name which needs deleting, you started from that but do not have it now. You either then have to reconstruct the name from the date (nasty/dodgy?), or you should actually create/store/sort a struct for each one containing the date plus the original name. Alternatively you might want to use a QMap or std::map for this (key is date, value is name), e.g. see https://stackoverflow.com/a/26695746. If you use a map (at least QMap, not sure about std::map), you don't ever have to do a sort(), the sorting is done when keys are inserted into the map and so you just iterate the keys and they will already be in sorted order.

P.S.(!)
If you feel/find you can trust https://doc.qt.io/qt-5/qdir.html#SortFlag-enum, changing over to QDir::entryList() with QDir::Time instead of QDirIterator, you can avoid all your own sorting. You'd have to test it out on directories under Windows....

[ODБOï]

@JonB said in Delete old dirs:

you started from that but do not have it now

yes, i just was thinking about a way to implement the mapping between my sorted dates list and the dirs list , and i thought about QMap also, struct one seems nice also. Thank you

@JonB said in Delete old dirs:

P.S.(!)
...

i will check also

[ODБOï]

finally i did the deletion part with 2 nested for loops. I know this has horrible complexity but for my use case it should feet.
--the max number of folders will be 4 and every time i will delete only one (oldest)

void deleteOldDirs2(){
        QDirIterator it(WORKINGDIR);
        QStringList pathList;
        QList <QDateTime> dts;
        QString format = "ddd MMM d yyyy_hh-mm-ss";
        while (it.hasNext())
        {
            QString name = it.next();
            pathList<<name;
        }
        if(pathList.length()>=4){
            for(int i = 0; i<pathList.length() ;i++){
                QDir dir(pathList[i]);
                if(dir.dirName()!="."&&dir.dirName()!=".."){
                    QDateTime dt = QDateTime::fromString(dir.dirName(),format);
                    if(dt.isValid()){
                        dts << dt;
                    }
                }
            }
            std::sort(std::begin(dts), std::end(dts));
            for(int j=0; j<dts.length()-2;j++){ // keep 3 folders (!iterate)
                //qApp->processEvents(); // ?
                for(int o=0;o<pathList.length();o++){
                    //qApp->processEvents(); // ?
                    QDir dir(pathList[o]);
                    if(dir.dirName()!="."&&dir.dirName()!=".."){
                        QDateTime dt = QDateTime::fromString(dir.dirName(),format);
                        if(dt.isValid()){
                            if(dts[j]==dt){
                                qDebug()<<"Deleting : " << dir;
                                dir.removeRecursively();
                            }
                        }
                    }
                }
            }
        }
    }

I wonder if i have to uncomment the qApp->processEvents() calls, there is maybe 30-40000 files in each folder

[J.Hilk]

@LeLev
you can easily move that into a thread

//Where you would call deleteOldDirs2

QThread *t = QThreat::create(deleteOldDirs2);
t->start();

emit a signal once your deleteOldDirs2 returns to delete the QThread object and you're done.

[ODБOï]

hi
@J.Hilk said in Delete old dirs:

//Where you would call deleteOldDirs2

i call it in the ctor of the same class,
So i tryed

  QThread *th = QThread::create(deleteOldDirs2);
    th->start();

but output is :
erreur : reference to non-static member function must be called; did you mean to call it with no arguments?
erreur : no matching function for call to 'create'
C:\Qt\5.12.0\mingw73_64\include\QtCore\qthread.h:174: candidate template ignored: substitution failure [with Function = void, Args = <>]: cannot form a reference to 'void'

Do you know what the problem is ?

@J.Hilk said in Delete old dirs:

= QThreat

a slip ? :D

edit i just tryed with a static function and it works, but my function can't be static because i need to access some members

[J.Hilk]

@LeLev auto complete corrects QThread to Thread, every time x)

Well, member access in a threaded function is a bad idea, you could capture them and pass them as arguments ?

That would work.

[ODБOï]

@J.Hilk
sorry! i'm actually not accessing any member at all in that function, so i made it static, and it works, now i will try to see how to emit a signal from it to delete the qthread
thank you

[J.Hilk]

@LeLev keep me posted, I haven't used the create function of QThread jet, bu it should be more lightweight that QtConCurrent, which would be your other alternative.

[ODБOï]

@J.Hilk
i can't not find a way to emit a signal from the static function..
but i think simply delaying the deletion of the QThread is enought for my usecase, deleteOldDirs2() will take 5-s seconds maximum (+the object is created only once and the app will restard automatically every 5 hours so no memory increase).

   QThread *th = QThread::create(deleteOldDirs2); 
   th->start();
   QTimer::singleShot(20000, [=]() { // 20sec must be safe 
        th->deleteLater();
   });