How to monitor files on an external device?
-
Hello friends. For some time now I have been trying to develop an app to control the files that are copied or moved on the external device. Let's see, I explain them better. I need to keep a record of the volume of information copied to an external device connected to my pc, but only the new files, not the ones that are moved or renamed. Does anyone have any idea how to do something similar?
-
Hi
How many files are we talking about ?
If you need to handle files that are move to other folders or renamed, then
you need to check all files before copying the truly new ones.
This gets very heavy for many files.
You need to keep record of last name, and some sort of hash / checksum for each file.
Then you can detect if one is moved but not really altered.
There is backup tools that can do this.
Its not trivial to program.
Can I ask what the overall goal is ?
and why you need your own app to do and not some backup tool? -
Hi
How many files are we talking about ?
If you need to handle files that are move to other folders or renamed, then
you need to check all files before copying the truly new ones.
This gets very heavy for many files.
You need to keep record of last name, and some sort of hash / checksum for each file.
Then you can detect if one is moved but not really altered.
There is backup tools that can do this.
Its not trivial to program.
Can I ask what the overall goal is ?
and why you need your own app to do and not some backup tool?@mrjj The number of files is not defined, what I need is to know the space occupied by the new files.
I need to make an application of this type because I do not find any that do what I need.
It is an entity intended to control information and the owner needs to keep track of the amount of information that leaves the company. You just need to know the volume ... not what information came out. -
@mrjj The number of files is not defined, what I need is to know the space occupied by the new files.
I need to make an application of this type because I do not find any that do what I need.
It is an entity intended to control information and the owner needs to keep track of the amount of information that leaves the company. You just need to know the volume ... not what information came out. -
@Blaster
Hi
you mean you just need to calculate the amount of MB on the drive ?
With the added requirement that files already on drive, even if moved around
to new folder or renamed, do not count to that total? -
@mrjj Yes, that's it. Initially it had occurred to me to see the size when putting the device and then verify it when extracting it, but then the calculation fails when files are deleted.
@Blaster
If think you need to use CRC for each file and store in a database.
This way you can discover if file has been moved or renamed and/or changed.
https://stackoverflow.com/questions/27939882/fast-crc-algorithm -
@Blaster
If think you need to use CRC for each file and store in a database.
This way you can discover if file has been moved or renamed and/or changed.
https://stackoverflow.com/questions/27939882/fast-crc-algorithm -
@mrjj but how do I get the amount of megas that are created from the new files? In addition, the process of checking each file is too expensive in time
@Blaster
When I first saw this question days ago I would have suggestedQFileSystemWatcher. I'm sure someone did but there's no mention. So I wonder why @VRonin has deleted a reply, which I would have guessed suggested that?Anyway, let's restart. You're going to have to think out just what you want/can live with, because nothing is going to do exactly as you ask, because there's so many things you can do with files it's hard to define just what you will want here.
I agree you won't want to have examine file contents for CRC, too slow.
Most importantly: when do you need to monitor/examine this information?
-
If you need to in real-time, as the user is doing things, can you leverage
QFileSystemWatcherfor this device? I don't know whether it will support it. -
If you only need to discover this when your program is run, in some shape or form you want to save a "database" of what was there before and compare it to what is there now.
Which approach?
BTW, just maybe, if all you want to know about is the "new" ones, you could simply look at the files' "creation time" attribute, if that's acceptable for your purposes?
-
-
@Blaster
When I first saw this question days ago I would have suggestedQFileSystemWatcher. I'm sure someone did but there's no mention. So I wonder why @VRonin has deleted a reply, which I would have guessed suggested that?Anyway, let's restart. You're going to have to think out just what you want/can live with, because nothing is going to do exactly as you ask, because there's so many things you can do with files it's hard to define just what you will want here.
I agree you won't want to have examine file contents for CRC, too slow.
Most importantly: when do you need to monitor/examine this information?
-
If you need to in real-time, as the user is doing things, can you leverage
QFileSystemWatcherfor this device? I don't know whether it will support it. -
If you only need to discover this when your program is run, in some shape or form you want to save a "database" of what was there before and compare it to what is there now.
Which approach?
BTW, just maybe, if all you want to know about is the "new" ones, you could simply look at the files' "creation time" attribute, if that's acceptable for your purposes?
-
-
@JonB said in How to monitor files on an external device?:
So I wonder why @VRonin has deleted a reply
Because
QFileSystemWatchercan't differentiate about what is happening to the file so the solution is not as easy as it seams@VRonin
Then, my friend, an edit to the post explaining why not might have better than a delete :)I still think that's where you'd start from if you needed it in real-time. And I'm asking the OP to think about just what he wants to know. Maybe all he needs to know about is new file creation, and he'll ignore rename/move? He needs to clarify whether real-time or one-off run program is what he wants.
-
actually, maybe there's a easy way:
Untested Code
// QFileSystemWatcher* fwatcher = new QFileSystemWatcher; QDir targetDir("path/to/folder/"); fwatcher->addPath(targetDir.canonicalPath()); // private: uint oldCount; oldCount = targetDir.count(); QObject::connect(fwatcher,&QFileSystemWatcher::directoryChanged,[targetDir,&oldCount](const QString& path)->void{ if(path!=targetDir.canonicalPath()) return; if(oldCount<targetDir.count()) qDebug() << "New File Added"; oldCount=targetDir.count(); }); -
actually, maybe there's a easy way:
Untested Code
// QFileSystemWatcher* fwatcher = new QFileSystemWatcher; QDir targetDir("path/to/folder/"); fwatcher->addPath(targetDir.canonicalPath()); // private: uint oldCount; oldCount = targetDir.count(); QObject::connect(fwatcher,&QFileSystemWatcher::directoryChanged,[targetDir,&oldCount](const QString& path)->void{ if(path!=targetDir.canonicalPath()) return; if(oldCount<targetDir.count()) qDebug() << "New File Added"; oldCount=targetDir.count(); }); -
@Blaster
When I first saw this question days ago I would have suggestedQFileSystemWatcher. I'm sure someone did but there's no mention. So I wonder why @VRonin has deleted a reply, which I would have guessed suggested that?Anyway, let's restart. You're going to have to think out just what you want/can live with, because nothing is going to do exactly as you ask, because there's so many things you can do with files it's hard to define just what you will want here.
I agree you won't want to have examine file contents for CRC, too slow.
Most importantly: when do you need to monitor/examine this information?
-
If you need to in real-time, as the user is doing things, can you leverage
QFileSystemWatcherfor this device? I don't know whether it will support it. -
If you only need to discover this when your program is run, in some shape or form you want to save a "database" of what was there before and compare it to what is there now.
Which approach?
BTW, just maybe, if all you want to know about is the "new" ones, you could simply look at the files' "creation time" attribute, if that's acceptable for your purposes?
-
-
@VRonin What you propose only returns me the amount of new files in the directory and what I need is the amount of space it occupies, but that's the point ... I think I can go from there
@Blaster said in How to monitor files on an external device?:
@VRonin What you propose only returns me the amount of new files in the directory and what I need is the amount of space it occupies, but that's the point ... I think I can go from there
QFileSystemWatcherreturns path of file changed. The important thing is that only fires when a change is being made. (Remember, you still need to test whatever your "device" is to make sure it fires change events.) You would use aQFileInfoon that to get the size information you are interested in. @VRonin'scount()is only an example. This combination is as efficient as possible.EDIT
Wait, I see, I get it now.QFileSystemWatcherwon't tell you which file(s). OK, that's problematic.You'd have to do something like: iterate all children by creation time to spot which ones are new. At least you'll only do it when something gets added or deleted, but not great.
Go back to question: do you need to know in real-time when user does something, or just once at some specified point when they run an app of yours?
P.S. What OS? If Windows, and you find you really need this and have to roll your own, maybe some of the suggestions in https://stackoverflow.com/questions/2107275/does-anyone-have-a-filesystemwatcher-like-class-in-c-winapi would help? Better: https://developersarea.wordpress.com/2014/09/26/win32-file-watcher-api-to-monitor-directory-changes/ And the WinAPI is around https://docs.microsoft.com/en-gb/windows/desktop/api/winbase/nf-winbase-readdirectorychangesw , that leads to event "FILE_ACTION_ADDED 0x00000001 The file was added to the directory. ". You may need the Win level to get exactly what you want specifically rather than
QFileSystemWatcher.Otherwise, this might sound like a journaling requirement? It's all getting a bit hairy...!
-
@Blaster said in How to monitor files on an external device?:
@VRonin What you propose only returns me the amount of new files in the directory and what I need is the amount of space it occupies, but that's the point ... I think I can go from there
QFileSystemWatcherreturns path of file changed. The important thing is that only fires when a change is being made. (Remember, you still need to test whatever your "device" is to make sure it fires change events.) You would use aQFileInfoon that to get the size information you are interested in. @VRonin'scount()is only an example. This combination is as efficient as possible.EDIT
Wait, I see, I get it now.QFileSystemWatcherwon't tell you which file(s). OK, that's problematic.You'd have to do something like: iterate all children by creation time to spot which ones are new. At least you'll only do it when something gets added or deleted, but not great.
Go back to question: do you need to know in real-time when user does something, or just once at some specified point when they run an app of yours?
P.S. What OS? If Windows, and you find you really need this and have to roll your own, maybe some of the suggestions in https://stackoverflow.com/questions/2107275/does-anyone-have-a-filesystemwatcher-like-class-in-c-winapi would help? Better: https://developersarea.wordpress.com/2014/09/26/win32-file-watcher-api-to-monitor-directory-changes/ And the WinAPI is around https://docs.microsoft.com/en-gb/windows/desktop/api/winbase/nf-winbase-readdirectorychangesw , that leads to event "FILE_ACTION_ADDED 0x00000001 The file was added to the directory. ". You may need the Win level to get exactly what you want specifically rather than
QFileSystemWatcher.Otherwise, this might sound like a journaling requirement? It's all getting a bit hairy...!
@JonB Exact. It is a bit tangled.
I need to know the volume of information, by flash drive inserted in the pc. It is not necessary for the user to know that the new file was registered in the application, only those who have the administration password of the application should know it, when they open it. The OS is Windows.
