Watching a directory tree without inotify

Question

I'm trying to write a backup utility that is supposed to handle a rough 2 terabytes of data in a lot of folders.

I want it to perform actions on files when they get created/edited/deleted preferably also file moves or renames.

I've messed around with fanotify, only to realize it only works with file edits. And I'm against using inotify if I can since I would have to edit the maximum file watches, which I don't want to do. It would have a big performance impact I presume.

I'd preferably just set a single filewatch that works recursively to all files underneath it. Is there anyone who has experience with this who knows what a good method is? Should I go for inotify and just take the performance hit? Or is there a different approach I can take?

Answer 1

I don't think there is a way to recursively watch for changes in a directory tree. On the other hand, with inotify, you don't need to create one file descriptor per directory you watch. You create a single inotify object and then add many directories to it with inotify_add_watch

int inotify_add_watch(int fd, const char *pathname, uint32_t mask);

Answer 2

I wrote a polling backup application for a tiny RAMDrive; I have tons of experience restoring fixed a few issues. I am not generally a fan of polling but I was using windows and std.file libs are ancienct: Windows 10 makes their lib 5 Major versions behind; so don't worry about portability.

I was planning 2 modes: active and sleeping that would prevent disks from spinning down pointlessly once before they switch to a longer time that would allow them to stay spun down. I have only begun to think about detecting and repairing and the ZFS solution of medium replication seems naive because WinRAR allows you to do just this with archives.

I bought a new computer a month ago and I'm still trying to backup my files. Beware using phobos i.e. std.anything. std.file' copy changes creation time which is incorrect for restored files; after copying you need to read the old file's creation time and call the OS function yourself and set the creation time; all that meta-data comes with a windows system call so that is a good place to start researching the data in Linux system calls.

Now is probably a good time to re-visit Python and try: https://docs.python.org/3/library/pathlib.html Spoiler: it is broken in 3.5.2.