how to determine if a file is completely downloaded using kqueue?

Question

I want to implement a function which monitor a directory and perform some action when a new file is downloaded from the Internet, but found it difficult to determine if the file is completely downloaded, is there a way to do that?

Answer 1

Usually tools that show the hash of a file will give the state of a file - this should be compared to the hash of another file - if identical then we know the file has downloaded successfully.

md5 (native to bsd) is available - but is only practical on a local file -

If you are retrieving the remote file via HTTP , then there is no way to get the hash of the file without downloading it first (whether it is to STDOUT or piped to file , using wget -O- or curl )

If the file host has a second file that contains the md5 hash of the file being downloaded - then a comparison of the locally downloaded hash is comparable to the hash provided by the file provider.

To do anything more swish will require a comprehensive program to be written - such as the combination of this question and accepted answer : Python Compare local and remote file MD5 Hash

Answer 2

Besides MD5, there is a simple way to do this:

Partially downloaded file usually has a temporary filename, and it will be renamed to original filename after fully downloaded. You can make your program to ignore or monitor only certain filename extensions.