Simultaneous uploading of a file to Azure File share using portal and downloading it using ShareClient libraries as WebJob

Question

I have a scheduled WebJob which is triggered every 5 minutes.

1. It checks if there are any files in Azure File Share using ShareClient libraries.
2. If files are available then the WebJob downloads the file and deletes the original file.
3. So when there is a large file being uploaded from portal to the File Share(>2GB its taking ~1 minute to upload and the file share is not mounted)
4. And simultaneously WebJob is triggered and starts running, the WebJob finds the file being uploaded and it is downloading part of the file content which is uploaded till that time.
5. The rest of the content is not downloaded.

How can I download the entire file in this scenario? Is it possible to download the file only once it is completely uploaded?

Sample code:

var shareItems = shareDirectory.GetFilesAndDirectories().Where(item => item.IsDirectory == false).ToList();
                foreach (var item in shareItems)
                {
                    ShareFileClient file = shareDirectory.GetFileClient(item.Name);
                    if (await file.ExistsAsync())
                    {
                        string filePath = Path.Combine(destPath, item.Name);
                        using (FileStream stream = File.OpenWrite(filePath))
                        {
                            try
                            {
                                ShareFileDownloadInfo download = await file.DownloadAsync();
                                download.Content.CopyTo(stream);
                            }
                            catch (Exception ex)
                            {
                                throw;
                            }
                            stream.Flush();
                            stream.Close();
                        }
                    }
                    //Delete the original file from file share 
                    file.Delete();
                }

Answer 1

Per my understanding, you want to make sure that only a big file has been uploaded completely that you will download it. If so, maybe you can just check the time gap between the file last modified timestamp and the current UTC timestamp.

If a file is being uploaded, its LastModified property will be changing all the time, per my testing, the gap between the current UTC timestamp is about 1-3 seconds.

So you can just try this:

var shareItems = shareDirectory.GetFilesAndDirectories().Where(item => item.IsDirectory == false).ToList();
            foreach (var item in shareItems)
            {
                ShareFileClient file = shareDirectory.GetFileClient(item.Name);
                if (file.ExistsAsync().GetAwaiter().GetResult())
                {

                    var fileLastModifiedTime = file.GetProperties().Value.LastModified;
                    var currentTime = DateTime.UtcNow;
                    
                    //Only download files that have finished uploading 5 minutes ago
                    if (fileLastModifiedTime.AddMinutes(5).CompareTo(currentTime) < 0) {

                        string filePath = Path.Combine(destPath, item.Name);
                        using (FileStream stream = File.OpenWrite(filePath))
                        {
                            try
                            {
                                ShareFileDownloadInfo download = file.DownloadAsync().GetAwaiter().GetResult();
                                download.Content.CopyTo(stream);
                            }
                            catch (Exception ex)
                            {
                                throw;
                            }
                            stream.Flush();
                            stream.Close();
                        }
                    }