Problem verifying content uploaded to Azure Blob

Question

Good morning.

Currently we have an aggregate with the volume practically occupied, a snapmirror has been configured to copy the data to an Azure storage blob, in order, once copied to be able to free up space.

The problem comes given that once launched the sync, completes the copy but with errors, without specifying the files/directories that may have failed.

We are not able to list the failed files so we can not proceed to free the space, as the file structure of the blob is different, we can not find a way to compare files in source and destination.

I would appreciate if someone can shed some light on this issue.

Thank you very much.

The launching of the copied files has been tried with a small volume of directories and they still fail.

Answer 1

The issue you are getting in verifying the content uploaded to an Azure Blob storage.

When using snapmirror to copy the data to Azure Blob storage, the process completes with errors but doesn't specify which files or directories have failed.

You are unable to list the failed files and compare them between the source and destination.

Steps to follow.

• Verify the logs generated during the sync process to see if they provide any specific error messages.

Below is the command to check the logs.

AzureMetrics  
| where ResourceProvider == "MICROSOFT.STORAGE"  
| where TimeGenerated >= datetime(2023-06-01) and TimeGenerated <= datetime(2023-06-16)

Azure Blob Storage Metrics

Enable metrics logging for your Azure Blob storage helps to identify the failed operations related to the copy process.

• Use AzCopy instead of using snapmirror, a command-line tool provided by Microsoft for copying data to and from Azure Blob storage.

• Check the account or service principal used for the copy process has the permissions to access both the source and destination storage locations.

• Breakdown the Sync into smaller subsets Instead of syncing the entire aggregate at once.

Command to sync the source and destination files.

azcopy sync "<source-path>" "<destination-path>" --recursive --mirror-mode --include-pattern "<pattern>"

For more information, check blog on SnapMirror.