Azcopy source and destination conflicts

Question

I am confused about how azcopy interprets the blobs

My source URL has files under "staging" "folder".

My destination is supposed to receive these files under "staging/outputtestfinal"

However, azcopy is copying the blobs under "staging/outputtestfinal/staging"

Why the second staging in my destination ?

I tried to *** https://myurlsource/staging/* but then the output is staging/outputtestfinal/staging/*

So a folder as "*" as created.. Really ? How I do a simple thing like this ?

azcopy copy "https://myurlsource/staging/" "https://myurldestin.blob.core.windows.net/staging/outputtestfinal/" --include-pattern '*.json' --recursive=true

Answer 1

When you use azcopy to copy from one blob storage location to another, it will try to retain the folder structure relative to the source URL provided. This is why you are observing that behavior.

If your source URL is https://myurlsource/staging/, and you have a blob in there named file1.json, and you are copying it to https://myurldestin.blob.core.windows.net/staging/outputtestfinal/, the destination path becomes https://myurldestin.blob.core.windows.net/staging/outputtestfinal/staging/file1.json.

To avoid this, you can try the following:

Use wildcard * without trailing slash:
This tells azcopy to consider only the blobs directly under the staging virtual directory but not the directory itself.

azcopy copy "https://myurlsource/staging/*" "https://myurldestin.blob.core.windows.net/staging/outputtestfinal/" --include-pattern '*.json' --recursive=true

Answer 2

So a folder as "*" as created.. Really ? How I do a simple thing like this

In my environment, my source container looks like you.

Portal:

To copy the destination you need to use a wildcard like * parameter to copy only files and it will exclude the folder.

From my side container is in private so I used SAS token.

Command:

azcopy copy "https://venkat123.blob.core.windows.net/test/staging/*?sas-token" "https://venkat123.blob.core.windows.net/test1/staging/outputtestfinal?sas-token" --include-pattern '*.json' --recursive=true

Output:

Destination:

Reference:

Transfer data to or from Azure Files by using AzCopy v10 | Microsoft Learn