I am very new to DVC and I encounter a few problems with remote storage. I stored my data into dvc remote storage here (.dvc/config file):
[core]
remote = dvc-remote
['remote "dvc-remote"']
url = /tmp/dvc-storage
Questions:
Where can I access it in my file explorer? Or is there any way to check for the content inside without
dvc pull?I first store a data name
datasetinto this remote storage, after I retrieve and deleted some pictures in thedatasetand push it back into the storage, is my originaldatasetoverwrite ordatasetfiles are kept?I only
dvc addthedatasetfile into dvc remote, why is it that on Iterative Studio the path of my other files changed to/tmp/dvc-storage/d5/df97ac43b0as well?
To recap, your DVC project's default remote found in a local directory (
/tmp/dvc-storage). OKAll your data files are in
/tmp/dvc-storageso that's where you could point your file explorer to, but this type* of DVC remote (local directory) is not meant for direct human handling. They're been renamed and reorganized in the same way as the project cache.Basically, the directory structure (let's call it space dimension) AND data versions (time dimension) are flattened into a content-addressable data store. This is why you see all those 2 letter directories containing long hex file names (similar to Git Object Storage).
By default nothing is deleted from the cache (or remote storage) during regular
dvcoperations. The data store is append-only for the most part. This way you cangit checkoutanddvc checkout(ordvc pull) the data for a previous project version (past Git repo commit).You'd have to specifically garbage collect certain data from cache or storage locations using
dvc gc, and even then it's designed to try preserving stuff you might need in the future.Note that
dvc adddoes not affect remote storage, it only works with the local cache. You need todvc pushanddvc pullto sync the data cache with a DVC remote.Wrt the Studio UI, I'm not sure where you see that path but its correct (as its hopefully clearer now). You'd get the same from
dvc get --show-url, so maybe reading that reference helps.