Changing Python /user/.cache folder

Question

When I install and run natural language processing modules from like huggingspace, they often download large machine learning modules into: C:\Users\User\.cache directories or similar directories on the C:\ drive in Windows.

My C: drive is a SSD with limited space, I keep on running out of space on it.

I would like to change the python .cache folder environment variable to a folder of my choice, as i use a development disk drive that is spacious and i got especially for this use case.

I could not find any way to change these directories, does anyone know how we can do this?

Answer 1

Cache directory

The default cache directory is ~/.cache/huggingface/datasets. Change the cache location by setting the shell environment variable, HF_DATASETS_CACHE to another directory:

$export HF_DATASETS_CACHE="/path/to/another/directory"

When you load a dataset, you also have the option to change where the data is cached. Change the cache_dir parameter to the path you want:

>>> from datasets import load_dataset
>>> dataset = load_dataset('LOADING_SCRIPT', cache_dir="PATH/TO/MY/CACHE/DIR")

Similarly, you can change where a metric is cached with the cache_dir parameter:

>>> from datasets import load_metrit
>>> metric = load_metric('glue', 'mrpc', cache_dir="MY/CACHE/DIRECTORY")

from: https://huggingface.co/docs/datasets/cache