How to set directory for hydra generated log file rather than the current working directory?

Question

I am using hydra version 1.2. I changed the default working directory of hydra to another directory. Now I want to save hydra generated log file in some other directory. How can I do that?

Currently, the log file is saved on the current working directory and I couldn't find a way to define a directory where I can save the log file.

Answer 1

You can configure the Hydra run dir by overriding it in a custom config file. See the documentation here: https://hydra.cc/docs/configure_hydra/workdir/ for more information.

Below is a simplified example of how I did it where name is something I plan to use in other places of my config.

name: benchmark
device: cuda:1
#------------------------- Hydra Config ------------------------#
override hydra/hydra_logging: colorlog
override hydra/job_logging: colorlog
hydra:
  job:
    name: ${name}
  run:
    dir: outputs/${hydra.job.name}/${now:%Y-%m-%d_%H-%M-%S}
  sweep:
    dir: ${hydra.job.name}
    subdir: ${hydra.job.num}

You can test if it's working by using the HydraConfig import like so:

from hydra.core.hydra_config import HydraConfig

@hydra.main()
def my_app(cfg: DictConfig) -> None:
    print(HydraConfig.get().runtime.output_dir)