Logging in Dask

Question

I am using a SLURM cluster and want to be able to be able to add custom logs inside my task that should appear in the logs on the dashboard when inspecting a particular worker.

Alternatively I would like to be able to extract the name of the worker to use the log_event function and include the name of the worker in such a log in a way that matches the name on the dashboard.

The reason is so that I can see the logs of any long running workers that seem to be hanging or having issues.

Answer 1

Maybe you can get the default logger of dask ditributed by

import logging
logger=logging.getLogger("distributed.worker")

Then output what you want to logger like logger.info(...) or logger.debug(...).