I'm trying to set a custom statsd stat_name_handler for Airflow 2.6.3 (part of GCP Composer 2) to prevent the DAGs from failing with airflow.exceptions.InvalidStatsNameException - e.g. due to the stat_name exceeding a length of 250 characters.
I haven't found much documentation about this topic, but from the Airflow source it seems the function need to be referenced with a dotted path.
What I tried
I tried to follow the Airflow module management guide and to create the folder structure under the plugins folder - which should be added to the PYTHONPATH by Airflow - with the following configuration:
Folder structure:
plugins/
└── my_company/
├── __init__.py
└── custom_handlers.py
custom_handlers.py
def shorten_stat_name_handler(stat_name: str) -> str:
# my logic
return stat_name
configuration value: my_company.custom_handlers.shorten_stat_name_handler
Outcome
airflow-triggerer No module named 'my_company'
What is the correct way to pass a custom stat_name_handler?