Filter metrics sent to Graphite/Prometheus from Spark

Question

I'm trying to monitor Spark. Spark pushes metrics to Graphite and Prometheus but it seems I can't choose the metrics it sends and there are so many.

I don't want to overload the Graphite and Prometheus instances.

Is there a way of filtering those metrics? In spark configuration or Graphite/Prometheus configuration?

Answer 1

This configuration will send only metrics that fit in the provided regex.

"spark.metrics.conf.*.sink.graphite.regex"="optional_regex_to_send_matching_metrics"

You can see it I the documentation.