What hostname should I use for sending custom metrics to datadog's agent, inside k8s?

Question

I've got a k8s cluster where the datadog's helm is installed.

I want to send custom metrics to the datadog's agent, but I don't know what value to set for the dogstatsd client host.

These agents are actually a daemonset so I assumed localhost would work, but it doesn't.

Also, given the fact that it's UDP, it's very difficult to know if requests are failing to be delivered or not, so it's very difficult to trial and error it.

Any idea?

Answer 1

Generally any metric you send using DogStatsD or through a custom Agent Check is a custom metric.

There are multiple ways to send metrics to Datadog:

The easiest way to get your custom application metrics into Datadog is to send them to DogStatsD, a metrics aggregation service bundled with the Datadog Agent.

Set host and port to hostname/IP and port of the agent (if different from the default 127.0.0.1:8125)


statsd_host':'127.0.0.1

statsd_port':8125

Refer to this doc for more information about custom Metric Submission through DogStatsD

You can also use one of the Datadog official and community contributed API and DogStatsD client libraries to submit your custom metrics

Note: There are no enforced fixed rate limits on custom metric submission. If your default allotment is exceeded, you are billed according to Datadog’s billing policy for custom metrics.