I written a python code to create metrics and send them to telegraf and create graphs in grafana. I use prometheus + telegraf + grafana stack. the problem is that there is no error in my python code and every thing seems to be ok but there are no metric saved in telegraf.
I implemented in docker
docker-compose:
version: '3.8'
volumes:
prometheus_data: {}
grafana_data: {}
networks:
grafana:
name: grafana
services:
prometheus:
hostname: prometheus
container_name: prometheus
image: prom/prometheus
restart: always
volumes:
- ./prometheus:/etc/prometheus/
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
- '--web.console.templates=/usr/share/prometheus/consoles'
ports:
- 9090:9090
networks:
- grafana
grafana:
hostname: grafana/grafana:10.0.1
container_name: grafana
image: grafana/grafana
user: '472'
restart: always
environment:
GF_INSTALL_PLUGINS: 'grafana-clock-panel,grafana-simple-json-datasource'
volumes:
- grafana_data:/var/lib/grafana
- ./grafana/provisioning/:/etc/grafana/provisioning/
env_file:
- ./grafana/config.monitoring
ports:
- 3000:3000
depends_on:
- prometheus
networks:
- grafana
telegraf:
hostname: telegraf
container_name: telegraf
image: telegraf:1.28-alpine
restart: always
ports:
- 9100:9100
- 8125:8125/udp
volumes:
- ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf
networks:
- grafana
my telegraf.conf:
# Telegraf Configuration
[agent]
interval = "10s"
round_interval = true
metric_buffer_limit = 10000
flush_buffer_when_full = true
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = ""
debug = true
quiet = false
hostname = ""
omit_hostname = false
[[outputs.prometheus_client]]
listen = "telegraf:9100"
[[inputs.statsd]]
name_prefix = "local_statsd_"
protocol = "udp"
service_address = ":8125"
metric_separator = "_"
[[inputs.cpu]]
percpu = true
totalcpu = true
fielddrop = ["time_*"]
[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs"]
[[inputs.diskio]]
[[inputs.kernel]]
[[inputs.mem]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]
[[inputs.net]]
# Specify specific interfaces if needed, e.g., interfaces = ["eth0"]
[[inputs.netstat]]
[[inputs.interrupts]]
[[inputs.linux_sysctl_fs]]
my python code ('docker' is my docker machine):
class StatDMetricProvider(BaseMetricProvider):
def __init__(self, host='docker', port=8125, *args, **kwargs):
super().__init__(*args, **kwargs)
self.statsd = statsd.StatsClient(host, port, prefix="local_statsd_")
def error_counter_metric(self, method, status_code, hot_wallet):
metric_name = f'response_error_exceptions.method:{method}.status_code:{status_code}'
self.statsd.incr(metric_name)