I am setting up self-managed grafana in my windows server and running Tempo services. Open telemetry agent is running on my application server and sending the trace data to this local Tempo service.
Facing some issues when run the on-prem grafana & tempo.
Issue Getting the Gateway timeout error most of the time when explore the traces. Sometime the traces are visualized but not all the time.
Tempo.yaml
server:
http_listen_port: 3200
#log_level: debug
distributor: # receive spans and forward them to the appropriate ingesters.
receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318
#endpoint: localhost:4318 # No connection could be made because the target machine actively refused it.
grpc:
endpoint: 0.0.0.0:4317
compactor:
compaction:
block_retention: 720h # configure total trace retention here. 30days.
storage:
trace:
backend: local
wal:
path: C:\Program Files\GrafanaLabs\tempo\wal # where to store the head blocks while they are being appended to.
local:
path: C:\Program Files\GrafanaLabs\tempo\blocks # where to store the blocks.
Any help why getting inconsistent behavior? Thanks.