Reset prometheus metric if it was not reported for period of time in python

Question

I have a python app with Gauge metric where I report latest value reported by some device. I want to create a mechanism that will reset the value to zero if the metric was not reported for more than 30 min. Is there a way to do it without keeping the map of last report times? Is there a way to get the last report time from the metric object itself?

Answer 1

As far as I can tell, Gauge in prometheus_client only uses timestamp for changes of value internally, and only is some of the multiprocess modes, without a way to extract it.

You can probably create your own wrapper class that will inherit Gauge, and will wrap method set (and all other methods that you need) with custom logic, that will track last changes and through callback return corrected value as you need.

---

On the other hand, you can rather easily achieve same behaviour in promql.
Query metric * (changes(metric [30m]) > bool 0) returns value of metric if it was changed in the last 30 minutes, and 0 otherwise.¹

Here, function changes is used to track number of changes in metric over specified range. And comparison operator > is used with modifier bool ir used to return result of 1 if expression is true, and 0 - otherwise.

¹ : This solution, being "implemented" fully on Prometheus' side can not differentiate between metric not being changed, or it being set to the same value over and over. Please take this into account when choosing a solution.