spark batch prometheus last over time fill with previous

34 Views Asked by At

I have these four metrics. They come from four executions of same short living spark batch job lasting 5 minute executed once an hour. Each metric is a timeseries lasting for these ~5minutes.

So, it is the same metric name but labels differs. I want to ignore labels.

There are gaps between these metrics so they are not continous time series but rather 4 separate short lasting timeseries.

I would like to create single continous timeseries connecting them.

metrics_foobar_driver_AccumulatorSource_xxx_Value{container="spark-kubernetes-driver", type="gauges"}
metrics_foobar_driver_AccumulatorSource_xxx_Value{container="spark-kubernetes-driver", type="gauges"}
metrics_foobar_driver_AccumulatorSource_xxx_Value{container="spark-kubernetes-driver", type="gauges"}
metrics_foobar_driver_AccumulatorSource_xxx_Value{container="spark-kubernetes-driver", type="gauges"}

So I come up with this query but still it results in 4 data points.

metrics_foobar_driver_AccumulatorSource_xxx_Value and timestamp(metrics_foobar_driver_AccumulatorSource_xxx_Value{}) == on() group_left() max(timestamp(metrics_foobar_driver_AccumulatorSource_xxx_Value{}))

How to connect these 4 data points into continous time series filling gaps with last value ?

0

There are 0 best solutions below