I have created Google Monitoring alert policy for Google monitoring SLO( 95% to total https requests for 7 days should have less than 5s latency and should receive alert when burn_rate consumes 2% of total 5% failure allowed in 24 hours.)
question- Not sure What should be exact threshold_value for alert policy. I have set it to 2.5 now.
source code-
resource "google_monitoring_custom_service" "lb-backend-latency" {
service_id = "lb-backend-latency"
project = var.project_id
display_name = "Load Balancer Backend Latency service for SLO"
}
resource "google_monitoring_slo" "lb_backend_latency_slo" {
project = var.project_id
service = google_monitoring_custom_service.lb-backend-latency.service_id
slo_id = "slo-number-of-requests-faster-than-5s"
display_name = "95 % of valid HTTPS requests served faster than 5s"
goal = 0.95
rolling_period_days = 7
request_based_sli {
distribution_cut {
distribution_filter = "metric.type=\"loadbalancing.googleapis.com/https/backend_latencies\" resource.type=\"https_lb_rule\" metric.label.\"response_code_class\"=\"200\" resource.label.\"backend_target_name\"=\"${var.backend_name}\""
range {
max = 5000
min = 0
}
}
}
user_labels = {
slo_label = "slo-1"
}
}
#monitoring alert policy
resource "google_monitoring_alert_policy" "lb_backend_latency_slo_alert" {
project = var.project_id
combiner = "OR"
display_name = "lb_backend_latency_slo_alert_policy"
notification_channels = [ google_monitoring_notification_channel.sl3_notification_channel.id]
conditions {
display_name = "Burn Rate condition for response total latency SLO"
condition_threshold {
filter = "select_slo_burn_rate(\"projects/${var.project_id}/services/${google_monitoring_custom_service.lb-backend-latency.service_id}/serviceLevelObjectives/${google_monitoring_slo.lb_backend_latency_slo.slo_id}\",\"3600s\")"
duration = "300s"
comparison = "COMPARISON_GT"
threshold_value = 2.5
}
}
}
tried to put threshold value = 2.5 value. but not sure how to calculate exact threshold value in case of this alert