I've created custom metric filters from my microservices logs (deployed on AWS EKS) in AWS Cloudwatch console which set the value of these metrics to 1 if an ERROR log has been found. I want to use these metrics to create an alarm when a microservice starts producing ERROR logs.
I've successfully created one metric filter for each microservice (each metric has value 1 when a log with ERROR level is found) and now I want to use these custom metrics for creating alert. The problem is that the custom metrics are not visible and selectable inside the metrics namespace, so I'm not able to create any alert.
Only one of the custom metrics I've created is visibile and selectable inside the metrics namespace: this metric is the only that has matches after the creation (ERROR logs were produced after the custom metric filter creation).
Any suggestion about it? I've not found nothing yet about this issue. It seems strange to me that I've to force ERROR logs inside my microservices to be able to create alarms, I want to create them even if no match has occurred.