Creating alert for the aws cloudwatch logs

Question

I have a AWS cloudwatch log which have logstreams that contains logs like

{"levelname": "INFO", "asctime": "2024-01-27T05:09:56Z+0000", "name": "root", "processName": "MainProcess", "filename": "app.py", "funcName": "lambda_handler", "lineno": 85, "module": "app", "threadName": "MainThread", "message": "reset successful", "aws_request_id": null, "export_rs": true, "user_id": "abcd3928784798ahHBS", "type_of_event": "RESET_SUCCESSFUL", "event_ts": "2024-01-27 05:09:56", "timestamp": "2024-01-27T05:09:56Z+0000"}

Here I need to Create an alert when the user_id which has more than x times of type_of_event RESET_SUCCESSFUL occurances in 6hrs within the mentioned logroup, and the alert message should contain the user_id and the no of occurrences of type_of_event ..

I have tried using metric filters for cloudwatch log group, but it filters both user_id and type_of_event and I am not able to count user_id which has more occurrences of type_of_event.

Is there a way to achieve this, Can anyone help me here please?

Answer 1

As far as I know metric filters from logs will give "flat" numeric results, no grouping available. For your need custom metrics are the best fit! Here is how it can be implemented:

1. Change you app top publish custom metric with standard resolution.
2. Custom metric will send value of 1 as we will count for events.
3. Add dimensions to custom metric: UserId and TypeOfEvent.
4. Go to CloudWatch Alarms and click "Create alarm".
5. Select your custom metric and select metric using "query" where you can filter group timeseries using dimensions. More on metrics query language

This is examples of how to post custom metrics via cli: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html, by analogy should be no problem to find out how to do it via SDK.