Breaking down CloudWatch Logs costs to the Log Stream level

Question

I have a common log group which collects the logs of around 6 ECS services. I see there is a spike in billing of cloud watch for the current month and upon triaging found the common group used for storing ECS services logs is responsible for the spike which is having high putlog events.

I checked the possibility of getting the cost details of particular log stream inside log group but not able to get instead able to get cost details of whole log group. Is there any way to get the cost details of log streams of particular ECS service inside a common log group?

Answer 1

While you can tag log groups and those should work with cost allocation tags, you can't differentiate between the log streams that way as they don't support tagging.

CloudWatch Logs are mostly priced based on the amount of data you're ingesting/storing. You'd think that you could use the DescribeLogStreams API to figure out which log streams are especially large. Unfortunately the storedBytes in the response has been set to 0 since 2019.

We can get an approximation of the size of all Log streams using this Cloudwatch Logs Insights query. It sums up the length of all messages per stream (approximate_bytes) as well as the number of messages per stream (n_entries).

stats count(*) as n_entries
    , sum(strlen(@message)) as approximate_bytes by @logStream
| display @logStream, n_entries, approximate_bytes

This will exclude the size of the metadata (e.g. timestamp) that's stored per record, thus it's an approximate size but it could help you identify the culprit.