AWS Glue metrics to populate Job name, job Status, Start time, End time and Elapsed time

641 Views Asked by At

I tried various metrics options using glue.driver.* but there is no clear way to get Job name, job Status, Start time, End time and Elapsed time in Cloudwatch metrics. This info is already available under Job Runs history but no way to get this on Metrics.

I found few solutions where this can be achieved using Lambda function but there should be an easy way.

Please share ideas. thanks.

1

There are 1 best solutions below

2
Satheesh V On

We had the same issue. In order to track glue job runs we ended up writing a small shell script which transformed the JSON Output of -> https://docs.aws.amazon.com/cli/latest/reference/glue/list-jobs.html to CSV. The final Output resembled the following :

enter image description here