AWSGlue AccessDeniedException, Status Code 400

Question

I am trying to build a data pipeline for a data engineering project With the help of S3, Glue, Athena, etc., I am stuck when setting up glue crawler for indexing over data. Even I set up the role according to the need, but still it's giving me the following error.

 {"service":"AWSGlue","statusCode":400,"errorCode":"AccessDeniedException","requestId":"7bd42729- 
 bc4b-4e22-af2a-553860002c64","errorMessage":"Account 834025784276 is denied 
 access.","type":"AwsServiceError"}

Answer 1

I know this is an old question, but wanted to provide a feedback on our experience with that issue, as we didn't find much on the internet.

So we started experiencing this issue at a time that seems kind of random to us, as we were using the service for 3 months, the account having been created more than 2 years ago.

We did reach out to AWS support, which conducted several steps in investigating that issue:

• Technical investigation for AWS Glue: all seemed fine
• AWS Fraud evaluation: Got a negative result (no fraud detected)
• AWS "containment score" evaluation & increase: solved the issue

Funny enough, we had an issue with another service (CodePipeline, which could not start a new CodeBuild anymore) which led to a similar solution and us finding this explanation of the containment score: https://towardsaws.com/containment-score-of-aws-3a893231e948

As a TLDR; in case link goes down:

Author had an issue with not being able to start a CodeBuild build saying "Cannot have more than 0 builds in queue for the account"

Support engineer mentioned the "containment score":

Based on the below table and score, the account has a concurrent limit is 0,

0 to 200 Concurrent limit is 0

201 and more (concurrent build limit is 1)

251 and more (concurrent build limit is 5)

301 and more (concurrent build limit is 150)

901 and more (concurrent build limit is 300)

The containment score needs to be increased to resolve this issue.

So it seems that AWS evaluates for each account and depending on their use a "containment score" allowing to set limits of use of services and maybe avoid scaling up crazy in case of a hack or something. I have no idea if this score is "per service" or "global" (probably "per service")

So if you encounter this, make sure to mention this to the support engineer as it could help resolving the issue. For us, the solution took ~5 days to be found, which is a crazy delay if you're thinking about a production environment blocking issue.

Hope this helps!

Answer 2

I had the same problem and I spent more than one day to troubleshoot, apply all the advice (endpoints, permissions, roles, lake formation troubleshooting), arriving at a point where, even if it would be working, I would not consider it as solution (working by accident is not a solution). I had the idea to change the region (from eu-west to central) and it works perfectly.