Managed Airflow (MWAA) not able to reach Elastic Load Balancer URL

83 Views Asked by At

I have brought up a MWAA env in AWS using the GUI, and have created a separate VPC for it using the "Recomended" settings using the CloudFormation stack. The Webserver is behind is inside a pubkic network so no extra configs there.

Essentially in the DAG I'd like to make a curl request to an endpoint (i.e. curl mycompany.ai) that points to an Elastic Load Balancer target group. This load balancer is in the default VPC, and is only accesible to the outside world (i.e. internet) by connecting to the site-to-site VPN setup in AWS.

To allow MWAA-VPC to communicate with the default VPC, I have created a peering connection, and modified the route tables to use the peering connection to reach the default VPC's CIDR block.

However, when the curl request is started, it gets stuck and nothing happens, until it eventually times out. To verify that the two VPCs can communicate with each other, I made a Lambda function and configured it to use the exact same VPC/Subnet/Security Group. And it works perfectly fine on Lambda!!

Any idea why this traffic works from the Lambda function but not the MWAA env? Thanks in advance!

0

There are 0 best solutions below