Failed to deploy: Automatic rollback disabled for this deployment

677 Views Asked by At

I am trying to run Deploy multiple machine learning models for inference on AWS Lambda and Amazon EFS, I have done all the steps correctly, however, when I use the sam build --use-container command, I was falling into memory error.

In order to solve this problem I resizing EBS volume manually to 30 Gib and I used the sam build --debug --use-container command because I realized that cloud9 and my runtime had different versions of python.

After much research and trying I was able to build it successfully. But, when I went to deploy, using the command sam deploy --guided, I got the following:

CREATE_FAILED                         AWS::Lambda::Function                 MLModelUploadFunction                 Resource handler returned message:  
                                                                                                                  "'MemorySize' value failed to       
                                                                                                                  satisfy constraint: Member must     
                                                                                                                  have value less than or equal to    
                                                                                                                  3008 (Service: Lambda, Status Code: 
                                                                                                                  400, Request ID: 3503cc50-aa63-45ca 
                                                                                                                  -837a-e66085f8e215)" (RequestToken: 
                                                                                                                  c2277c5f-0fa3-cd7f-df01-6c7e19d1ad0 
                                                                                                                  a, HandlerErrorCode:                
                                                                                                                  InvalidRequest)                     
CREATE_FAILED                         AWS::EC2::NatGateway                  NatGatewayA                           Resource creation cancelled         
ROLLBACK_IN_PROGRESS                  AWS::CloudFormation::Stack            sam4-app                              The following resource(s) failed to 
                                                                                                                  create: [NatGatewayA,               
                                                                                                                  MLModelUploadFunction]. Rollback    
                                                                                                                  requested by user.                  
DELETE_IN_PROGRESS                    AWS::EC2::SubnetRouteTableAssociati   RouteTableAssociationAPrivate         -                                   
                                      on                                                                                                              
DELETE_IN_PROGRESS                    AWS::IAM::Role                        


.
.
.


           InternetGateway                       -                                   
DELETE_COMPLETE                       AWS::EC2::InternetGateway             InternetGateway                       -                                   
DELETE_COMPLETE                       AWS::EC2::VPC                         EfsLambdaVpc                          -                                   
ROLLBACK_COMPLETE                     AWS::CloudFormation::Stack            sam4-app                              -                                   
-----------------------------------------------------------------------------------------------------------------------------------------------------
Error: Failed to create/update the stack: sam4-app, Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "ROLLBACK_COMPLETE" at least once

Researching about this error, I noticed that it is a problem with the lambda function. When I access Applications in the AWS Lambda it shows a creation error.

I tried various solutions like directly specifying the s3 bucket, using another version of sam, I tried running the MLInference-Lambda-EFS code, but nothing works.

Does anyone know how I can solve this problem? I have been struggling with this project for a long time.

1

There are 1 best solutions below

1
Efidelity On

Please run sam delete --stack-name aws-sam-cli-managed-default to remove the broken managed stack before going through guided deploy or --resolve-s3 again.

I ran the deploy command again sam deploy --s3-bucket test-bucket --guided and it worked when I specified a bucket name. This bucket is ignored by SAM and the stack is created without problem.