I'm encountering issues with fetching the OpenAPI JSON from a FastAPI application running inside a Docker container as part of an AWS CodeBuild process. Despite the FastAPI application running successfully, attempts to fetch the OpenAPI schema result in connection errors.
Setup:
FastAPI application containerized with Docker.
AWS CodeBuild used for CI/CD.
Docker image built and pushed to Amazon ECR as part of the build process.
Need to fetch the OpenAPI JSON from the running FastAPI app, modify it, and use it in further steps (e.g., API Gateway updates).
Build Process:
Docker image is built and tagged.
Docker container is started with the FastAPI app running on port 8000.
Attempt to fetch OpenAPI JSON using curl from both inside the Docker container and the AWS CodeBuild environment.
The fetched schema is modified and used in subsequent steps (e.g., uploaded to S3, updating API Gateway).
Issues Encountered:
When attempting to fetch the OpenAPI JSON from the running FastAPI application using curl http://localhost:8000/openapi.json, I encounter curl: (56) Recv failure: Connection reset by peer.
Running curl inside the Docker container results in curl: (7) Failed to connect to localhost port 8000: Connection refused.
The FastAPI application logs show it starting successfully and listening on port 8000.
Here's a snippet of my buildspec.yml that outlines the key steps of the build process:
build:
commands:
- echo "Building the Docker image..."
- docker build --no-cache -t $IMAGE_REPO_NAME:$BRANCH .
- docker tag $IMAGE_REPO_NAME:$BRANCH $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$BRANCH
- echo "Starting Docker container to run FastAPI app..."
- docker run -d --name fastapi-app -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION -p 8000:8000 $IMAGE_REPO_NAME:$BRANCH
- sleep 30 # Give FastAPI time to start
- echo "TEST DEBUG"
- ls -l
- docker ps -a
- docker exec fastapi-app curl http://localhost:8000/openapi.json -o /app/openapi.json
- echo "Fetching OpenAPI schema from the running container..."
- python get_openapi_json.py
- ls -l
- echo "Running OpenAPI modification script..."
- python modify_openapi_script.py
- echo "Modified OpenAPI spec ready for upload."
- sleep 10 # Additional wait, if necessary
- echo "Stopping and removing the Docker container..."
- docker stop fastapi-app
- docker rm fastapi-app
Additional Attempts:
In an effort to work around the issues encountered with fetching the OpenAPI JSON directly using curl commands in the buildspec, I also created a Python script named get_openapi_json.py. This script is designed to programmatically make a request to http://localhost:8000/openapi.json to fetch the OpenAPI schema:
python
import requests
def fetch_and_save_openapi_schema(url: str, output_file: str):
response = requests.get(url)
response.raise_for_status() # Raises an error for bad responses
with open(output_file, "w") as file:
file.write(response.text)
if __name__ == "__main__":
openapi_url = "http://localhost:8000/openapi.json"
output_file = "openapi_schema.json"
fetch_and_save_openapi_schema(openapi_url, output_file)
Despite this script working as expected in a local environment (where it successfully fetches and saves the OpenAPI JSON), when executed within the AWS CodeBuild process (as outlined in the buildspec.yml), it fails to connect to the FastAPI application running inside the Docker container. The error returned is similar to what I encountered with curl: connection refused or reset by peer.
Seeking Solutions:
Given these persistent issues with both curl and the Python requests approach within AWS CodeBuild, I'm looking for any insights, alternative approaches, or configurations I might have overlooked that could resolve these connectivity issues. Specifically, how can I ensure successful communication with the FastAPI application to fetch the OpenAPI schema, or is there a more reliable method to achieve this within the context of AWS CodeBuild and Docker?
Additional Context:
Dockerfile and buildspec.yml are configured to expose and map port 8000.
AWS credentials are passed to the container environment to allow access to AWS services from within the FastAPI application.
Directly accessing the FastAPI application endpoints via browser works as expected when running the container locally.
Any insights or suggestions on how to resolve these issues would be greatly appreciated.