Boto3 - how to connect to S3 via proxy?

Question

I'm using a container that simulate a S3 server running on http://127.0.0.1:4569 (with no authorization or credentials needed)

and I'm trying to simply connect and print a list of all the bucket names using python and boto3

here's my docker-compose:

s3:
image: andrewgaul/s3proxy
environment:
  S3PROXY_AUTHORIZATION: none
hostname: s3
ports:
  - 4569:80
volumes:
  - ./data/s3:/data

here's my code:

s3 = boto3.resource('s3', endpoint_url='http://127.0.0.1:4569')

for bucket in s3.buckets.all():
    print(bucket.name)enter code here

here's the error message that I received:

botocore.exceptions.NoCredentialsError: Unable to locate credentials

I tried this solution => How do you use an HTTP/HTTPS proxy with boto3?

but still not working, I don't understand what I'm doing wrong

Answer 1

First, boto3 always try to handshake with S3 server with AWS API key. Even your simulation server don't need password, you still need to specify them either in your .aws/credentials or inside your program. e.g.

[default] 
aws_access_key_id = x
aws_secret_access_key = x

hardcoded dummy access key example

import boto3
session = boto3.session(
  aws_access_key_id = 'x', 
  aws_secret_access_key = 'x')

s3 = session.resource('s3', endpoint_url='http://127.0.0.1:4569')

Second, I don't know how reliable and what kind of protocol is implemented by your "s3 simulation container". To make life easier, I always suggest anyone that wants to simulate S3 load test or whatever to use fake-s3