Exposing Azure Arc identity endpoint to docker container

Question

I have Azure Arc installed on all the nodes of our docker swarm. I want to use their managed identities to access Azure resources. Querying an access token from the host machine works fine, but I am struggling to get the same access from inside a container.

I tried extra_host, port mapping, etc, but I keep getting connection refused/failed to connect errors. The host network mode option would probably work, but is unavailable in docker swarm. I also can't find any documentation on changing the Azure Arc endpoint binding from 127.0.0.1 to a different network interface.

What I basically have at this point:

version: '3'
services:
  api:
    ports:
      - '5127:8080'
      - '40342:40342'
    environment:
      - 'IMDS_ENDPOINT=http://127.0.0.1:40342'
      - 'IDENTITY_ENDPOINT=http://127.0.0.1:40342/metadata/identity/oauth2/token'

How do give the docker container access to the identity endpoint, so that my ASP.NET server can authenticate itself in azure?

Answer 1

The Azure Arc-enabled servers expose the IMDS (Azure Instance Metadata Service) endpoint on the local host of the physical server. That endpoint is inaccessible from within a Docker container by default because the container is isolated from the host network.

Since you cannot directly alter the binding of the Azure Arc endpoint, you could try and run a proxy server on the host that forwards requests to the IMDS endpoint. Docker containers can then be configured to communicate with this proxy instead of directly with the IMDS endpoint.

events { }
http {
    server {
        listen 40345;
        
        location / {
            proxy_pass http://127.0.0.1:40342;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
        }
    }
}

You would then run nginx on the host, outside of Docker, but listening on a network interface that is accessible to Docker containers.

In your Docker Compose file, you need to set the environment variables to point to the host proxy instead of the localhost. You also need to mount /var/opt/azcmagent because ASP.NET will try to store tokens in that directory.

version: '3'
services:
  api:
    ports:
      - '5127:8080'
    environment:
      - 'IMDS_ENDPOINT=http://host.docker.internal:80'
      - 'IDENTITY_ENDPOINT=http://host.docker.internal:80/metadata/identity/oauth2/token'
    extra_hosts:
      - 'host.docker.internal:host-gateway'
    volumes:
      - '/var/opt/azcmagent:/var/opt/azcmagent'

With these changes, your ASP.NET server within the Docker container should be able to access the Azure Arc identity endpoint via the host proxy.