Unable to resolve docker.host.internal with docker-compose on windows server 2019

Question

I need to connect to docker host from a linux container in my windows server machine. But docker dns is not able to resolve docker.host.internal. I have tried to add host.docker.internal in docker compose extra_hosts tag and have also tried to map my machine ip with this. But none of this is working.

extra_hosts:

• "host.docker.internal:host-gateway"

extra_hosts:

• "host.docker.internal:my_ip"

I have also tried to check the /etc/hosts file in my container and its is not there. Since the hosts file is not there so docker dns cannot resolve host.docker.internal.

docker exec my_container_id cat /etc/hosts
/etc/hosts: No such file or directory

My Environment:

• Host Machine: Windows Server 2019
• docker version: 20.10.10
• lcow version: v4.14.35-v0.3.9
• docker-compose version: v2.9.0

I have also tried to update my docker version to 20.10.11 which is working fine on my windows 10 machine.

PS C:\Windows\system32> docker version
 Client:
 Version:           20.10.11
 API version:       1.41
 Go version:        go1.16.10
 Git commit:        dea9396
 Built:             Thu Nov 18 00:42:51 2021
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.11
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.16.9
  Git commit:       847da18
  Built:            Thu Nov 18 00:38:11 2021
  OS/Arch:          windows/amd64
  Experimental:     true

I am unable to understand why docker is unable to create /etc/hosts file.

Answer 1

I have made this work by adding a manual entry in /etc/hosts file. Since I was not using docker desktop so docker deamon was not able to create /etc/hosts file inside my container. And due to this missing file my container was not able to connect to host.docker.internal. I made sure few things before doing this.

I checked if bridge network was created (from my host machine):

PS C:\Windows\system32> docker network ls
NETWORK ID     NAME                   DRIVER    SCOPE
9f3ee06eaa1f   nat                    nat       local
1b17a917b877   none                   null      local
56f23d8f2228   talflow_code-default   nat       local

Then I inspected the nat network which is the default bridge network created by docker deamon:

PS C:\Windows\system32> docker network inspect nat
[
    {
        "Name": "nat",
        "Id": "9f3ee06eaa1f3ec923e734b18228d08b2cda9242293a2b71fd82c7a413109609",
        "Created": "2022-08-11T12:18:01.4356971-07:00",
        "Scope": "local",
        "Driver": "nat",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "windows",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.31.80.0/20",
                    "Gateway": "172.31.80.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "com.docker.network.windowsshim.hnsid": "9C6F07F8-D106-4CD4-B3B7-C303886D4815",
            "com.docker.network.windowsshim.networkname": "nat"
        },
        "Labels": {}
    }
]

Here the bridge IP is 172.31.80.1

Next I verified if this IP is accessible inside my container. Note that I had to use ubuntu container to verify this. As containers only have basic commands so I pulled the latest ubuntu image from dockerhub and installed ping, ip and route commands on it to verify the connectivity of container with host.

Inside my ubuntu container I typed route command to check the IP table

root@8a7d33102d90:/# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         172.31.80.1     0.0.0.0         UG    1      0        0 eth0
172.31.80.0      *               255.255.0.0     U     0      0        0 eth0

In the IP route table I can see the bridge IP as default route. This entry verifies that my container can communicate with the docker bridge network. I can either use this IP to access any service on my host machine or my host machine IP (I used the host machine IP as this was static, in case of dynamic IP prefer to use bridge network IP or dns)

Now inside my ubuntu container I can ping bridge IP and my host machine IP both:

root@8a7d33102d90:/# ping 172.31.80.1
PING 172.31.80.1 (172.31.80.1) 56(84) bytes of data.
64 bytes from 172.31.80.1: icmp_seq=1 ttl=64 time=0.070 ms
64 bytes from 172.31.80.1: icmp_seq=2 ttl=64 time=0.201 ms
64 bytes from 172.31.80.1: icmp_seq=3 ttl=64 time=0.116 ms

root@8a7d33102d90:/# ping 10.25.241.37
PING 10.25.241.37 (10.25.241.37) 56(84) bytes of data.
64 bytes from 10.25.241.37: icmp_seq=1 ttl=64 time=0.070 ms
64 bytes from 10.25.241.37: icmp_seq=2 ttl=64 time=0.201 ms
64 bytes from 10.25.241.37: icmp_seq=3 ttl=64 time=0.116 ms

Note that 10.25.241.37 is the IPv4 address of my Ethernet0 adapter, you can check this by typing ipconfig command in powershell:

PS C:\Windows\system32> ipconfig

Windows IP Configuration


Ethernet adapter Ethernet0:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : 10.25.241.37
   Subnet Mask . . . . . . . . . . . : 255.255.255.240
   Default Gateway . . . . . . . . . : 10.25.241.33

Ethernet adapter vEthernet (nat):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::81af:fd91:a6cf:6158%15
   IPv4 Address. . . . . . . . . . . : 172.31.80.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . :

Ethernet adapter vEthernet (56f23d8f2228039):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::7453:b5f6:e090:b8a4%22
   IPv4 Address. . . . . . . . . . . : 172.20.0.1
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :

Once I verified that there was no issue with networking. I simply created the /etc/hosts file inside my actual container from where I wanted to access docker host.

PS C:\Windows\system32> docker exec -it a027f718c24e /bin/bash
root@a027f718c24e:/usr/local/airflow# echo '10.25.241.37 host.docker.internal' | tee -a /etc/hosts

After adding this entry I can access host.docker.internal from my container. Not that I added this entry only for testing purpose. As I had to add this inside the running container. So once the container is stopped this file will be gone and you will need to create that again which is not right. So I will instead use the staic IP (10.25.241.37) of my machine instead of host.docker.internal to make request to services hosted on host machine.