Persistent error deploying Docker image to Scrapy Cloud

Question

I'm not sure what is causing the problem, but while attempting to deploy Docker image to Scrapy Cloud (using shub deploy) with my spider I'm getting following error:

$ shub deploy
Error:
Detected error connecting to Docker daemon's host.

Please ensure that you have Docker installed, configured and running locally,
that's essential for running shub image command. To check that run command

    docker version

and check its output: it should contain Docker client and server versions and
should not contain any errors. The minimum API Version is: 1.21

You can learn about Docker at https://www.docker.com/.

Despite the fact that Docker API version is correct and I don't see any issues when checking that:

$ docker version
Client: Docker Engine - Community
 Version:           25.0.1
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        29cf629
 Built:             Tue Jan 23 23:09:35 2024
 OS/Arch:           linux/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.1
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       71fa3ab
  Built:            Tue Jan 23 23:09:35 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.27
  GitCommit:        a1496014c916f9e62104b33d1bb5bd03b0858e59
 runc:
  Version:          1.1.11
  GitCommit:        v1.1.11-0-g4bccb38
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Is there something else I might be missing?

Answer 1

This is a bug in shub when using Docker server 25+. It should be fixed soon.

Workaround: set the DOCKER_API_VERSION environment variable to 1.24 or any other version that your Docker server supports.