conveyor failed while building an apptainer image from defintion file

Question

I am trying to run inference results from https://github.com/mlcommons/inference_results_v3.1/blob/main/closed/Intel/code/gptj-99/pytorch-cpu/docker/Dockerfile_int4 which uses docker however instead of docker I have `apptainer so I have used https://singularityhub.github.io/singularity-cli/ to convert the docker file to an apptainer defintion file and trying to build the image.

Now while building the image I am getting the below error.

INFO:    Adding environment to container
FATAL:   While performing build: conveyor failed to get: reading manifest latest in docker.io/library/dev-base: errors:
denied: requested access to the resource is denied
unauthorized: authentication required

To resolve the error above I tried deleting apptainer entries such as

Bootstrap: docker
From: dev-base 
Stage: conda

which is an corresponding entry in apptainer for docker entry FROM dev-base as conda. However deleting the entries throws another error.

Looking for correct way to parse docker entries which are defined in docker for stages and what would be the corresponding entry for singularity/apptainer.

Answer 1

When converting Dockerfiles to Apptainer (formerly, Singularity) definition files, it is important to understand that Docker's multi-stage build process does not have a direct equivalent in Apptainer.
However, you can still achieve a similar outcome by carefully managing the build steps within a single Apptainer definition file. The error you have could mean an issue with accessing the Docker Hub repository, which could be due to missing authentication or the repository not being public.

If dev-base is a public image, make sure it is correctly named and publicly accessible. If it is private, you will need to authenticate. Apptainer allows you to pull Docker images by authenticating with Docker Hub using the apptainer remote login command.

Since direct stage references (FROM AS ...) from Docker do not translate directly, you must manually manage the build stages. Typically, this involves consolidating the steps into a single build process within your Apptainer definition file, or building intermediate images separately and then copying necessary files from those images.
I.e:

• Pull or build the base image (dev-base) separately if it involves complex setups.
• Create an Apptainer definition file that starts from a base equivalent to your final Docker stage.
• Manually perform the steps you would have in the Dockerfile stages, making sure all required software and dependencies are installed.

As a simplified example for your Apptainer definition file, based on a hypothetical Docker multi-stage build:

Bootstrap: docker
From: ubuntu:20.04  # Assuming the final stage base image

%files
    # If needed, copy files from your host or other images

%post
    # Install dependencies as you would in the Dockerfile
    apt-get update && apt-get install -y software-properties-common
    # If you had a custom base image like dev-base, replicate its setup here
    
    # Follow with the setup from your Dockerfile's subsequent stages
    # Example: setup for your application

That would not directly translate Docker's multi-stage functionality but allows you to build a similar end-result by manually managing the steps.
If using custom base images not available on public registries, you may need to build and store them in a location accessible to Apptainer, such as a private registry with authentication or directly on the filesystem where Apptainer can access them.