Encountered Errors in Storing Images using Singularity in Cluster

Question

I am trying to get myself familiar with operations related to our cluster, so I tried using singularity there. I am having issues with pulling a specific docker image into singularity in the cluster. I have done the following steps:

cd /scratch/$USER
mkdir -p singularity/{cache,temp}
mkdir my_containers
module load singularity/3.7.2
export SINGULARITY_CACHEDIR=/scratch/$USER/singularity/cache
export SINGULARITY_TMPDIR=/scratch/$USER/singularity/temp
cd my_containers
singularity pull ./cms7poet.sif docker://cmsopendata/cmssw_7_6_7-slc6_amd64_gcc493

After the conversion of blob OCI to SIF, specifically saying

Writing manifest to image destination
Storing signatures
... warn xattr{mnt}
... warn xattr{lib}
... warn rootless{rootfs/}
...

it then throws a bunch of warnings and at the end of the day, returns the error

While making image from oci registry: error fetching image to cache: while building SIF from layers: packer failed to pack: while unpacking tmpfs: error unpacking rootfs: unpack layer: unpack entry: var/lib/yum/yumdb/m/d1e78f5adcafeb6c7d05cf71d17fb5196f487db5-mesa-libgbm-11.0.7-4.el6-x86_64/checksum_type: link: unpriv.link: unpriv.wrap target: too many links

I tried adding chmod -R 777 [directory] from the very beginning, but it did not work. I also tried googling first the issue, but never got a particular answer for it. Does anyone know about how to get around this kind of error? The available version of singularity that I am using is version 3.7.2, and the docker image is from CERN.

Answer 1

The error is probably caused by running the build command in home or scratch directory. Try to set the temporary directory to /tmp by using the option --tmpdir.

singularity pull --tmpdir=/tmp ./cms7poet.sif docker://cmsopendata/cmssw_7_6_7-slc6_amd64_gcc493

Check https://rcifw.github.io/software/singularity.html.