Intermittent Dockerfile Issue: Problem executing scripts APT::Update::Post-Invoke

Question

This pipeline ran fine a week ago, but without any changes on my side to the dockerfile and the .gitlab-ci.yml, I started experiencing this issue.

I have a pipeline running through gitlab and its somewhat consistently ending in this:

$ docker build -t ${CI_REGISTRY}/${CI_PROJECT_PATH}:**REVOKED** -f **REVOKED**/Dockerfile .
Step 1/42 : FROM nginx
 ---> eb4a57159180
Step 2/42 : RUN echo "Current user: $(whoami)"
 ---> Using cache
 ---> 87ad120d9e8f
Step 3/42 : RUN apt-get update &&     apt-get -y install wget &&     apt-get -y install python3 &&     apt-get -y install git &&     apt-get -y install ca-certificates &&     apt-get -y install procps &&     apt-get -y install cron &&     apt-get -y install python3-pip &&     apt-get -y install logrotate
 ---> Running in fa2f3712df3d
Get:1 http://deb.debian.org/debian bookworm InRelease [147 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8904 kB]
Get:5 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [30.4 kB]
Fetched 9182 kB in 2s (5286 kB/s)
Reading package lists...
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code
The command '/bin/sh -c apt-get update &&     apt-get -y install wget &&     apt-get -y install python3 &&     apt-get -y install git &&     apt-get -y install ca-certificates &&     apt-get -y install procps &&     apt-get -y install cron &&     apt-get -y install python3-pip &&     apt-get -y install logrotate' returned a non-zero code: 100
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 100

It is preventing my pipeline from completing consistently and with it being sent off to three separate bases, it's stopping the entire deployment of the service.

I've tried adding lines to the beginning of the file by adding RUN apt-get update -o APT::Cache::Start=25165824 or RUN echo 'APT::Update::Post-Invoke {"true";};' > /etc/apt/apt.conf.d/99no-cache-clean but I'm unable to consistently prevent this error.

The start of this dockerfile is

FROM nginx

RUN echo "Current user: $(whoami)"

RUN apt-get update && \
    # apt-get upgrade -y && \
    apt-get -y install wget && \
    apt-get -y install python3 && \
    apt-get -y install git && \
    apt-get -y install ca-certificates && \
    apt-get -y install procps && \
    apt-get -y install cron && \
    apt-get -y install python3-pip && \
    apt-get -y install logrotate

RUN ln -s /usr/bin/python3 /usr/bin/python
RUN wget https://staticdownloads.site24x7.com/server/Site24x7InstallScript.sh
RUN bash Site24x7InstallScript.sh -i -key=**REVOKED** -automation=true -gn=toronto
RUN mkdir -p /opt/site24x7/monagent/plugins/nginx/
RUN wget -P /opt/site24x7/monagent/plugins/nginx/ https://raw.githubusercontent.com/site24x7/plugins/master/nginx/nginx.py
RUN chmod +x /opt/site24x7/monagent/plugins/nginx/nginx.py
ADD confs/site24x7/nginx.cfg /opt/site24x7/monagent/plugins/nginx/

Answer 1

This is most likely a Docker version issue. I'm unfamiliar with gitlab-ci but there should be a way to run an install script as part of the process, in which case as part of your image build add:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce

This will install the latest version of docker which should work.