Refresh token for AWS ECR registry

Question

I have a helm chart used to deploy docker images. I configured secret into kubernetes using:

apt-get install docker

sudo apt-get install python python3-pip

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789.dkr.ecr.us-east-1.amazonaws.com

root@node1:~# cd .docker
root@node1:~/.docker# ls
cat config.json

kubectl create secret generic regcred \
--from-file=.dockerconfigjson=/root/.docker/config.json \
--type=kubernetes.io/dockerconfigjson

Into helm chart I placed:

imagePullSecrets:
  - name: regcred

When I deploy the helm chart it's working fine but after around 24 hours I get:

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  91s                default-scheduler  Successfully assigned default/mockup-6c5cfb4677-qztdv to node2
  Normal   BackOff    24s (x5 over 90s)  kubelet            Back-off pulling image "381491899314.dkr.ecr.us-east-1.amazonaws.com/mockup:0.0.1"
  Warning  Failed     24s (x5 over 90s)  kubelet            Error: ImagePullBackOff
  Normal   Pulling    11s (x4 over 91s)  kubelet            Pulling image "123456789.dkr.ecr.us-east-1.amazonaws.com/mockup:0.0.1"
  Warning  Failed     11s (x4 over 90s)  kubelet            Failed to pull image "123456789.dkr.ecr.us-east-1.amazonaws.com/mockup:0.0.1": failed to pull and unpack image "123456789.dkr.ecr.us-east-1.amazonaws.com/mockup:0.0.1": failed to resolve reference "123456789.dkr.ecr.us-east-1.amazonaws.com/mockup:0.0.1": unexpected status from HEAD request to https://123456789.dkr.ecr.us-east-1.amazonaws.com/v2/mockup/manifests/0.0.1: 403 Forbidden
  Warning  Failed     11s (x4 over 90s)  kubelet            Error: ErrImagePull
root@node1:~#

Looks like the token expires. How I can make a permanent token? Is there some solution to this problem?

EDIT: I managed to create a cronjob. Full steps below. I'm facing this issue:

kubectl create secret generic regcred \
--from-file=.dockerconfigjson=/root/.docker/config.json \
--type=kubernetes.io/dockerconfigjson


kubectl -n default create sa secrets-manager

touch /opt/clusterrole.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secrets-manager-role
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch", "create", "update", "delete"]


touch /opt/clusterrolebinding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: secrets-manager-binding
subjects:
- kind: ServiceAccount
  name: secrets-manager
  namespace: default
roleRef:
  kind: ClusterRole
  name: secrets-manager-role
  apiGroup: rbac.authorization.k8s.io


touch /opt/serviceaccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: secrets-manager
  namespace: default


kubectl apply -f /opt/clusterrole.yaml
kubectl apply -f /opt/clusterrolebinding.yaml
kubectl apply -f /opt/serviceaccount.yaml


kubectl create job --from=cronjob/refresh-aws-secret manual-001 -n default




# Create a file /opt/aws.yml and paste this content
#
# Apply using
#
# kubectl apply -f /opt/aws.yml
# kubectl -n default create sa secrets-manager

apiVersion: batch/v1
kind: CronJob
metadata:
  name: refresh-aws-secret
  namespace: default
spec:
  schedule: "* */6 * * *" # Run each 6 hours
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          serviceAccount: secrets-manager
          containers:
            - name: refresh-aws-secret
              image: bitnami/kubectl:latest
              imagePullPolicy: IfNotPresent
              env:
                - name: SECRET_NAME
                  value: regcred
              envFrom:
                - secretRef:
                    name: regcred
              command:
                - /bin/sh
                - -c
                - |-
                  kubectl delete secret --ignore-not-found ${SECRET_NAME}
                  kubectl create secret docker-registry ${SECRET_NAME} \
                  --docker-server=123456789.dkr.ecr.us-east-1.amazonaws.com \
                  --docker-username=AWS \
                  --docker-password=$(aws ecr get-login-password)


kubectl apply -f /opt/aws.yaml


kubectl create job --from=cronjob/refresh-aws-secret manual-001 -n default



root@node1:/opt# kubectl logs manual-001-n2x5g
secret "regcred" deleted
/bin/sh: 5: aws: not found
error: either --from-file or the combination of --docker-username, --docker-password and --docker-server is required
root@node1:/opt#

Answer 1

Updated

Documentation says that:

The authorization token is valid for 12 hours.

There is no option for extending token validity interval was provided for that command, so I assume this is not possible to do.

In order to keep token valid, I suggest to create cron job to run each 10-11 hours:

build docker image:

# create Dockerfile with this content
FROM alpine:latest
RUN apk --no-cache add aws-cli wget \
    && wget https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl \
    && mv kubectl /usr/local/bin/kubectl \
    && chmod +x /usr/local/bin/kubectl \
    && apk del wget

Build and publish publicly.

docker build -t myimage .
docker tag myimage:latest myregistry/myimage:1.0
# publish puplicly
docker push myregistry/myimage:1.0

Alternatively, you can look for already awailable images with aws-cli and kubectl awailable.

# Preparation for env and SA
kubectl create generic aws-credentials --from-env-file=<file path>
kubectl -n default create sa secrets-manager  

apiVersion: batch/v1
kind: CronJob
metadata:
  name: refresh-aws-secret
  namespace: default
spec:
  schedule: "* */11 * * *" # Run each 11 hours
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          serviceAccount: secrets-manager
          containers:
            - name: refresh-aws-secret
              image: myregistry/myimage:1.0
              imagePullPolicy: IfNotPresent
              env:
                - name: SECRET_NAME
                  value: aws
              envFrom:
                - secretRef:
                    name: aws-credentials
              command:
                - /bin/sh
                - -c
                - |-
                  kubectl delete secret --ignore-not-found ${SECRET_NAME}
                  kubectl create secret docker-registry ${SECRET_NAME} \
                  --docker-server=${AWS_ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com \
                  --docker-username=AWS \
                  --docker-password=$(aws ecr get-login-password)