Skip to main content

Troubleshooting - Back-off pulling image


The Trustnest Software Factory has recently changed the URL of the gitlab registry during the migration of the 28/29 august. This procedure helps you to patch your configuration.

The issue

Previously, the url of the gitlab registry was: So during the first deployment, most of you deployed a dockerconfigjson that point to this previous url.

And now, you should see in the kubernetes events:

$ kubectl get events -n prod -w
Back-off pulling image ""

As you can see, the cluster is not able to get the docker images from the registry, because the url has changed.

  • Previously:
  • Now:

Confirm the issue

First, look at the secret, and fine some dockerconfigjson

$ kubectl get secrets -A|grep dockerconfigjson
prod esaas-internal-documentation-docker-pull-secret 1 167d
prod esaas-public-documentation-docker-pull-secret 1 167d
prod internal-documentation-docker-pull-secret 1 203d
prod public-documentation-docker-pull-secret 1 203d
prod website-docker-pull-secret 1 138d

You can download a secret and check the url in the hash using the following command:

$ kubectl get secrets $SECRET_NAME -n $NAMESPACE -o=json |jq -r '.data | to_entries | .[].value'|base64 -d

You should the previous url

HOWTO fix it ?

K8Saas team provides a small script that allows you to patch easily your dockerconfigjson.

The script is available here as innersource

To patch all dockerconfigjson:

  • close the project
  • list your secrets with a dockerconfigjson format and its namespace



Once Done, you probably have to delete all the ImagePullBackOff pods