MicroK8s does not authenticate with gitlab private registry

I have an instance of MicroK8s, I am currently migrating my applications. But I can't authenticate to Gitlab's private registry.

About my environment:

• MicroK8s
• Gitlab (with registry working securely)

Tests I performed

To test my private Gitlab registry I used another machine and did a docker pull registry.mydomain.com/my-group/my-image:my-tag.

In K8s I followed the instructions available on the K8s website and also the instructions in the answer to this question on SO.

I've already checked the namespaces, the secret was created correctly! I restored the credentials to confirm and they are correct.

Different ways I used to create the secret for imagePullSecrets.

First try

kubectl create secret docker-registry regcred \  --namespace=my-namespace \  --docker-server=registry.mydomain.com \  --docker-username="my-user" \  --docker-email="my-mail" \  --docker-password="my-gitlab-personal-access-token"

Second attempt

kubectl create secret generic regcred \ --namespace=my-namespace \ --from-file=.dockerconfigjson=/home/hsouza/.docker/config.json \ --type=kubernetes.io/dockerconfigjson

Both ways of creating my secret work, however when creating my pod the ImagePullBackOff error is presented.

For both attempts to generate the regcred I created the POD like this:

apiVersion: v1kind: Podmetadata:  namespace: my-namespace  name: test-private-register-podspec:  containers:  - name: test-private-register-pod    image: registry.mydomain.com/my-group/my-image:1.0.0  imagePullSecrets:  - name: regcred

When analyzing with kubectl describe pod/test-private-register-pod the following message is returned:

Name:             test-private-register-podNamespace:        my-namespacePriority:         0Service Account:  defaultNode:             k8s-dev/10.0.0.6Start Time:       Sat, 07 Oct 2023 11:25:23 -0300Labels:           <none>Annotations:      cni.projectcalico.org/containerID: 0d1a8c2d3f54ea9dfeccb69b96f930a6c5f40d6c9fa8c16994ac24676cecb5be                  cni.projectcalico.org/podIP: 10.1.252.205/32                  cni.projectcalico.org/podIPs: 10.1.252.205/32Status:           PendingIP:               10.1.252.205IPs:  IP:  10.1.252.205Containers:  test-private-register-pod:    Container ID:       Image:          registry.mydomain.com/my-group/my-image:1.0.0    Image ID:           Port:           <none>    Host Port:      <none>    State:          Waiting      Reason:       ImagePullBackOff    Ready:          False    Restart Count:  0    Environment:    <none>    Mounts:      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8gj47 (ro)Conditions:  Type              Status  Initialized       True   Ready             False   ContainersReady   False   PodScheduled      True Volumes:  kube-api-access-8gj47:    Type:                    Projected (a volume that contains injected data from multiple sources)    TokenExpirationSeconds:  3607    ConfigMapName:           kube-root-ca.crt    ConfigMapOptional:       <nil>    DownwardAPI:             trueQoS Class:                   BestEffortNode-Selectors:              <none>Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300sEvents:  Type     Reason     Age                From               Message  ----     ------     ----               ----               -------  Normal   Scheduled  16s                default-scheduler  Successfully assigned my-namespace/test-private-register-pod to k8s-dev  Normal   Pulling    15s                kubelet            Pulling image "registry.mydomain.com/my-group/my-image:1.0.0"  Warning  Failed     15s                kubelet            Failed to pull image "registry.mydomain.com/my-group/my-image:1.0.0": rpc error: code = Unknown desc = failed to pull and unpack image "registry.mydomain.com/my-group/my-image:1.0.0": failed to resolve reference "registry.mydomain.com/my-group/my-image:1.0.0": failed to authorize: failed to fetch oauth token: unexpected status: 403 Forbidden  Warning  Failed     15s                kubelet            Error: ErrImagePull  Normal   BackOff    14s (x2 over 15s)  kubelet            Back-off pulling image "registry.mydomain.com/my-group/my-image:1.0.0"  Warning  Failed     14s (x2 over 15s)  kubelet            Error: ImagePullBackOff

In the gitlab nginx logs, the status code for K8s access is 401. I put it in a fictitious way to try to pull from https://requestcatcher.com/ and when analyzing the request made by K8s it does not send the header with Basic Auth .

This is my problem, I appreciate the community's help in trying to identify where I'm going wrong.