Context:
I got a K8s cluster where I setup AWX using the official Helm install. On this cluster, there I also setup a Sonatype Nexus repository with a docker(hosted) private registry configured in order to pull custom AWX EE image I built.
At the moment, the private registry works only with a HTTP listener configured on port 8080.I'm able to login and push/pull my custom image to this registry from another server using docker cli.
Configuration/AWX Setup for image pulling:
In order to pull my custom image into my AWX project, I configure AWX the following way :
And finally setup Execution Environment on my templates.
Problem:
When i launch a template, it keeps running for a long time, and here's error i got from kubectl :
PS > kubectl describe pods automation-job-50-78mp5Name: automation-job-50-78mp5Namespace: awxPriority: 0Service Account: defaultNode: XXXXXXStart Time: Tue, 22 Aug 2023 11:42:04 +0200Labels: ansible-awx=4b53c506-cd99-41df-a4d5-1ed0395e8906 ansible-awx-job-id=50Annotations: cni.projectcalico.org/containerID: 66b9d3f6852b82b5e1d3b37ad8fc199667170c207f5f79af2c2e5d3a67007d9e cni.projectcalico.org/podIP: 172.20.5.208/32 cni.projectcalico.org/podIPs: 172.20.5.208/32Status: PendingIP: 172.20.5.208IPs: IP: 172.20.5.208Containers: worker: Container ID: Image: awx-repo.mydomain.com/awx-repo Image ID: Port: <none> Host Port: <none> Args: ansible-runner worker --private-data-dir=/runner State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Requests: cpu: 250m memory: 100Mi Environment: <none> Mounts: <none>Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled TrueVolumes: <none>QoS Class: BurstableNode-Selectors: <none>Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300sEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 20s default-scheduler Successfully assigned awx/automation-job-50-78mp5 to frhd01cor003-ebeb10-worker-3 Normal BackOff 19s (x2 over 20s) kubelet Back-off pulling image "awx-repo.mydomain.com/awx-repo" Warning Failed 19s (x2 over 20s) kubelet Error: ImagePullBackOff Normal Pulling 5s (x2 over 20s) kubelet Pulling image "awx-repo.mydomain.com/awx-repo" Warning Failed 5s (x2 over 20s) kubelet Failed to pull image "awx-repo.mydomain.com/awx-repo": rpc error: code = Unknown desc = failed to pull and unpack image "awx-repo.mydomain.com/awx-repo:latest": failed to resolve reference "awx-repo.mydomain.com/awx-repo:latest": failed to do request: Head "https://awx-repo.mydomain.com/v2/awx-repo/manifests/latest": x509: certificate signed by unknown authority Warning Failed 5s (x2 over 20s) kubelet Error: ErrImagePull
My question are :
- How can I double check the pod is using credentials I set from AWX GUI ?
- As credentials works from the server, is there a "dedicated format" we have to set from AWX GUI ?
- Does the certificate x509 warning may impact the pull action ? (even if my repo seems well configured with a wildcard certificate"