Hi,
I’am trying to deploy an old version of kong ingress controller (0.8.0) via helm chart but I run into the problem that the ingress controller is missing permissions.
Ingress container logs
W0413 06:31:21.521615 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0413 06:31:21.521943 1 main.go:442] Creating API client for https://10.245.0.1:443
I0413 06:31:21.534489 1 main.go:486] Running in Kubernetes Cluster version v1.20 (v1.20.15) - git (clean) commit 8f1e5bf0b9729a899b8df86249b56e2c74aebc55 - platform linux/amd64
I0413 06:31:21.554906 1 main.go:189] kong version: 2.6.0
I0413 06:31:21.555010 1 main.go:198] Kong datastore: off
E0413 06:31:21.580906 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:22.585289 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:23.590497 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:24.596900 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:25.601854 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:26.605591 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:27.609661 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:28.613140 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:29.618513 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:30.622440 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:31.626627 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:32.633030 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:33.636804 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:34.639811 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:35.644392 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:36.647068 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:37.650975 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:38.656188 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:39.659921 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:40.665819 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:41.670366 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:42.675694 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:43.679496 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:44.685144 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:45.704000 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:46.708121 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:47.711185 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:48.714917 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:49.718255 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
E0413 06:31:50.722434 1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.4/tools/cache/reflector.go:105: Failed to list *v1.KongCredential: kongcredentials.configuration.konghq.com is forbidden: User "system:serviceaccount:kong:kong-1648710167-kong" cannot list resource "kongcredentials" in API group "configuration.konghq.com" at the cluster scope
I’ve tried to set the permissions on the service account with the following yaml file:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kong-ingress-additional-permissions
rules:
- apiGroups:
- configuration.konghq.com
resources: ["KongCredentials"]
verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kong-1648710167-kong-additional-permissions
namespace: kong
subjects:
- kind: ServiceAccount
name: kong-1648710167-kong
namespace: kong
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kong-ingress-additional-permissions
kubectl describe pod
Name: kong-1648710167-kong-55795545b4-dpbgx
Namespace: kong
Priority: 0
Node: pool-test-do-ams3-01-uzoo7/10.133.93.230
Start Time: Thu, 31 Mar 2022 09:02:50 +0200
Labels: app=kong-1648710167-kong
app.kubernetes.io/component=app
app.kubernetes.io/instance=kong-1648710167
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=kong
app.kubernetes.io/version=2.7
helm.sh/chart=kong-2.7.0
pod-template-hash=55795545b4
version=2.7
Annotations: kuma.io/gateway: enabled
traffic.sidecar.istio.io/includeInboundPorts:
Status: Running
IP: 10.244.2.149
Controlled By: ReplicaSet/kong-1648710167-kong-55795545b4
Init Containers:
clear-stale-pid:
Container ID: containerd://e3c1b212002b2f87623e97ff649e30b36c49e2a3be5536fc72b46fa944169105
Image: kong:2.6
Image ID: docker.io/library/kong@sha256:5f819f70e609596a17c858ad6c932492f8aca8e6d0776926e854ba292a46b69a
Port: <none>
Host Port: <none>
Command:
rm
-vrf
$KONG_PREFIX/pids
State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 31 Mar 2022 09:02:51 +0200
Finished: Thu, 31 Mar 2022 09:02:51 +0200
Ready: True
Restart Count: 0
Environment:
KONG_ADMIN_ACCESS_LOG: /dev/stdout
KONG_ADMIN_ERROR_LOG: /dev/stderr
KONG_ADMIN_GUI_ACCESS_LOG: /dev/stdout
KONG_ADMIN_GUI_ERROR_LOG: /dev/stderr
KONG_ADMIN_LISTEN: 127.0.0.1:8444 http2 ssl
KONG_CLUSTER_LISTEN: off
KONG_DATABASE: off
KONG_KIC: on
KONG_LUA_PACKAGE_PATH: /opt/?.lua;/opt/?/init.lua;;
KONG_NGINX_WORKER_PROCESSES: 2
KONG_PLUGINS: bundled,ticketengine-auth
KONG_PORTAL_API_ACCESS_LOG: /dev/stdout
KONG_PORTAL_API_ERROR_LOG: /dev/stderr
KONG_PORT_MAPS: 80:8000, 443:8443
KONG_PREFIX: /kong_prefix/
KONG_PROXY_ACCESS_LOG: /dev/stdout
KONG_PROXY_ERROR_LOG: /dev/stderr
KONG_PROXY_LISTEN: 0.0.0.0:8000, 0.0.0.0:8443 http2 ssl
KONG_STATUS_LISTEN: 0.0.0.0:8100
KONG_STREAM_LISTEN: off
Mounts:
/kong_prefix/ from kong-1648710167-kong-prefix-dir (rw)
/opt/kong/plugins/ticketengine-auth from kong-plugin-ticketengine-auth (ro)
/tmp from kong-1648710167-kong-tmp (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kong-1648710167-kong-token-7f5dp (ro)
Containers:
ingress-controller:
Container ID: containerd://cc6d8e9e3d0fc1222e82a4a3562c5f0f82d7b1af481b14d2969b0e4cba381e53
Image: kong/kubernetes-ingress-controller:0.8.0
Image ID: docker.io/kong/kubernetes-ingress-controller@sha256:4bc984a937cefa1aba1cfe283d757e959350abd3143178b44e415dd0d40680ab
Port: <none>
Host Port: <none>
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 2
Started: Wed, 13 Apr 2022 08:31:21 +0200
Finished: Wed, 13 Apr 2022 08:31:51 +0200
Ready: False
Restart Count: 6154
Liveness: http-get http://:10254/healthz delay=5s timeout=5s period=10s #success=1 #failure=3
Readiness: http-get http://:10254/healthz delay=5s timeout=5s period=10s #success=1 #failure=3
Environment:
POD_NAME: kong-1648710167-kong-55795545b4-dpbgx (v1:metadata.name)
POD_NAMESPACE: kong (v1:metadata.namespace)
CONTROLLER_ELECTION_ID: kong-ingress-controller-leader-kong
CONTROLLER_INGRESS_CLASS: kong
CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY: true
CONTROLLER_KONG_ADMIN_URL: https://localhost:8444
CONTROLLER_PUBLISH_SERVICE: kong/kong-1648710167-kong-proxy
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kong-1648710167-kong-token-7f5dp (ro)
proxy:
Container ID: containerd://633a7f29596ce69bfd69702bfac064768daced4bb4b715084a921e681371c8e8
Image: kong:2.6
Image ID: docker.io/library/kong@sha256:5f819f70e609596a17c858ad6c932492f8aca8e6d0776926e854ba292a46b69a
Ports: 8000/TCP, 8443/TCP, 8100/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
State: Running
Started: Thu, 31 Mar 2022 09:02:53 +0200
Ready: True
Restart Count: 0
Liveness: http-get http://:status/status delay=5s timeout=5s period=10s #success=1 #failure=3
Readiness: http-get http://:status/status delay=5s timeout=5s period=10s #success=1 #failure=3
Environment:
KONG_ADMIN_ACCESS_LOG: /dev/stdout
KONG_ADMIN_ERROR_LOG: /dev/stderr
KONG_ADMIN_GUI_ACCESS_LOG: /dev/stdout
KONG_ADMIN_GUI_ERROR_LOG: /dev/stderr
KONG_ADMIN_LISTEN: 127.0.0.1:8444 http2 ssl
KONG_CLUSTER_LISTEN: off
KONG_DATABASE: off
KONG_KIC: on
KONG_LUA_PACKAGE_PATH: /opt/?.lua;/opt/?/init.lua;;
KONG_NGINX_WORKER_PROCESSES: 2
KONG_PLUGINS: bundled,ticketengine-auth
KONG_PORTAL_API_ACCESS_LOG: /dev/stdout
KONG_PORTAL_API_ERROR_LOG: /dev/stderr
KONG_PORT_MAPS: 80:8000, 443:8443
KONG_PREFIX: /kong_prefix/
KONG_PROXY_ACCESS_LOG: /dev/stdout
KONG_PROXY_ERROR_LOG: /dev/stderr
KONG_PROXY_LISTEN: 0.0.0.0:8000, 0.0.0.0:8443 http2 ssl
KONG_STATUS_LISTEN: 0.0.0.0:8100
KONG_STREAM_LISTEN: off
KONG_NGINX_DAEMON: off
Mounts:
/kong_prefix/ from kong-1648710167-kong-prefix-dir (rw)
/opt/kong/plugins/ticketengine-auth from kong-plugin-ticketengine-auth (ro)
/tmp from kong-1648710167-kong-tmp (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kong-1648710167-kong-token-7f5dp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kong-1648710167-kong-prefix-dir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kong-1648710167-kong-tmp:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kong-plugin-ticketengine-auth:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kong-plugin-ticketengine-auth
Optional: false
kong-1648710167-kong-token-7f5dp:
Type: Secret (a volume populated by a Secret)
SecretName: kong-1648710167-kong-token-7f5dp
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 19m (x13795 over 12d) kubelet, pool-test-do-ams3-01-uzoo7 Readiness probe failed: Get "http://10.244.2.149:10254/healthz": dial tcp 10.244.2.149:10254: connect: connection refused
Warning BackOff 4m41s (x75224 over 12d) kubelet, pool-test-do-ams3-01-uzoo7 Back-off restarting failed container
Does anyone have a suggestion to get the kong ingress up and running?
Thanks