We are running AKS clusters to provide access to a custom application for multiple customers. Resources and network traffic needs to be separated as much as possible for security reasons. Each customer’s copy of the application runs in its own namespace, has its own public IP, data store, etc. The application has a High Availability configuration and runs as a StatefulSet with 3 replicas.
This was all set up and working pretty well by a previous employee using Traefik. However we are now hitting limitations with that primarily due to Ingressclasses not available on namespace scope level · Issue #7729 · traefik/traefik · GitHub. So after some research it looks like Kong is the best option and I am working on translating the configuration accordingly. Unfortunately I’m having trouble with how to configure per namespace and using replicas, so have some queries. (BTW we are using Helm.)
-
The documentation states per-namespace configuration to use watchNamespaces, enable_controller_kongclusterplugin=false and enable_controller_ingress_class_networkingv1=false. However I’ve noticed the logs complain about Kong not having permission to access cluster ingress resources. Can this be ignored or is something being missed?
-
I have set number of replicas to 3. When installed I see 3 pods with two containers for KIC and Proxy in each. Is this expected, and preferred given the security and HA requirements? Or should the control plane and data plane be separated, or something else?
-
Kong can’t find a matching route for the application, although this was working when using 1 node and 1 replica. Do I need to set ingressClass to something unique per pod? Or how do I diagnose this?
I’m clearly out of my depth here so thank you for any help!