Kong Ingress Controller and Network Policy

Kubernetes
1

Hello !

I’m using Kong Ingress Controller for multiple projects
So far, I did not encounter major problems with the Ingress Controller

I’m trying to secure it the maximum possible, and I’m stuck at NetworkPolicy objects at the moment.
It seems that Kong needs some flows opened to work correctly
Currently, I have opened everything from Ingress and Egress, but I would want to limit the Egress if possible
Do you have some example of NetworkPolicy on your end ?

To explain a but further : the Ingress Controller starts and it seems stuck
-------------------------------------------------------------------------------
Kong Ingress controller
Release: 0.7.0
Build: e4605db
Repository: git@github.com:kong/kubernetes-ingress-controller.git
Go: go1.13.1
-------------------------------------------------------------------------------

I0121 15:20:22.263329       1 main.go:407] Creating API client for https://some_ip:443
# He is stuck after this step. So ... what he is trying to do ?
I0121 15:20:22.284682       1 main.go:451] Running in Kubernetes Cluster version v1.14 (v1.14.9) - git (clean) commit 500f5aba80d71253cc01ac6a8622b8377f4a7ef9 - platform linux/amd64
I0121 15:20:22.578361       1 main.go:187] kong version: 1.4.3
I0121 15:20:22.578408       1 main.go:196] Kong datastore: off
I0121 15:20:22.745033       1 controller.go:224] starting Ingress controller

Thanks in advance :slight_smile:

2

It seems the controller can’t talk to the Kubernetes API-server. Please allow the pod to talk to the API-server.

Unfortunately, I don’t have any NetworkPolicies with me right now. I’ve helped folks with it but never got around to documenting it. It comes up often enough that we should document it. Can you share the policy once we get this working?

3

I’ve tried to add the api-server to the NetworkPolicy, and still no luck.
You will find 3 files:

I must be missing something, but I’m must be blind …

4
  1. Can you start with a dumb network policy that allows all ingress and egress policy to make sure that works?
  2. Once that works, can you try using a liberal egress policy to allow all egress to kube-system namespace?
5
  1. NetworkPolicy with Ingress from everything and Egress to everything => Works
  2. NetworkPolicy with Ingress from kube-system namespace and Egress to everything => Works
  3. NetworkPolicy with Ingress from kube-system and Egress to kube-system namespace => Fail
  4. NetworkPolicy with Ingress from everything and Egress to kube-system namespace => Fail

So … what Kong is trying to contact at startup ?

6

I’m assuming the failure is exactly the same as before, that is it gets stuck at starting Ingress controller log.

The next thing that happens at startup is leader election. That is it contacts the API-server to acquire a lock on a ConfigMap resource. This is done via client-go.

Can you try a pod label selector as well?
Also, make sure DNS is working correctly?

7

Using a pod selector => No more luck
For the DNS, it’s working on my end. It may fail at one answer from time to time but that should not be an issue.

8

Related to this question (it’s not clear to me how you solved it in the end), I’m now trying to implement the scenario on the ingress controller as recommended on the kong docs for the admin API:

  • Expose the admin API by modifying the startup of the ingress to allow 0.0.0.0:8444 ssl
  • Add a new service that points to this port of the ingress pod
  • Add an ingress for /admin that forwards to the admin API, protected with the ACL plugin. This allows to expose the API from the outside, only for admins with valid credentials

Now two questions remain to protect the endpoint internally:

  • I’m using a NetworkPolicy to restrict connectivity to the API to only allow the ingress itself, forbidding any other pods running internally. This seems to work, but I’m not sure if it’s the recommended approach? E.g.

    apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: kong-admin
  namespace: kong
spec:
  podSelector:
    matchLabels:
      app: ingress-kong
  ingress:  # deny all except itself
    - from:
        - podSelector:
            matchLabels:
              app: ingress-kong
      ports:
        - port: 8444
  policyTypes:
    - Ingress

  • Is there any way to forbid the creation of new ingress routes pointing to this new service? I’d like to avoid that another namespace uses the kong ingress controller to add a new route that could actually point to the internal API, and this would bypass the check since it’d be actually be triggered by the controller.

Thanks!