Kong plugins not working for traffic originating from same kubernetes cluster

We have a KIC setup in our kubernetes cluster. As mentioned in https://docs.konghq.com/hub/kong-inc/key-auth/#enable-the-plugin-on-a-service, I am trying to use key-auth plugins with our service.

Below setup works for for traffic through ingress e.g. curl https://demo-app.elample.com/health-check throws error {"message":"No API key found in request"}. But when I am trying to access the service endpoint from inside the cluster then the API-KEY error is not thrown. e.g. curl http://demo-app.demo-ns.svc.cluster.local:8080/health-check returns 200 ok.

I want the KEY-AUTH plugin should work for all kind of traffic, (external and internal). How is that possible?

apiVersion: v1
kind: Service
metadata:
  name: demo--service
  annotations:
    konghq.com/plugins: demo-auth
spec:
  selector:
    app: demo-app
  ports:
    - port: 8080
      targetPort: 8080
      protocol: TCP

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: demo-auth
  annotations:
    kubernetes.io/ingress.class: demo-kong
config:
  anonymous: null
  hide_credentials: false
  key_in_body: false
  key_names:
    - x-api-key
  run_on_preflight: true
plugin: key-auth

apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
  name: demo-user
  annotations:
    kubernetes.io/ingress.class: demo-kong
username: demo-user
credentials:
  - demo-user-api-key-secret

By sending request to demo-app.demo-ns.svc.cluster.local:8080/health-check directly you are bypassing the Ingress Controller.

You might need service mesh to restrict access between internal services, for example enable mTLS.

As fomm mentioned, requests to the Service’s svc.cluster.local address aren’t actually going through Kong, they’re just traversing the cluster’s internal network. Kong is primarily designed as an edge gateway, for traffic inbound into your cluster from outside. You can send internal traffic through Kong by sending it to the demo-app.elample.com hostname (or some internal hostname that resolves to the Kong proxy Service) instead, but it’s a bit atypical.

Our mesh proxy, Kuma, is designed for internal network policies, and uses sidecar proxies for each Service rather than a single gateway. You can restrict traffic between Services using a combination of the Traffic Permission and mTLS policies.