Kong routing requests across two namespaces

Hello team, need your advice…

We have deployed an application instance with Kong (using Helm chart) into two different namespaces (dev & test) within AKS.

Problem Description:

When accessing the controller, ingress is routing the traffic to svc:port, however balancing across both namespaces.

Here the log from kong ingress pod -

time=“2021-06-16T13:51:06Z” level=debug msg=“found endpoints: [{10.15.0.33 80}]” component=store service_name=my-svc service_namespace=test service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"
time=“2021-06-16T13:51:06Z” level=debug msg=“fetching endpoints” component=store service_name=my-svc service_namespace=dev service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"
time=“2021-06-16T13:51:06Z” level=debug msg=“found endpoints: [{10.15.0.85 80}]” component=store service_name=my-svc service_namespace=dev service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"
time=“2021-06-16T13:51:09Z” level=debug msg=“fetching endpoints” component=store service_name=my-svc service_namespace=dev service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"
time=“2021-06-16T13:51:09Z” level=debug msg=“found endpoints: [{10.15.0.85 80}]” component=store service_name=my-svc service_namespace=dev service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"
time=“2021-06-16T13:51:09Z” level=debug msg=“fetching endpoints” component=store service_name=my-svc service_namespace=cluster-test service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"
time=“2021-06-16T13:51:09Z” level=debug msg=“found endpoints: [{10.15.0.33 80}]” component=store service_name=my-svc service_namespace=test service_port="&ServicePort{Name:,Protocol:TCP,Port:8090,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,}"

Are you indeed seeing requests hit both instances of the Service, or are you only seeing them both listed in the controller logs?

Seeing both in the logs is normal. The controller’s mechanism for learning about Service topology in the cluster actually fetches all Services and their Endpoints initially, and then only later determines which it will route traffic to based on which Services are referred to by an Ingress. The logs you’re seeing are from the first, un-filtered phase.

We’re working on limiting this to only Services that the controller will eventually need to avoid this confusion and reduce log spam, but it’s normal for now.