In our kubernetes deployment, we have a requirement to deploy the cluster behind azure application gateway. This means that I need to manually add a rule to AppGw for every ingress route. I would like the kubernetes ingress controller to have the ability to automatically add rules to azure application gateway. Maybe a custom annotation per ingress to instruct kong to add this route to application gateway
1 Like
Controllers in k8s are usually focused on one thing or one application, and this seems to be clubbing an Azure specific use case into Kong’s Ingress Controller.
Perhaps, you could use a templating system to automate the manual part of it?
Ingress controller has an IP. We have a setup where ingress controller(s) are surfuced out of Azure AKS Kubernetes using internal private load balancer. Like 10.240.123.123
As long as your Azure App Gateway is on the same virtual network, or you have your kubernetes network peered with your Gateway Virtual network you should be able to add your 1 or many ingress IP’s to Azure GW as a backend pool.
You pretty much need 1 IP only per ingress service, and let Kubernetes do the load balancing. We only use App Gateway for SSL offloading and WAF functionality. But if you have different requirements I do not see why you couldn’t float every Kong instance with it’s own private IP.
You will not be able to automate this very easily as you would need to have a service principal available inside the Kubernetes cluster with sufficient permissions to adjust App Gateway settings. Even if you got this to work I’d put a questionmark with the security concerns there. Also App Gateway updates usually are not instantaneous. On V1 gateways you can wait 1-20 minutes. V2 Gateways are a bit better but still in preview and not available in all Azure regions.
I assume you are using path based routing. Something to consider here is putting all your kong routes behind a common URL prefix like http://host.com/api. Then you can route /api to kong ingress controller backend pool. Then in the HTTP settings you can rewrite the URL. For backend upstreams after Kong you can rewrite the URL to remove the /api prefix.
I recommend host name routing over path based routing though but it depends what your current config looks like. http://api.host.com/ then could route to your Kong ingress. And you only would need the 1 single routing rule. Set it up only once.