What exactly are the disadvantages of using the DB-less version of Kong (if there are any) on K8s compared to the DB one? Total redeployment of Kong on config changes is one which I heard of (Not sure about its authenticity though)
There isn’t really much in terms of K8S-specific negatives for DB-less mode (standard DB-less caveats always apply). DB-less is considered a positive in general for K8S deployments because most K8s deployments do use the ingress controller and use Kubernetes resources as the store of truth for configuration, so they don’t need to store config in a separate Kong-specific persistent database.
It sounds like you’re maybe halfway between a controller-managed and manually-managed deployment, that you’re transitioning traditional configuration into configuration stored in K8S YAML resources. If you do move your configuration into K8S resources, the controller will keep Kong’s configuration updated without a redeploy: it will continually send the latest config to the admin API /config endpoint while Kong is running, and you do not need to redeploy to recognize changes.
You do need to redeploy Kong to recognize DB-less configuration changes if you’re building the DB-less config file manually (without the ingress controller) at present to work around how DB-less configuration loads work in Kubernetes, but this isn’t a hard requirement–new functionality would allow less disruptive updates. Right now, DB-less can receive configuration via the API /config endpoint (what the controller uses) or by loading config from a filesystem location. The latter version currently only loads configuration at startup. A K8S ConfigMap mount with a manually written configuration will update that filesystem location when modified, but Kong doesn’t watch that file for changes currently. As such, the only way to recognize updates is to perform a rolling restart of the Deployment to do another initial startup load. You could reasonably update Kong’s code to instead watch that file location and, on observation of a change, load the updated configuration without a restart. This functionality doesn’t exist yet, but it’d be a reasonable feature to add if you wanted to make an OSS contribution.
If we use the DB-less version of Kong on K8s, exactly which manifest do we need to make changes in for our custom configs of Kong? Is it recommended to store this in Github? I am facing difficulties with configuring all our custom plugins and configurations on K8s.
What exactly do you mean by KongConsumer in the previous answer?
Enabling DB-less itself requires changes to the Kong container template env block to set KONG_DATABASE=off and other associated settings. Routing/plugin/etc. configuration is then either handled by providing a manually-written configuration (uncommon and unwieldy due to the functionality gap covered above) or using the ingress controller. When you use the controller, we provide K8S custom resource definitions that allow you to create K8S resources that the controller translates into Kong configuration. KongConsumer is one such custom resource: you create a KongConsumer K8S resource and the controller generates a Kong configuration with a matching consumer entity.
What are all the resources required on K8s to manage Kong?
The best answer we can provide is essentially “it’s complicated; we’re working to make it as efficient as we can, but it always depends on the specifics of your environment.” Linear increase in ingress-controller memory with increasing number of ingress routes is an example of a previous dive into the performance characteristics of the controller. General guidance here is to monitor it and, if you see something that looks like a problem, bring it to our attention; we’ll work to see if it’s something we can optimize.
What exactly needs to be done to enable custom plugins written by myself. First, I am adding this to my custom Kong image? What else needs to be done?
luarocks installing a custom plugin into your image (or otherwise adding code to the image is the correct first step). You’ll then need to add them to the loaded plugins list and possibly update the Lua search paths for more complex plugins.