I’ve released decK, a tool to manage Kong’s configuration using a declarative config file.
I’ve had this project on my personal project todo list for quite a while and
I finally had some time over the holidays to get it polished and out for a public release.
Here are some feature highlights:
Export
Existing Kong configuration to a YAML configuration file This can be used to backup Kong’s configuration.
Import
Kong’s database can be populated using the exported or a hand written config file.
Diff and sync capabilities
deck can diff the configuration in the config file and the configuration in Kong’s DB and then sync it as well. This can be used to detect config drifts or manual interventions.
Reverse sync
deck supports a sync the other way as well, meaning if an entity is created in Kong and doesn’t add it to the config file, deck will detect the change.
Reset
This can be used to drops all entities in Kong’s DB and reset a Kong cluster.
Parallel operations
All Admin API calls to Kong are executed in parallel using threads to speed up the sync.
Neat project, I am specifically looking to give export and import a go and see how stable it is. Could be a great tool for before migrations just do a quick export and keep a copy on file and if migrations somehow went horribly wrong I could just restore to a totally fresh keyspace the same db entries using my exported record.
Your supported entities seems to be lacking Consumers(and credentials?)
Supported entities
Routes and services
Upstreams and targets
Certificates and SNIs
Plugins (Global, per route and per service)
Ohh I see its on the road-map , as soon as that is in place and it can do full backup/restores I will give it a whirl, keep us posted on your progress!
Ooo and RESET will be just a helpful option too in sandbox modes to clear out a kong node when you have been just loadtesting resource creations.
Consumers and credentials are a little tricky with a declarative configuration and expect to have some limitations:
With a lot of consumers (>10k+), the performance will slow down a little bit but for smaller scale clusters it should be possible.
Some credentials can actually never be exported, like basic-auth. Kong stores the hash with a salt and there is no way to actually export and then re-import from the Admin API.
Feel free to try out now for other entities as support consumers entity is being developed!
I’m curious if you considered configuring Kong declaratively using Terraform, which is also in Go? (The Terraform provider & its underlying API client). I ask because Terraform is amazing for configuring many separate technologies together as an integrated system, inherent to Kong’s gateway/routing role, and it would be great to have more momentum to those Go language projects (the client & provider are lagging behind Kong 1.0, now broken )
Having the community behind one of the options feels like a better option for everyone. The Terraform provider approach is where I would put my chips. I haven’t dug into the existing TF Kong provider but we use TF providers whenever possible - it simplifies the overall workflow. And, streamlines dependencies, CI/CD, etc.
I use Terraform in my day-to-day a lot and love it and have used the Kong provider in the past and I’d encourage users to use it if they already use it.
decK is suited more for teams which manage Kong outside the infrastructure layer (which is where Terraform is used the most).
decK is backed by an API client which is compatible with Kong 1.0 .
Consumers and credentials are a little tricky with a declarative configuration and expect to have some limitations:
Not sure if it makes any sense, but would it be a possible solution to keep the configuration sort of “hybrid” for large scale deploys? Consumers (and auth?) still db"full", and general configuration (services/routes/upstreams/plugins) db"less"? Just as an option, although it would be lot harder to maintain, I understand.
Mixing up dbless and db approach is not possible today but does make sense for cases with large number of consumers.
But the essence of what you’re thinking is that we should manage routing entities and consumers differently in Kong, which is surely a pattern that we’ve seen in the community.
You can use decK or Kong’s Ingress Controller to manage all the routing and plugins in a declarative fashion and then consumers and credentials are dynamic (assuming they are created via another API call).
You can still periodically export entire Kong’s configuration using decK for disaster recovery purposes.
, as soon as that is in place and it can do full backup/restores I will give it a whirl, keep us posted on your progress!
(the client & provider are lagging behind Kong 1.0, now broken 
)
.