We are in the process of designing a native declarative configuration format to be used in Kong.
Declarative config — specifying your full configuration in a declarative way, rather than an imperative sequence of Admin API operations — has been a desire from the community for awhile. A number of features that we want to eventually have depend on having a way to specify a full Kong configuration: defining the declarative config format is the foundational step to get there.
Declarative config has been on our radar for a while, and our first stab at it was decK, an external declarative config tool. With the experience gained with this project, we’ve been working on the idea of a native declarative config format. We’d like to share with the you all our current work-in-progress design, in order to gather community feedback.
Kong Declarative Config Format (latest update 2019-02-20)
The format will be presented below in YAML, but it will be supported in JSON as well. At this point we do not have a formal specification, but we will present the intended feature set through a complete commented example:
# Metadata fields start with an underscore (_)
# Fields that do not start with an underscore represent Kong entities and attributes
# Matches Kong minimum version that supports the format
_format_version: "1.1"
# YAML supports comments, but JSON doesn't. Our format
# supports the "_comment" field in the top-level and within entities
_comment: "general comment about the file"
# The top-level and entities also support the "_ignore" field,
# which keeps an array of entries that are ignored by Kong.
# You can use it to store your additional metadata.
_ignore:
- my_own_data: 123
- foo: bar
kong_ignores: "all of this"
# Each Kong entity (core entity or custom entity introduced by a plugin)
# can be listed in the top-level as an array of objects:
services:
- name: my-service
url: http://mockbin.org
# Entities will be able to store tags as metadata
tags: [desktop, example]
# Entities that have a foreign-key relationship can be nested:
routes:
- name: example-route
# Each entity has its own, optional, set of tags
tags: [desktop, something_else]
hosts:
- example.test
# Plugins applied to example-route:
plugins:
- name: basic-auth
# Plugins applied to my-service:
plugins:
- name: key-auth
routes:
- name: my-route
# Relationships can also be specified between top-level entities,
# either by name or by id
service: my-service
hosts: ["hello.com"]
consumers:
- username: Frodo
# Custom entities from plugin can also be specified
# If they specify a foreign-key relationshp, they can also be nested
keyauth_credentials:
- key: my-key
plugins:
- name: rate-limiting
_comment: "these are default rate-limits for user Frodo"
config:
second: 5
hour: 10000
# When an entity has multiple foreign-key relationships
# (e.g. a plugin matching on both consumer and service)
# it must be specified as a top-level entity, and not through
# nesting.
plugins:
- name: rate-limiting
consumer: Frodo
service: my-service
_comment: "Frodo is extra limited when using my-service"
config:
hour: 100
# tags are for your organization only and have no meaning for Kong:
tags:
- extra_limits
- my_tag
We are trying to keep a good balance between functionality an simplicity. Please let us know your impressions!
Seems fine to me. As with no db I imagine certain plugins like OAuth2 with client credentials flow through Kong would be unsupported and only a local rate limiting policy applied and things of that nature but overall it provides a solid way to cut out the db.
Other aspects I think will be doing it in a manner that does not require an actual redeploy/restart and parse of a local yaml file sitting on the Kong node itself, but rather can each node point to a schema file hosted remotely at a central location possibly and do some sort of kong rebuild -f https://myrepo.com/mykongrepo/kongconfig.yaml at runtime with no downtime? And then just run that cmd on every declarative kong node we run to keep them in sync? Ofc a local file should be reasonable too if thats how a user wants to do it , but I would prefer one remote yaml file hosted elsewhere and say my 6 kong nodes all reference that one yaml.
Other random features floating in my head would be a way to maybe encrypt/decrypt the kong config file for some potential use case I haven’t fully vetted(share the file/keep it visible in public repo but not expose secrets stored in plaintext maybe? idk, maybe just add a layer of encryption on just the secrets in a way only Kong cluster “declarative” nodes would be able to decipher.).
If I have any other ideas i will drop them in place. I probably, won’t be leveraging this flow anytime soon due to being unable to support oauth2 cc by nature, but I do like the idea of it! Not being tied to a db and latency/connections held around that would provide a large amount of stability and not cut down too many features.
Curious what is the underlying plan for how this config is used by Kong:
Will it be read at start-up, and the config loaded into the backing datastore like API-based configuration? In other words, can this declarative config be mixed with API-based config, or are they exclusive to one another?
For those who use k8s ingress for declarative configuration already, would this proposal dovetail with that work at all?
There are plenty of situations where a user will want to test the same set of routes and services as they have in k8s in a local development environment. This RFC would allow them to create that local dev environment declaratively. However, they’ll end up with the same rules defined in k8s ingress definitions and in this RFC’s config format, won’t they. Is there a way we could avoid the duplication?
Yes, that’s the idea. For rate-limiting, one can also use the redis backend (effectively having a separate datastore just for that).
As for the usage workflow, in the first iteration we’re planning to provide an endpoint for (re-)uploading the configuration file, as well as a kong.conf option for specifying the file at startup (which can be overriden via env vars as usual).
Our initial plan is for this declarative file format to be used in both database-backend and DB-less scenarios:
when using a database, the file can be used for bulk import (essentially every item gets upserted on top of the current configuration)
when using db-less, the file is a truly declarative configuration: i.e. it represents the entire state of the world for Kong, so pushing a new one means the old configuration is discarded and replaced by the new one entirely.
We do have plans for that (and it’s helpful to hear that it’s a desired feature!) but the first release will probably be a single file.
We plan supporting both YAML and JSON; the conversion from one to the other is the one you expect (the _comment metadata field was added for the sake of JSON).
We will certainly look into this. We are constantly making steps to have the K8s experience for Kong to be as seamless as possible.
Sure. YAML provides syntax for that; this is a part of a generated config produced by our test suite testing mesh-related functionality using declarative config:
Note that in this first iteration we’re not tackling the issue of securing any secrets in the config file (the format accepts in principle any core or plugin entity and configuration); at this point the user’s workflow will have to take this into consideration.
Hi,
We are doing exactly the same thing using .net, and for us will be very useful if kong supports this by default.
One thing that we are creating on top of this is some kind of abstraction of KONG, to avoid that 1000 engineers need to know everything of Kong. The idea is a Resource concept with yaml blueprint that is transformed on kong model.
Are you thinking in create some kind on extension/abstraction to allow anyone to create their own model?
declarative configuration is definitely a great improvement towards infrastructure as a code. After some considerations we want to propose some further improvements or fine tuning:
support parsing without a running kong instance:
this would make it easier to build an automated delivery pipeline where the configuration is maintained in an e.g. git repository. A standard build server could perform a parse before a merge of a feature branch or before a deployment to kong without the need to be able to access the Admin API (which could for security reasons be forbidden for a build server which is used for checking the new configuration)
support for multiple configuration files:
this would enable to separate configuration of different services into different files, especially with larger configuration this can potentially increase the overview of the relevant parts for a service.
this would further allow to separate sensitive configurations for certain plugins, i.e. acl plugin.
personally I see two different approaches, either only a solution like textual import aka include or something like modules (based on services?) where the separation has semantics, too.
, but I would prefer one remote yaml file hosted elsewhere and say my 6 kong nodes all reference that one yaml.