Kong 1.1.0rc1 available for testing


#1

Hello Kong people!

We are happy to announce the first release candidate for our next release, Kong 1.1.0rc1!

It comes packed with new features, including declarative configuration, a new database-less in-memory mode, database bulk import, entity tagging, and more!

:package: Download Kong 1.1.0rc1 today — we are eager for feedback!

What’s new in Kong 1.1.0rc1 :fireworks:

  • Kong can now run without a database, using in-memory storage only:
    • New option in kong.conf: database=off to start Kong without a database
    • New option in kong.conf: declarative_config=kong.yml to load a YAML file using
      Kong’s new declarative config format
    • New command: kong config init to generate a template kong.yml file to get you started
    • New command: kong config parse kong.yml to verify the syntax of the kong.yml file before using it
    • New Admin API endpoint: /config to replace the configuration of Kong entities entirely,
      replacing it with the contents of a new declarative config file
  • Bulk database import, using the same declarative configuration format as the in-memory mode
    • New command: kong config db_import kong.yml to upsert all entities specified in the kong.yml file in bulk
  • Support for tags in entities:
    • Every core entity now adds a tags field
    • Admin API endpoints now support searching by tag (for example, /consumers?tags=example_tag)
    • You can search by multiple tags:
      • /services?tags=serv1,mobile to search for services matching tags serv1 and mobile
      • /services?tags=serv1/serv2 to search for services matching tags serv1 or serv2
    • New Admin API endpoint /tags/ for listing entities by tag: /tags/example_tag
  • Improvements for orchestration scenarios:
    • The Kubernetes Sidecar Injection plugin is now bundled into Kong for a smoother K8s experience
    • New option --wait in kong quit to ease graceful termination when using orchestration tools
  • Improvements for stream handling:
    • The stream_realip_module is now bundled
    • The stream subsystem now supports Nginx directive injections
    • New PDK functions: kong.client.get_protocol (returning “http”, “https”, “tcp” or “tls”)
      and kong.client.get_subsystem (returning “http” or “stream”)
  • Support for ACL authenticated groups
  • Various bugfixes

Quick start for using Kong without a database

  1. To do a test run of Kong using database-less mode, start Kong as usual, adding the KONG_DATABASE=off environment variable (you can also set database=off in kong.conf instead if you want to make the changes permanent):
$ export KONG_DATABASE=off
$ kong start -c kong.conf
  1. You can generate an example kong.yml file in the current directory:
$ kong config init -c kong.conf
  1. Edit the kong.yml and add entities there — the generated file has instructions.

  2. Now you can load your declarative config using the /config endpoint and all entities in that file will be populated in Kong’s memory:

$ http :8001/config config=@kong.yml

Upgrade Path :arrow_up:

You can upgrade from 0.14.1 or any release in the 1.0 series (Note: upgrading from a pre-0.14 cluster is not supported). You can use a Blue/Green deployment method to update: “Blue” means your current version (0.14.1 or 1.0), “Green” means the new version 1.1.0rc1:

  1. Download 1.1.0rc1, and configure it to point to the same datastore as your Blue cluster (running 0.14 or 1.0). Run kong migrations up.
  2. Both Blue and Green nodes can now run simultaneously on the same datastore. Start provisioning 1.1.0rc1 nodes, but do not use their Admin API yet. Prefer making Admin API requests to your Blue cluster instead.
  3. Gradually divert traffic away from your Blue nodes, and into your 1.1.0rc1 cluster. Monitor your traffic to make sure everything is going smoothly.
  4. When your traffic is fully migrated to the Green 1.1.0rc1 cluster, decommission your Blue nodes.
  5. From your 1.1.0rc1 cluster, run: kong migrations finish. From this point on, it will not be possible to start Blue nodes pointing to the same datastore anymore. Only run this command when you are confident that your migration was successful. From now on, you can safely make Admin API requests to your 1.1.0rc1 nodes.

Install 1.1.0rc1 on a fresh datastore :arrow_down:

Since version 1.0, Kong uses a new, improved migrations framework. The following commands can be run to prepare a new 1.1 cluster from a fresh datastore:

$ kong migrations bootstrap [-c kong.conf]
$ kong start [-c kong.conf]

What’s next :rocket:

Depeding on the community feedback, we are going to either tag and release 1.1.0 final or make a new release candidate 1.1.0rc2.

As always, consider yourself encouraged to reach out to us here or via GitHub issues if you have any questions or feedback about this release! Your feedback is essential to make Kong better!


#2

Working 2 days at 4 dev envs with new declarative configs. This is awesome!
Exactly what we were looking for.


#3

rc1 is working great on Heroku! Cheers :beers::smile:

The version 1.1.0 changes include switching the built-in secure Admin API feature to be loaded by kong config db_import instead of a hacky SQL technique. Looking forward to releasing this!


#4

Does the 1.1.0 still constitute a “minor” upgrade from 1.0.3, and not require db migrations? And if true then Kong 2.x.x will require db migrations later down the road for users on the 1.x.x versions right? I ask this because in the past 0.13.x to say 0.14.x meant a major upgrade and we would do db migrations, so now that we are in the x.x.x range it adds new confusion :smile: .

Also just for clarity for users that want to continue leveraging their databases I am guessing ENV variable wise or conf wise, all can stay the same with no change, meaning the new functionality allowing db-less config is not the defaulting in any way right?

Edit - Oh re-reading the top description it does sound like 1.1.x will require a migration from 1.0.x essentially keeping the standard the same as before where 0.13.x to 0.14.x meant a migration as well. For an ELI5 the middle number a changing means do a migration :laughing: .


#5

Yes, 1.0.0 to 1.1.0 does require a migration (specifically, for the new tagging feature) as mentioned in the above upgrade path.

However in the future, please do not assume whether a particular upgrade path needs migrations or not solely based on the version number. One of the goals of the new migrations framework is to allow us to introduce migrations in any release. The fact that we only introduced migrations in “major” release pre 1.0 was an old internal decision taken due to the lack of robustness of the previous migrations framework.

The new framework is still awaiting for a complete documentation section, but the gist of it is that one should take advantage of the kong migrations list command in order to determine whether migrations should be run or not. The -q option (quiet) along with the exit codes of that command allow for scripting the deployment workflow of a new Kong version:

  • download version x.y.z
  • test kong migrations list -q - does it require running migrations? (this command also considers plugins, so it can also be run when installing or upgrading plugins and determine whether those need migrations or not)
  • if so (exit code 5), then kong migrations up (runs with a cluster mutex, so no worries if several new nodes run this in parallel)
  • start new nodes
  • de-commission previous nodes
  • do the new nodes need to finish their 2 step migrations? kong migrations list -q again
  • if so (exit code 4), then kong migrations finish

The new migrations framework introduced in 1.0 allows for much smoother upgrades (with clear upgrade paths from our side, better scriptability, better re-entrancy, etc… read more in the PR https://github.com/Kong/kong/pull/3802), and in the future, we may introduce migrations in both major and minor releases, and we’ve considered doing so in patch releases as well if necessary (depending on the migration itself and the state of the framework by then, as it still needs some work and production mileage as of today).


Automating database migration (for a CD pipeline)
#6

Super insightful, getting all this goodness currently in your head into docs will be great :+1: . Also baking in the logic for how to best streamline the Kong migration deploy logic could be put in practice in the docker-kong repo for the docs to reference how to do it the right way stamped with Kong approval, probably just means bringing the shell scripts in there up to speed hah:


#7

Yes, we are planning on improving our Kong 1.0 documentation in the next few weeks! Nice suggestion, although I’m not sure if we want to force/automate migrations at the Docker image level… Feels like the wrong place. Running migrations (or even checking if they should be run) is still a user decision as of today, and the day we can include them in the Docker Image, means they probably should be in Kong itself.


#8

I wonder what the best strategy is to use the declarative configuration while still being able to update consumers on demand.

What happens if I combine the database with the config and I remove something from the latter (like a service or route), would that be reflected in the database?

Can you still create consumers and keys when the database is disabled? (Without persistence of course)

Or would it be a good idea to store all consumers in the config and just call the /config endpoint whenever one changed? What if there are 1k+ consumers?


#9

Docker running 1.1.0rc1 on start up
2019/03/21 16:52:29 [error] 26#0: init_by_lua error: /usr/local/share/lua/5.1/resty/dns/client.lua:748: no request
kong_1 | stack traceback:
kong_1 | [C]: in function ‘timer_at’
kong_1 | /usr/local/share/lua/5.1/resty/dns/client.lua:748: in function ‘asyncQuery’
kong_1 | /usr/local/share/lua/5.1/resty/dns/client.lua:779: in function ‘lookup’
kong_1 | /usr/local/share/lua/5.1/resty/dns/client.lua:1150: in function ‘resolve’
kong_1 | /usr/local/share/lua/5.1/resty/dns/client.lua:1424: in function ‘toip’
kong_1 | …share/lua/5.1/kong/db/strategies/cassandra/connector.lua:21: in function ‘new’
kong_1 | /usr/local/share/lua/5.1/kong/db/strategies/init.lua:30: in function ‘new’
kong_1 | /usr/local/share/lua/5.1/kong/db/init.lua:70: in function ‘new’
kong_1 | /usr/local/share/lua/5.1/kong/init.lua:332: in function ‘init’
kong_1 | init_by_lua:3: in main chunk
kong_1 | nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/resty/dns/client.lua:748: no request
kong_1 | stack traceback:
kong_1 | [C]: in function ‘timer_at’

Any ideas?


#10

@flowdopip could you try it with Kong 1.1.0rc2? We updated the resty.dns.client library bundled with it, it includes important fixes.


#11

can you provide the docker image for that version?
is just replicate the docker file from 1.1.0rc1 and replace version variable?


#12

It’s up for PR now ( https://github.com/docker-library/official-images/pull/5586 ) or you can build it yourself from here https://github.com/Kong/docker-kong/tree/1.1.0rc2


#13

Thanks, its working now based on docker tag 1.1.0rc2. But it´s very strange not working with the official image 1.1.0rc1


#14

Not that strange: there was a bug in rc1 which was fixed in rc2 (specifically, in https://github.com/Kong/kong/pull/4378) :smiley:

Cheers,