Kong 1.0 (rc4) available for testing

Hello everyone,

Continuing our release candidate cycle for Kong 1.0, we’re happy to announce the availability of RC4!

:package: Download Kong 1.0.0rc4 today and give us your feedback!

This fourth release candidate contains fixes for Service Mesh that were detected after releasing RC3. Thanks again to everyone who reported issues - they helped a lot!

RC4 Contains the Following Improvements over RC3:

Removal of deprecated features

As one of the final steps in preparing 1.0.0 final, this release removes from the codebase features and concepts that were marked as deprecated earlier in the 0.x cycle. In this release:

  • the API entity and related concepts such as the /apis endpoint, are removed (deprecated since 0.13.0, March 2018). Use Routes and Services instead.
  • the custom_plugins directive is removed (deprecated since 0.14.0, July 2018). Use plugins instead.
  • the galileo plugin is removed (deprecated since 0.13.0)
  • the kong.tools.ip module was removed (deprecated since the introduction of the PDK in 0.14.0). Use kong.ip from the PDK instead.
  • the kong.tools.public module was removed (deprecated since the introduction of the PDK in 0.14.0). Use the PDK instead.
  • the kong.api.crud_helpers module was removed (deprecated since the introduction of the new DAO in 0.13.0). Use kong.api.endpoints instead if you need to customize the auto-generated endpoints.
  • finally, the old DAO implementation is removed, along with the old schema validation library (apis was the last entity using it). Use the new schema format instead in custom plugins (full documentation is on the way). To ease the transition of plugins, the plugin loader in 1.0 includes a best-effort schema auto-translator, which should be sufficient for many plugins (in 1.0.0rc1, our bundled plugins used the auto-translator; since 1.0.0rc2 they now use the new format).

We plan to make a 0.15.0 release alongside 1.0.0, which will still include these deprecated features but will be otherwise identical to 1.0.0, to ease the migration path. If you have a custom plugin using any of the featues listed above, now is a great time to update them! All plugins bundled with Kong 1.0.0rc4 have been converted to remove dependency on these features, so they serve as good examples, along with the PDK documentation.

Service Mesh and Stream Routing

  • TLSv1.3 added to the template protocols as a workaround for openssl#7660
  • Fixed an issue where lua_package_path was not properly defined for stream requests
  • Fixed an issue where active healthchecks would execute on stream modules incorrectly
  • Fixed several other issues related with the custom patches we use for OpenResty


  • The base migration did not correctly reflect the state of indexes in postgres when updating from 0.14
  • Fixed an error involving offsets and pagination of Targets in the Admin API
  • Ensured that Kong is compatible with unpatched OpenResty for API Gateway mode (Service Mesh still requires patched OpenResty)
  • Made kong.response.exit work on the header_filter phase, when invoked with an empty body
  • Fixed an issue in kong.response.get_source where the “exit” status could be shadowed by the “proxied” status.
  • New --force option in kong migrations to allow upgrading between release candidates


  • The HTTP-log plugin handles nil values more gracefully now (thanks @jeremyjpj0916!)
  • The CORS plugin now returns 200 instead of 204 on preflight requests (thanks @aslafy-z!)
  • Request and Response Transformer plugins now tolerate errors and empty headers better
  • Fixed an issue in the Azure Functions plugin where some PDK and ngx_lua methods were incorrectly referenced
  • The Azure Functions plugin now filters out headers disallowed by HTTP/2 when proxying HTTP/1.1 responses to HTTP/2 clients

You can browse the complete list of changes for this release on GitHub. Additionally, Kong 1.0rc4 contains all changes introduced by 1.0rc1, 1.0rc2 & 1.0rc3.

Upgrade Path from 0.14 :arrow_up:

We have made sure that the following upgrade path from 0.14.x is supported, including “cold” nodes (Note: upgrading from a pre-0.14 cluster is not supported):

  1. Download 1.0rc4, and configure it to point to the same datastore as your 0.14 cluster. Run kong migrations up.
  2. Both 0.14 and 1.0rc4 nodes can now run simultaneously on the same datastore. Start provisioning 1.0rc4 nodes, but do not use their Admin API yet. Prefer making Admin API requests to your 0.14 nodes instead.
  3. Gradually divert traffic away from your 0.14 nodes, and into your 1.0rc4 cluster. Monitor your traffic to make sure everything is going smoothly.
  4. When your traffic is fully migrated to the 1.0rc4 cluster, decommission your 0.14 nodes.
  5. From your 1.0rc4 cluster, run: kong migrations finish. From this point on, it will not be possible to start 0.14 nodes pointing to the same datastore anymore. Only run this command when you are confident that your migration was successful. From now on, you can safely make Admin API requests to your 1.0rc4 nodes.

Upgrade Path from 1.0rc1, 1.0rc2 and 1.0rc3 :arrow_up:

The process is the same as for upgrading for 0.14.x, but on step 1 you should run kong migrations up --force instead.

Install 1.0rc4 on a Fresh Datastore :arrow_down:

1.0 introduces a new, improved migrations framework. The following commands can be run to prepare a new 1.0 cluster from a fresh datastore:

$ kong migrations bootstrap [-c config]
$ kong start [-c config]

What’s Next :rocket:

RC4 is our last planned release candidate. If no major issues are reported, it will be tagged and released as Kong 1.0!

As always, feel free to reach out to us here or via GitHub issues if you have any questions or feedback about this upgrade path, or the contents of this release in general! We appreciate the feedback we receive from our release candidate testers — thank you all in advance!


:partying_face: been waiting for this! Excited to get down to testing.


Edit - slight typo here step 5 in migrating existing data store:

Great news !

Could you publish this rc4 on docker hub ?

It seems that you didn’t create your PR for this rc4 : https://github.com/docker-library/official-images/pulls?q=is%3Apr+kong

Are there any breaking change with 0.14?

I just try to run my test suite with the latest RC4 and get this error:

2018/12/12 10:48:24 [error] 27#0: *25 lua coroutine: runtime error: /usr/local/share/lua/5.1/kong/db/schema/init.lua:1297: attempt to index local 'field_schema' (a nil value)
stack traceback:
coroutine 0:
	/usr/local/share/lua/5.1/kong/db/schema/init.lua: in function 'adjust_field_for_context'
	/usr/local/share/lua/5.1/kong/db/schema/init.lua:1382: in function 'adjust_field_for_context'
	/usr/local/share/lua/5.1/kong/db/schema/init.lua:1382: in function 'process_auto_fields'
	/usr/local/share/lua/5.1/kong/db/dao/init.lua:689: in function 'insert'
	/usr/local/share/lua/5.1/kong/api/endpoints.lua:235: in function </usr/local/share/lua/5.1/kong/api/endpoints.lua:217>
coroutine 1:
	[C]: in function 'resume'
	/usr/local/share/lua/5.1/lapis/application.lua:393: in function 'handler'
	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'resolve'
	/usr/local/share/lua/5.1/kong/api/init.lua:117: in function 'handler'
	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'handler'
	/usr/local/share/lua/5.1/lapis/application.lua:163: in function </usr/local/share/lua/5.1/lapis/application.lua:159>
	[C]: in function 'xpcall'
	/usr/local/share/lua/5.1/lapis/application.lua:159: in function 'dispatch'
	/usr/local/share/lua/5.1/lapis/nginx.lua:215: in function 'serve_admin_api'
	content_by_lua(nginx-kong.conf:166):2: in function <content_by_lua(nginx-kong.conf:166):1>, client:, server: kong_admin, request: "POST /plugins/ HTTP/1.1", host: "localhost:37403"

What I have changed: kong migrations up to kong migrations bootstrap

@david-maumenee Yes, rc4 for docker hub is coming! We’re sorting out some details of our docker image to improve its use in service-mesh use-cases.

is this a test suite for a custom plugin? is it published somewhere?

Thanks! Typo fixed :slight_smile:

It is Docker based test suite for our beta gluu plugins. It works fine with 0.14 but produce this error with rc3, rc4.
Our plugin code is not involved, I just try to configure our plugin and it fail within Kong internals.

Please note a big edit in the above announcement! We were so excited listing what’s added that we forgot to list what was removed. :sweat_smile:

The removal of “APIs” in favor of Routes and Services has been a long time coming and promised for 1.0. This is the rc that does it! :smiley:

1 Like

Looking into plugins shipped with the latest Kong I see some breaking changes compared to 0.14.
Are there any docs about?

I have checked all announcements here and don’t see any mentions about.

What is an abstract field?

I have added name field to a table returned by schema.lua and got response below trying to configure my plugin:

HTTP/1.1 400 Bad Request
Date: Wed, 12 Dec 2018 20:04:00 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.0rc4
Content-Length: 283

{"message":"2 schema violations (config: error in schema definition: abstract field was not specialized; service_id: unknown field)","name":"schema violation","fields":{"config":"error in schema definition: abstract field was not specialized","service_id":"unknown field"},"code":2}

The major change at play is that the config tables now use the new schema format of the new kong.db system (which replaces the old kong.dao system).

When a schema table in a custom plugin uses the old-format, Kong 1.0.0rc4 tries to do a best-effort auto-translation to the new format, but when you add a name field it assumes it is in the new format already, so it’s giving you these errors.

Full docs are on the way, but here’s a short summary example:

-- old way
return {
  no_consumer = true,
  fields = {
    foo = { type = "array" },
    bar = { type = "boolean" },

changes to:

-- new way
return {
  name = "my-plugin", 
  fields = {
    { config = {
       type = "record",
       fields = {
         { foo = { type = "array", elements = { type = "integer", between = {10, 20} } } },
         { bar = { type = "boolean", default = true } },
    } }
    { consumer = typedefs.no_consumer },

Though at first glance it may look like just a more verbose way of doing the same thing, here’s an explanation of the changes in the example:

-- old way

return {
  -- magic ad-hoc boolean for restricting consumers
  no_consumer = true,
  -- `fields` here are list just the subfields of the config property,
  -- so is not a full schema in the same way as the ones in Kong core
  -- and the custom DAOs that plugins define in daos.lua
  fields = {
    -- fields are unordered in a Lua table
    foo = { type = "array" }, -- array of what?
    bar = { type = "boolean" },

changes to:

-- new way

-- This is a full schema definition table, with the same syntax rules as
-- the custom DAO objects in daos.lua (and the Kong core objects)
return {
  -- entity name is in the table
  name = "my-plugin", 
  -- fields here is a list of fields of the my-plugin entity (not just of its `config` field)
  -- think of each plugin entity as a "subclass" of the parent Plugin entity
  -- we are then able to override the fields of Plugin here:
  fields = {
    -- we override the `config` field, which is left undefined (abstract) in the parent Plugin "class":
    { config = {
       -- we now define fully `config`, beginning with its type:
       type = "record",
       -- note that each entry in `fields`, both in the top-level
       -- and in the nested `config`, are wrapped in { }:
       -- this is for specifying field order
       fields = {
         { foo = {
           type = "array",
           -- we need to specify array element types,
           -- and we can specify constraints on them.
           -- here, to demonstrate, I added "between 10 and 20".
           -- all constraints you can use in a field can be used in "elements"
           elements = { type = "integer", between = {10, 20} }
         } },
         { bar = { type = "boolean", default = true } },
   -- instead of a magic boolean, here we use the same mechanism
   -- to impose a restriction to the `consumers` field of Plugin:
    { consumer = typedefs.no_consumer },
   -- "typedefs" are just shorthand predefined tables: the above means the same as
   -- { consumer = { type = "foreign", reference = "consumers", eq = null } }

Thanks Hisham, I will try to follow your guidance.
BTW old format doesn’t work at all with RC4 and RC3 for me.

So I assume a live upgrade with old plugins may be painful.

I pushed this image to Docker Hub. It is a build of the latest docker-kong repo. Only the centos tag is there.



In your PR to update docker hub https://github.com/docker-library/official-images/pull/5180 there is a concern about setcap cap_net_raw=+ep

In 1.0.0rc4 you are starting using a kong user instead of root user, great news !

I have just add a comment to this PR.

Unlike httpd or nginx, kong don’t open system ports (number 0 - 1023). So it seems that there is no need of setcap cap_net_raw=+ep. kong user already have permission to open port > 1024

I forked your repo to https://github.com/david-maumenee/docker-kong/tree/1.0.0rc4-no-cap_net_raw
to remove the line && setcap cap_net_raw=+ep /usr/local/openresty/nginx/sbin/nginx \

I have created two tag on my docker repo https://cloud.docker.com/repository/docker/dmaumenee/kong/tags

  • 1.0.0rc4-without-cap_net_raw-alpine
  • 1.0.0rc4-alpine

With my custom tag 1.0.0rc4-without-cap_net_raw-alpine it seems to work (host OS ubuntu 16.04).

Some command in my container named “kong_kong_1”

$ docker exec -it kong_kong_1 whoami

$ docker exec -it kong_kong_1 netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0  *               LISTEN
tcp        0      0  *               LISTEN
tcp        0      0*               LISTEN
tcp        0      0  *               LISTEN
udp        0      0*

$ docker exec -it kong_kong_1 ping localhost
PING localhost ( 56 data bytes
ping: permission denied (are you root?)

I was wrong, Kong image does need net_raw capability. see comment of james-callahan on the PR https://github.com/docker-library/official-images/pull/5180#issuecomment-447688905.

For early testing rc4 on alpine I built the alpine image