Kong 1.0 (rc4) available for testing

hisham · December 11, 2018, 9:17pm

Hello everyone,

Continuing our release candidate cycle for Kong 1.0, we’re happy to announce the availability of RC4!

Download Kong 1.0.0rc4 today and give us your feedback!

This fourth release candidate contains fixes for Service Mesh that were detected after releasing RC3. Thanks again to everyone who reported issues - they helped a lot!

RC4 Contains the Following Improvements over RC3:

Removal of deprecated features

As one of the final steps in preparing 1.0.0 final, this release removes from the codebase features and concepts that were marked as deprecated earlier in the 0.x cycle. In this release:

the API entity and related concepts such as the /apis endpoint, are removed (deprecated since 0.13.0, March 2018). Use Routes and Services instead.
the custom_plugins directive is removed (deprecated since 0.14.0, July 2018). Use plugins instead.
the galileo plugin is removed (deprecated since 0.13.0)
the kong.tools.ip module was removed (deprecated since the introduction of the PDK in 0.14.0). Use kong.ip from the PDK instead.
the kong.tools.public module was removed (deprecated since the introduction of the PDK in 0.14.0). Use the PDK instead.
the kong.api.crud_helpers module was removed (deprecated since the introduction of the new DAO in 0.13.0). Use kong.api.endpoints instead if you need to customize the auto-generated endpoints.
finally, the old DAO implementation is removed, along with the old schema validation library (apis was the last entity using it). Use the new schema format instead in custom plugins (full documentation is on the way). To ease the transition of plugins, the plugin loader in 1.0 includes a best-effort schema auto-translator, which should be sufficient for many plugins (in 1.0.0rc1, our bundled plugins used the auto-translator; since 1.0.0rc2 they now use the new format).

We plan to make a 0.15.0 release alongside 1.0.0, which will still include these deprecated features but will be otherwise identical to 1.0.0, to ease the migration path. If you have a custom plugin using any of the featues listed above, now is a great time to update them! All plugins bundled with Kong 1.0.0rc4 have been converted to remove dependency on these features, so they serve as good examples, along with the PDK documentation.

Service Mesh and Stream Routing

TLSv1.3 added to the template protocols as a workaround for openssl#7660
Fixed an issue where lua_package_path was not properly defined for stream requests
Fixed an issue where active healthchecks would execute on stream modules incorrectly
Fixed several other issues related with the custom patches we use for OpenResty

Core

The base migration did not correctly reflect the state of indexes in postgres when updating from 0.14
Fixed an error involving offsets and pagination of Targets in the Admin API
Ensured that Kong is compatible with unpatched OpenResty for API Gateway mode (Service Mesh still requires patched OpenResty)
Made kong.response.exit work on the header_filter phase, when invoked with an empty body
Fixed an issue in kong.response.get_source where the “exit” status could be shadowed by the “proxied” status.
New --force option in kong migrations to allow upgrading between release candidates

Plugins

The HTTP-log plugin handles nil values more gracefully now (thanks @jeremyjpj0916!)
The CORS plugin now returns 200 instead of 204 on preflight requests (thanks @aslafy-z!)
Request and Response Transformer plugins now tolerate errors and empty headers better
Fixed an issue in the Azure Functions plugin where some PDK and ngx_lua methods were incorrectly referenced
The Azure Functions plugin now filters out headers disallowed by HTTP/2 when proxying HTTP/1.1 responses to HTTP/2 clients

You can browse the complete list of changes for this release on GitHub. Additionally, Kong 1.0rc4 contains all changes introduced by 1.0rc1, 1.0rc2 & 1.0rc3.

Upgrade Path from 0.14

We have made sure that the following upgrade path from 0.14.x is supported, including “cold” nodes (Note: upgrading from a pre-0.14 cluster is not supported):

Download 1.0rc4, and configure it to point to the same datastore as your 0.14 cluster. Run kong migrations up.
Both 0.14 and 1.0rc4 nodes can now run simultaneously on the same datastore. Start provisioning 1.0rc4 nodes, but do not use their Admin API yet. Prefer making Admin API requests to your 0.14 nodes instead.
Gradually divert traffic away from your 0.14 nodes, and into your 1.0rc4 cluster. Monitor your traffic to make sure everything is going smoothly.
When your traffic is fully migrated to the 1.0rc4 cluster, decommission your 0.14 nodes.
From your 1.0rc4 cluster, run: kong migrations finish. From this point on, it will not be possible to start 0.14 nodes pointing to the same datastore anymore. Only run this command when you are confident that your migration was successful. From now on, you can safely make Admin API requests to your 1.0rc4 nodes.

Upgrade Path from 1.0rc1, 1.0rc2 and 1.0rc3

The process is the same as for upgrading for 0.14.x, but on step 1 you should run kong migrations up --force instead.

Install 1.0rc4 on a Fresh Datastore

1.0 introduces a new, improved migrations framework. The following commands can be run to prepare a new 1.0 cluster from a fresh datastore:

$ kong migrations bootstrap [-c config]
$ kong start [-c config]

What’s Next

RC4 is our last planned release candidate. If no major issues are reported, it will be tagged and released as Kong 1.0!

As always, feel free to reach out to us here or via GitHub issues if you have any questions or feedback about this upgrade path, or the contents of this release in general! We appreciate the feedback we receive from our release candidate testers — thank you all in advance!

jeremyjpj0916 · December 11, 2018, 11:45pm

been waiting for this! Excited to get down to testing.

Thanks.

Edit - slight typo here step 5 in migrating existing data store:

david-maumenee · December 12, 2018, 7:56am

Great news !

Could you publish this rc4 on docker hub ?

It seems that you didn’t create your PR for this rc4 : https://github.com/docker-library/official-images/pulls?q=is%3Apr+kong

Alexander_Altshuler · December 12, 2018, 11:08am

Are there any breaking change with 0.14?

I just try to run my test suite with the latest RC4 and get this error:

2018/12/12 10:48:24 [error] 27#0: *25 lua coroutine: runtime error: /usr/local/share/lua/5.1/kong/db/schema/init.lua:1297: attempt to index local 'field_schema' (a nil value)
stack traceback:
coroutine 0:
	/usr/local/share/lua/5.1/kong/db/schema/init.lua: in function 'adjust_field_for_context'
	/usr/local/share/lua/5.1/kong/db/schema/init.lua:1382: in function 'adjust_field_for_context'
	/usr/local/share/lua/5.1/kong/db/schema/init.lua:1382: in function 'process_auto_fields'
	/usr/local/share/lua/5.1/kong/db/dao/init.lua:689: in function 'insert'
	/usr/local/share/lua/5.1/kong/api/endpoints.lua:235: in function </usr/local/share/lua/5.1/kong/api/endpoints.lua:217>
coroutine 1:
	[C]: in function 'resume'
	/usr/local/share/lua/5.1/lapis/application.lua:393: in function 'handler'
	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'resolve'
	/usr/local/share/lua/5.1/kong/api/init.lua:117: in function 'handler'
	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'handler'
	/usr/local/share/lua/5.1/lapis/application.lua:163: in function </usr/local/share/lua/5.1/lapis/application.lua:159>
	[C]: in function 'xpcall'
	/usr/local/share/lua/5.1/lapis/application.lua:159: in function 'dispatch'
	/usr/local/share/lua/5.1/lapis/nginx.lua:215: in function 'serve_admin_api'
	content_by_lua(nginx-kong.conf:166):2: in function <content_by_lua(nginx-kong.conf:166):1>, client: 172.19.0.1, server: kong_admin, request: "POST /plugins/ HTTP/1.1", host: "localhost:37403"

What I have changed: kong migrations up to kong migrations bootstrap

hisham · December 12, 2018, 11:26am

@david-maumenee Yes, rc4 for docker hub is coming! We’re sorting out some details of our docker image to improve its use in service-mesh use-cases.

hisham · December 12, 2018, 11:27am

is this a test suite for a custom plugin? is it published somewhere?

hisham · December 12, 2018, 11:27am

Thanks! Typo fixed

Alexander_Altshuler · December 12, 2018, 11:41am

It is Docker based test suite for our beta gluu plugins. It works fine with 0.14 but produce this error with rc3, rc4.
Our plugin code is not involved, I just try to configure our plugin and it fail within Kong internals.

hisham · December 12, 2018, 5:43pm

Please note a big edit in the above announcement! We were so excited listing what’s added that we forgot to list what was removed.

The removal of “APIs” in favor of Routes and Services has been a long time coming and promised for 1.0. This is the rc that does it!

Alexander_Altshuler · December 13, 2018, 8:11am

Looking into plugins shipped with the latest Kong I see some breaking changes compared to 0.14.
Are there any docs about?

I have checked all announcements here and don’t see any mentions about.

What is an abstract field?

I have added name field to a table returned by schema.lua and got response below trying to configure my plugin:

HTTP/1.1 400 Bad Request
Date: Wed, 12 Dec 2018 20:04:00 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.0rc4
Content-Length: 283

{"message":"2 schema violations (config: error in schema definition: abstract field was not specialized; service_id: unknown field)","name":"schema violation","fields":{"config":"error in schema definition: abstract field was not specialized","service_id":"unknown field"},"code":2}

hisham · December 13, 2018, 11:33am

The major change at play is that the config tables now use the new schema format of the new kong.db system (which replaces the old kong.dao system).

When a schema table in a custom plugin uses the old-format, Kong 1.0.0rc4 tries to do a best-effort auto-translation to the new format, but when you add a name field it assumes it is in the new format already, so it’s giving you these errors.

Full docs are on the way, but here’s a short summary example:

-- old way
return {
  no_consumer = true,
  fields = {
    foo = { type = "array" },
    bar = { type = "boolean" },
  }
}

changes to:

-- new way
return {
  name = "my-plugin", 
  fields = {
    { config = {
       type = "record",
       fields = {
         { foo = { type = "array", elements = { type = "integer", between = {10, 20} } } },
         { bar = { type = "boolean", default = true } },
       }
    } }
    { consumer = typedefs.no_consumer },
  }
}

Though at first glance it may look like just a more verbose way of doing the same thing, here’s an explanation of the changes in the example:

-- old way

return {
  -- magic ad-hoc boolean for restricting consumers
  no_consumer = true,
  -- `fields` here are list just the subfields of the config property,
  -- so is not a full schema in the same way as the ones in Kong core
  -- and the custom DAOs that plugins define in daos.lua
  fields = {
    -- fields are unordered in a Lua table
    foo = { type = "array" }, -- array of what?
    bar = { type = "boolean" },
  }
}

changes to:

-- new way

-- This is a full schema definition table, with the same syntax rules as
-- the custom DAO objects in daos.lua (and the Kong core objects)
return {
  -- entity name is in the table
  name = "my-plugin", 
  -- fields here is a list of fields of the my-plugin entity (not just of its `config` field)
  -- think of each plugin entity as a "subclass" of the parent Plugin entity
  -- we are then able to override the fields of Plugin here:
  fields = {
    -- we override the `config` field, which is left undefined (abstract) in the parent Plugin "class":
    { config = {
       -- we now define fully `config`, beginning with its type:
       type = "record",
       -- note that each entry in `fields`, both in the top-level
       -- and in the nested `config`, are wrapped in { }:
       -- this is for specifying field order
       fields = {
         { foo = {
           type = "array",
           -- we need to specify array element types,
           -- and we can specify constraints on them.
           -- here, to demonstrate, I added "between 10 and 20".
           -- all constraints you can use in a field can be used in "elements"
           elements = { type = "integer", between = {10, 20} }
         } },
         { bar = { type = "boolean", default = true } },
       }
    }
   -- instead of a magic boolean, here we use the same mechanism
   -- to impose a restriction to the `consumers` field of Plugin:
    { consumer = typedefs.no_consumer },
   -- "typedefs" are just shorthand predefined tables: the above means the same as
   -- { consumer = { type = "foreign", reference = "consumers", eq = null } }
  }
}

Alexander_Altshuler · December 13, 2018, 12:29pm

Thanks Hisham, I will try to follow your guidance.
BTW old format doesn’t work at all with RC4 and RC3 for me.

So I assume a live upgrade with old plugins may be painful.

neomantra · December 15, 2018, 1:46pm

I pushed this image to Docker Hub. It is a build of the latest docker-kong repo. Only the centos tag is there.

neomantra/kong:1.0.0rc4-centos

david-maumenee · December 16, 2018, 9:30am

Hi,

In your PR to update docker hub https://github.com/docker-library/official-images/pull/5180 there is a concern about setcap cap_net_raw=+ep

In 1.0.0rc4 you are starting using a kong user instead of root user, great news !

I have just add a comment to this PR.

Unlike httpd or nginx, kong don’t open system ports (number 0 - 1023). So it seems that there is no need of setcap cap_net_raw=+ep. kong user already have permission to open port > 1024

I forked your repo to https://github.com/david-maumenee/docker-kong/tree/1.0.0rc4-no-cap_net_raw
to remove the line && setcap cap_net_raw=+ep /usr/local/openresty/nginx/sbin/nginx \

I have created two tag on my docker repo https://cloud.docker.com/repository/docker/dmaumenee/kong/tags

1.0.0rc4-without-cap_net_raw-alpine
1.0.0rc4-alpine

With my custom tag 1.0.0rc4-without-cap_net_raw-alpine it seems to work (host OS ubuntu 16.04).

Some command in my container named “kong_kong_1”

$ docker exec -it kong_kong_1 whoami
kong

$ docker exec -it kong_kong_1 netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:8001            0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.11:43332        0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:8443            0.0.0.0:*               LISTEN
udp        0      0 127.0.0.11:47307        0.0.0.0:*

$ docker exec -it kong_kong_1 ping localhost
PING localhost (127.0.0.1): 56 data bytes
ping: permission denied (are you root?)

david-maumenee · December 18, 2018, 8:23am

I was wrong, Kong image does need net_raw capability. see comment of james-callahan on the PR https://github.com/docker-library/official-images/pull/5180#issuecomment-447688905.

For early testing rc4 on alpine I built the alpine image
https://hub.docker.com/r/dmaumenee/kong/tags

Topic		Replies	Views
Kong 1.0 (rc1) available for download Announcements	3	1630	September 28, 2018
Kong 1.4.0rc1 available for testing! Announcements	0	692	October 2, 2019
Kong 1.0 (rc3) available for testing Announcements	5	1957	November 27, 2018
Kong 1.0rc2 available for testing Announcements	5	2271	November 15, 2018
Kong 1.4.0rc2 available for testing! Announcements	3	1451	October 17, 2019