Kong fails to register plugins (1.3.0rc1)

Kubernetes
1

Hi,

I am running Kong version 1.3.0rc1 databaseless with the kubernetes ingress controller 0.5.0. I ran into an issue where Kong appears to be unable to register plugins on routes. When looking in the Kong pods’ error logs I see this message repeated:

2019/08/06 17:43:54 [error] 35#0: *19056 lua entry thread aborted: runtime error: /usr/local/share/lua/5.1/kong/runloop/balancer.lua:249: attempt to index local 'healthchecker' (a nil value)
stack traceback:
coroutine 0:
        /usr/local/share/lua/5.1/kong/runloop/balancer.lua: in function 'callback'
        /usr/local/share/lua/5.1/resty/dns/balancer/base.lua:1270: in function </usr/local/share/lua/5.1/resty/dns/balancer/base.lua:1269>, context: ngx.timer
2019/08/06 17:43:54 [error] 39#0: *18914 [lua] balancer.lua:609: on_target_event(): target create: balancer not found for service.namespace.svc, context: ngx.timer

In the Kubernetes logs I see this in the Ingress Controller:

E0806 17:44:16.035853       1 controller.go:125] unexpected failure updating Kong configuration: 
posting new config to /config: 400 Bad Request {"fields":{"services":[null,{"routes":[{"plugins":[null,null,null,null,{"config":{"add":"expected a record"}}]}]}]},"name":"invalid declarative configuration","code":14,"message":"declarative config is invalid: {services={[2]={routes={{plugins={[5]={config={add=\"expected a record\"}}}}}}}}"}
W0806 17:44:16.035876       1 queue.go:113] requeuing namespace/service.plugin, err posting new config to /config: 400 Bad Request {"fields":{"services":[null,{"routes":[{"plugins":[null,null,null,null,{"config":{"add":"expected a record"}}]}]}]},"name":"invalid declarative

Any help with understanding these errors, would be much appreciated.

Thanks,
Arnav

2

The configuration of one of the plugins was off.

What was being used:

    Name:         service.request-transformer.0d89f0fd-d618-4a57-9b40-bab45ec7bf41
    Namespace:    arnav
    Labels:       <none>
    Annotations:  <none>
    API Version:  configuration.konghq.com/v1
    Config:
      Add:  multi_tenant:true
    Kind:   KongPlugin
    Metadata:
      Creation Timestamp:  2019-08-07T21:44:54Z
      Generation:          1
      Resource Version:    560200
      Self Link:           /apis/configuration.konghq.com/v1/namespaces/arnav/kongplugins/service.request-transformer.0d89f0fd-d618-4a57-9b40-bab45ec7bf41
      UID:                 97d5a7fe-b95c-11e9-bc08-06d77556a440
    Plugin:                request-transformer
    Events:                <none>
Correct Configuration:
    Name:         service.request-transformer.0d89f0fd-d618-4a57-9b40-bab45ec7bf41
    Namespace:    arnav
    Labels:       <none>
    Annotations:  <none>
    API Version:  configuration.konghq.com/v1
    Config.add.querystring:
      multi_tenant:true
    Kind:  KongPlugin
    Metadata:
      Creation Timestamp:  2019-08-07T21:44:54Z
      Generation:          4
      Resource Version:    563706
      Self Link:           /apis/configuration.konghq.com/v1/namespaces/arnav/kongplugins/service.request-transformer.0d89f0fd-d618-4a57-9b40-bab45ec7bf41
      UID:                 97d5a7fe-b95c-11e9-bc08-06d77556a440
    Plugin:                request-transformer
    Events:                <none>

Many thanks to @hbagdi for his help.