Hi all,
I am using the Kong Ingress Controller and kong databaseless to register routes using kubernetes resources on minikube.
When setting up ingress resources which link to a simple echo-service, I find that some requests to the kong gateway randomly timeout. The timeout occurs randomly, as sometimes the routes will respond as expected and sometimes they will fail.
I managed to gather these logs from the Kong container:
2019/07/23 00:40:11 [error] 36#0: *1049 [lua] balancer.lua:554: get_balancer(): balancer not found for test-service.default.svc, will create it, client: 172.17.0.1, server: kong, request: "GET /test-service?token=3333 HTTP/1.1", host: "..."
2019/07/23 00:40:41 [crit] 36#0: *1049 [lua] balancer.lua:615: on_upstream_event(): failed creating balancer for test-service.default.svc: timeout waiting for balancer for 6fe18e36-4728-40cf-9a08-3a666aadee21, client: 172.17.0.1, server: kong, request: "GET /test-service?token=3333 HTTP/1.1", host: "..."
2019/07/23 00:40:41 [error] 36#0: *1049 [lua] balancer.lua:584: on_target_event(): target create: balancer not found for test-service.default.svc, client: 172.17.0.1, server: kong, request: "GET /test-service?token=3333 HTTP/1.1", host: "..."
2019/07/23 00:41:11 [crit] 36#0: *1049 [lua] balancer.lua:615: on_upstream_event(): failed creating balancer for test-service.default.svc: timeout waiting for balancer for 6fe18e36-4728-40cf-9a08-3a666aadee21, client: 172.17.0.1, server: kong, request: "GET /test-service?token=3333 HTTP/1.1", host: "..."
2019/07/23 00:41:11 [error] 36#0: *1049 [lua] balancer.lua:584: on_target_event(): target create: balancer not found for test-service.default.svc, client: 172.17.0.1, server: kong, request: "GET /test-service?token=3333 HTTP/1.1", host: "..."
2019/07/23 00:41:41 [crit] 36#0: *1049 [lua] balancer.lua:615: on_upstream_event(): failed creating balancer for test-service.default.svc: timeout waiting for balancer for 6fe18e36-4728-40cf-9a08-3a666aadee21, client: 172.17.0.1, server: kong, request: "GET /test-service?token=3333 HTTP/1.1", host: "..."
I did see this topic on github that might be related, but I am using Kong v. 1.1.2, which includes the PR that fixes the linked issue.
Any help would be appreciated. Let me know if you need anything else.
Thanks,
Arnav