We recently migrated from 0.14.1 to 1.3.0 and switched to DB-less with the migration from Postgre. After migrating we’ve noticed intermittent 404s with the message { "message": "no Route matched with those values" } during periods with higher than usual traffic, around 5,000 ~ 10,000 requests per minute. We have roughly 200 combined services, routes, and plugins in use. Most of our plugins are custom plugins for header manipulation and authentication against a Redis instance. We never ran into this issue while on 0.14.1.
We’ve confirmed that during this periods the route configuration is removed from the cache and all of our previously configured services and routes are deleted. There’s no notable error messages displayed when the cache is cleared besides worker process 43 exited on signal 11.
We’ve increased the mem_cache_size to 1024m which has not had much effect. Many of the values in our nginx configuration file come from https://github.com/Kong/kong/blob/1.3.0/spec/fixtures/custom_nginx.template. At this point we’re unsure of what the issue could be. We’re thinking there might be some sort of memory leak as the issue tends to resolve itself when we restart the existing services, but still need to do more investigation before we can be sure.
Any suggested workarounds or fixes would be greatly appreciated. Thank you!