Currently, LB to targets in an upstream employs a weighted-RR LB pattern.
It would be very beneficial for certain deployment configurations (particularly multi-dc) if we could support load balancing based on data-center or (preferably) healthcheck ping latency.
What I suggest: Is to add a new loadbalancing option to Kong for a node to locally cache the most recent healthcheck ping response times for all targets in an upstream, and route exclusively to the target with the lowest response times. This would allow users to greatly customize their routing, as it would largely be based on the frequency and concurrency of healthchecks.
I’d love some guidance from the Kong team on where to start looking to add some of this functionality!
Kong would need to use the latencies to update weights in the LB
Some things that require attention:
weight = 0 means don’t add to LB currently
keep track of targets added by name (a single name might add multiple IP addresses to the balancer, there is a related issue with the healthchecks currently)
how many datapoints to use for latency calculation? incidental latency spikes might have a big impact otherwise
Take the avg latency of a configurable seen last X latency checks of the targets to dictate where to route. Until the latency aware routing can kick in(after # of checks) can route to various healthy targets. Or route to the first lowest latency target seen? Whatever makes sense there is fine.
The req is that the targets are healthy ofc too, we don’t care to route to a super fast target throwing Errors back at us on the health checks.