Currently, LB to targets in an upstream employs a weighted-RR LB pattern.
It would be very beneficial for certain deployment configurations (particularly multi-dc) if we could support load balancing based on data-center or (preferably) healthcheck ping latency.
What I suggest: Is to add a new loadbalancing option to Kong for a node to locally cache the most recent healthcheck ping response times for all targets in an upstream, and route exclusively to the target with the lowest response times. This would allow users to greatly customize their routing, as it would largely be based on the frequency and concurrency of healthchecks.
I’d love some guidance from the Kong team on where to start looking to add some of this functionality!
Kong would need to use the latencies to update weights in the LB
Some things that require attention:
weight = 0 means don’t add to LB currently
keep track of targets added by name (a single name might add multiple IP addresses to the balancer, there is a related issue with the healthchecks currently)
how many datapoints to use for latency calculation? incidental latency spikes might have a big impact otherwise