Latency-Based Loadbalancing in Upstreams

#1

Hi Kong Team,

Currently, LB to targets in an upstream employs a weighted-RR LB pattern.

It would be very beneficial for certain deployment configurations (particularly multi-dc) if we could support load balancing based on data-center or (preferably) healthcheck ping latency.

What I suggest: Is to add a new loadbalancing option to Kong for a node to locally cache the most recent healthcheck ping response times for all targets in an upstream, and route exclusively to the target with the lowest response times. This would allow users to greatly customize their routing, as it would largely be based on the frequency and concurrency of healthchecks.

I’d love some guidance from the Kong team on where to start looking to add some of this functionality!

#2

yes that is absolutely an interesting feature to add. But not an easy one to implement.

There are multiple components involved in this whole chain.

  1. The healthcheck library would need to track latencies.
  2. Kong would need to use the latencies to update weights in the LB

Some things that require attention:

  • weight = 0 means don’t add to LB currently
  • keep track of targets added by name (a single name might add multiple IP addresses to the balancer, there is a related issue with the healthchecks currently)
  • how many datapoints to use for latency calculation? incidental latency spikes might have a big impact otherwise
#3

Thanks! Those are some very solid pointers. I’ll see what I can do here - I’d love to ultimately PR this