Support sticky sessions for websockets


#1

Hello!

A sticky session is necessary in order to support load balancing websockets connections. Nginx allows for a sticky cookie to be added to route traffic to the correct server, but there doesn’t seem to be a way to support this with kong. I see there is the option to hash_on IP, but we haven’t been able to get this working, and our consumers for the service with websockets are all behind the same NAT gateway, so we really need to use cookie injection to load balance correctly.

If something like this is already supported, please point me in the right direction, but as stands, this is blocking us from fully integrating with kong.

Thank you!

Kong version
0.13.1


#2

when the handshake is completed the connection turns into an non-http socket connection. Hence, if the first request (containing the upgrade-request) is properly hashed, then each following request (websocket or not) should end up on the same server as long as the hashing is configured properly. Or stated otherwise, there is no difference in sticky sessions between a regular http request and a websocket from Kong perspective.


#3

To answer the question: we do not currently support that.

First thoughts would be to extend the consistent hashing with the option to hash on a cookie value (besides IP and headers), combined with a plugin that inserts the actual cookie.


#4

Thanks for the reply! I agree with your first thoughts on supporting this use case. I notice there are a couple of community plugins that could provide something closer to this when combined.

the device uid plugin adds a cookie with a uid to the client’s device
the cookies-to-headers plugin converts cookies to headers (only for the authorization header, but could be modified to make a sticky header)

We might try some approach of combining these as a workaround.

As for supporting hashing on cookies, is this something you’d be amenable to as a PR to the project, or is further discussion required from the core team? In other words, how can I help to make this dream a reality?


#5

had a quick look at those plugins. I think you can make it work like this:

  • Update the deviceuid plugin to generate the cookie on incoming requests, and also reinserting the same one on outging (otherwise the first request will not have a cookie yet, and won’t be picked up by the consistent-hashing algorithm)
  • Update the cookies-to-headers plugin, to make the headername configurable
  • make sure the priority is set such that deviceuid runs before cookies-to-headers

alternatively wrap the two into one


#6

Thanks for looking at that! We’ll be writing the plugin in the next couple of weeks (or finding some alternative way to fix our websockets while using kong). Will post the results here.


#7

We could alternatively modify the core, and add it as a generic option, if you’re willing to create a PR for that? (and be eligible for the famous Kong contributor t-shirt!)


#8

That would be the ideal solution :slight_smile:
Are you thinking of making hash_on: cookie an option, with sticky cookies as a plugin? Or were you considering making sticky sessions an option like the k8s nginx ingress controller does with affinity annotations?

I would definitely be up for working on this. If there’s anything we can do to kick this off, please let me know.


#9

I meant hash_on: cookies as an option for the hashing algorithm. But then without the plugin, as it would all be in the core.

  • when a request comes in (access phase) check for a cookie
  • if there is no cookie, create (not the cookie, but the uuid)
  • store the uuid from the cookie in ngx.ctx
  • in header-filter phase retrieve the uuid from ngx.ctx and add it to the response as a cookie

probably need some config items like cookie name and path

Does the above make sense?


#10

Yes, thank you. I’m working on a PR to add hashing on a cookie right now.


#11

For reference, WIP PR: https://github.com/Kong/kong/pull/3472