# Connection and Upgrade Headers modified **URL:** \ **Category:** Questions\ **Tags:** kubernetes\ **Created:** [January 2, 2021, 10:33am UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668 "2021-01-02T10:33:02Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![bit-monkey](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@bit-monkey](https://discuss.konghq.com/u/bit-monkey)\ **Post date:** [January 2, 2021, 10:33am UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/1 "2021-01-02T10:33:02Z")
Hi, I’m trying to run [JupyterHub](https://zero-to-jupyterhub.readthedocs.io/en/0.10.2/index.html) behind Kong in Kubernetes. Most things work fine, but Terminals are unresponsive to user input. When I log the user pod, I am seeing 400’s pop up on websocket connections: `400 GET /user/{username}/terminals/websocket/1` This is definitely a Kong issue, because if I port-forward the service directly everything works fine. I tried replicating the request with CURL using the same headers as the browser passes, and I got the response: `Can "Upgrade" only to "WebSocket"` This is something which [tornado does when the upgrade header does not equal websocket](https://stackoverflow.com/questions/32944339/tornado-can-upgrade-only-to-websocket-error), so I pointed the same request at an echo-server to see what headers the server is recieving. Indeed, the upgrade header is gone entirely, and the connection header was replaced: ``` CURL: > GET / HTTP/1.1 > Connection: keep-alive, Upgrade > Upgrade: websocket Echo: Request Headers: connection=keep-alive ``` How can I prevent Kong from overwriting these headers? [The documentation](https://docs.konghq.com/1.4.x/proxy/#proxy-websocket-traffic) says these headers should not be overwritten anyway? ---
**Author:** ![traines](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/traines/32/158_2.png) [@traines](https://discuss.konghq.com/u/traines)\ **Post date:** [January 5, 2021, 6:58pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/2 "2021-01-05T18:58:47Z")
Are you indeed on 1.4? That version had [a bug](https://github.com/Kong/kong/issues/5465) that mangles `Connection` improperly with Firefox-style values (`keep-alive, Upgrade` instead of `Upgrade` alone). [https://github.com/Kong/kong/commit/f1f1c61333f908c327e31f77c6c478a1902f0b2b](https://github.com/Kong/kong/commit/f1f1c61333f908c327e31f77c6c478a1902f0b2b) fixes it as of 2.0.1. It doesn’t look like we released any 1.x version with it; you’d need to patch the image manually if you don’t want to upgrade yet. ---
**Author:** ![tyree731](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/tyree731/32/1374_2.png) [@tyree731](https://discuss.konghq.com/u/tyree731)\ **Post date:** [January 6, 2021, 2:51pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/3 "2021-01-06T14:51:57Z")
We’ve been running JupityrHub behind Kong, and I’m just confirming traines idea, since we ran into that exact bug. ---
**Author:** ![bit-monkey](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@bit-monkey](https://discuss.konghq.com/u/bit-monkey)\ **Post date:** [January 7, 2021, 12:26am UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/4 "2021-01-07T00:26:17Z")
> Are you indeed on 1.4? Yes I am, but I’m unsure if that bug is the culprit. I have the same behavior on Chrome ---
**Author:** ![bit-monkey](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@bit-monkey](https://discuss.konghq.com/u/bit-monkey)\ **Post date:** [January 7, 2021, 9:46am UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/5 "2021-01-07T09:46:52Z")
Tried upgrading to Kong 2.2.1, but identical behavior. Terminal is unresponsive and websocket returns 400: ``` Request headers: GET /user/dummy/terminals/websocket/1 HTTP/1.1 Host: jupyter.. User-Agent: Mozilla/5.0 () Gecko/20100101 Firefox/84.0 Accept: */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate, br Sec-WebSocket-Version: 13 Origin: https://jupyter.. Sec-WebSocket-Extensions: permessage-deflate Sec-WebSocket-Key: Connection: keep-alive, Upgrade Cookie: jupyterhub-user-dummy= Pragma: no-cache Cache-Control: no-cache Upgrade: websocket Response Headers: HTTP/1.1 400 Bad Request content-security-policy: frame-ancestors 'self';report-uri /hub/security/csp-report Content-Type: text/html; charset=UTF-8 date: Thu, 07 Jan 2021 09:30:00 GMT server: TornadoServer/6.1 Via: kong/2.2.1 x-content-type-options: nosniff x-jupyterhub-version: 1.2.2 X-Kong-Proxy-Latency: 1 X-Kong-Upstream-Latency: 3 Content-Length: 34 Connection: keep-alive ``` I don’t have an echoserver set up to check the upstream headers on the cluster I’m testing on, but presume the same thing is happening with `Connection` being overridden and `Upgrade` being removed. @tyree731 Did you ever solve this? ---
**Author:** ![tyree731](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/tyree731/32/1374_2.png) [@tyree731](https://discuss.konghq.com/u/tyree731)\ **Post date:** [January 7, 2021, 2:22pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/6 "2021-01-07T14:22:52Z")
Since I believe you can run custom scripts on the JupityrHub side of things, maybe try printing out the request body JupityrHub is receiving and see if perhaps something is getting lost in translation. Also, what is your route and service configuration? Are you running any plugins that would affect the request? ---
**Author:** ![traines](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/traines/32/158_2.png) [@traines](https://discuss.konghq.com/u/traines)\ **Post date:** [January 8, 2021, 8:44pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/7 "2021-01-08T20:44:06Z")
What are you seeing with an echo-server now? Is there possibly any other HTTP-aware hop between Kong and JupyterHub? ---
**Author:** ![bit-monkey](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@bit-monkey](https://discuss.konghq.com/u/bit-monkey)\ **Post date:** [January 9, 2021, 9:34pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/8 "2021-01-09T21:34:26Z")
This is a clean install of Kong, no KongPlugins or KongIngresses configured. My Kubernetes Ingress is created by the JupterHub helm chart. The corresponding values are: ``` ingress: enabled: true annotations: kubernetes.io/ingress.class: kong hosts: - jupyter.newcluster.. ``` I’ll do an echo test and post that soon ---
**Author:** ![bit-monkey](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@bit-monkey](https://discuss.konghq.com/u/bit-monkey)\ **Post date:** [January 9, 2021, 10:05pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/9 "2021-01-09T22:05:09Z")
Request headers: GET / HTTP/1.1 Host: [echo.newcluster.company.com](http://echo.newcluster.company.com) User-Agent: curl/7.54.0 Accept: _/_ Connection: keep-alive, Upgrade Upgrade: websocket Upstream headers: accept=_/_ connection=keep-alive [host=echo.newcluster.company.com](http://host=echo.newcluster.company.com) user-agent=curl/7.54.0 x-forwarded-for=IP, IP [x-forwarded-host=echo.newcluster.company.com](http://x-forwarded-host=echo.newcluster.company.com) x-forwarded-path=/ x-forwarded-port=443 x-forwarded-proto=https x-real-ip=IP There is no hop between Kong and Jupyterhub, but kong is itself behind an AWS Classic Loadbalancer ---
**Author:** ![tyree731](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/tyree731/32/1374_2.png) [@tyree731](https://discuss.konghq.com/u/tyree731)\ **Post date:** [January 11, 2021, 1:02pm UTC](https://discuss.konghq.com/t/connection-and-upgrade-headers-modified/7668/10 "2021-01-11T13:02:35Z")
You’re using a classic load balancer? Can you try using an Application Load Balancer? I know ALB’s support websockets, but I’m not so sure classic load balancers do.