Is anyone able to proxy kubectl stream traffic (exec / logs / ..) with kong?

1

If so: How did you do it?

We’re using kong as our main lb on our rancher clusters and still have the problem that we can’t forward kubectl exec … stream traffic with kong.

We’re getting this error:

[{
  "metadata": {},
  "status": "Failure",
  "message": "Upgrade request required",
  "reason": "BadRequest",
  "code": 400
}]

The standard ingress controller can do it. I tried to compare the configs of the two and find differences without success.

Any help is appreciated.
Thanks

2

Kubernetes exec/attach/port-forward use SPDY, so I believe your issue will be related to How to enable SPDY support · Issue #6657 · Kong/kong · GitHub which seems to involve adding an extra nginx module somehow.

3

This is unfortunately not possible since that module got superseeded by http/2.

There was a patch from cloudflare for earlier nginx versions to enable both but since cloudflare deprecated spdy there is no patch for the current version.

Earlier versions of the main kubernetes nginx had that patch inside ([nginx-ingress-controller]: Add support for dynamic TLS records and spdy by aledbf · Pull Request #1238 · kubernetes-retired/contrib · GitHub) but I couldn’t find it in the current version.

Apparently also the standard nginx 1.14 can do it with that config.
I tried to compare this with the kong template but couldn’t really find major differences.
Also tried to inject the map part without success.

worker_processes 4;
worker_rlimit_nofile 40000;

events {
    worker_connections 8192;
}

http {
    upstream rancher {
        server rancher-server:80;
    }

    map $http_upgrade $connection_upgrade {
        default Upgrade;
        ''      close;
    }

    server {
        listen 443 ssl http2;
        server_name FQDN;
        ssl_certificate /certs/fullchain.pem;
        ssl_certificate_key /certs/privkey.pem;

        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Forwarded-Port $server_port;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://rancher;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
            # This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
            proxy_read_timeout 900s;
            proxy_buffering off;
        }
    }

    server {
        listen 80;
        server_name FQDN;
        return 301 https://$server_name$request_uri;
    }
}

Wondering why this is not working in kong.
Ideas anyone?

4

Hi @landor, I am trying to do the same thing. Have you been able to achieve this? If yes, please share the details and method of how to do it.

5

I somehow ended up back here, so I’ll add some further info. I’m not using Kong anymore though.

Contrary to my first message, Kubernetes apiserver doesn’t need Kong (or any proxy) to talk SPDY. It actually works exactly via the HTTP Upgrade mechanism. kubectl uses HTTP/1.1 to send a normal request with an Upgrade: SPDY/3.1 header, and then the connection switches to direct kubectl<-->kubelet SPDY.

Kong says it supports upgrades by default and the config has the right bits so it’s not clear what the issue would be if kubectl exec isn’t going thru.