Does Kong Ingress Controller support TLS passthrough for HTTPs upstream service?

Hi, I was trying to use TCPIngress to enable TLS passthrough for my HTTPs upstream service. However, it looks it doesn’t work for me. Here’s what I applied:

apiVersion: configuration.konghq.com/v1beta1
kind: TCPIngress
metadata:
  name: mysvc-tls
  namespace: default
spec:
  rules:
  - backend:
      serviceName: mysvc
      servicePort: 443
    host: example.com
    port: 9443

Because I was using kind to launch a local Kubernetes cluster, the port 9443 will be ultimately mapped to a node port as below:

$ kubectl get svc
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                    AGE
example-kong-kong-proxy   LoadBalancer   10.96.27.58     <pending>     80:30252/TCP,443:32000/TCP,9000:31245/TCP,9443:32763/TCP   19d

So, I was using the master node IP 172.17.0.4 with the node port 32763 to access the upstream service through Kong. But when I was trying to curl it, I got some error as below:

curl -k --resolve example.com:32763:172.17.0.4  https://172.17.0.4:32763
curl: (52) Empty reply from server

Then, I noticed, the proxy logs:

2020/05/11 14:55:10 [warn] 27#0: *60017 stream [lua] handler.lua:604: get_service_for_route(): service with protocol 'http' cannot be used with 'stream' subsystem, context: ngx.timer
172.17.0.4 - - [11/May/2020:14:55:14 +0000] "GET /services HTTP/1.1" 200 648 "-" "curl/7.29.0"
172.17.0.4 [11/May/2020:14:56:28 +0000] TCP 500 0 0 0.182
2020/05/11 14:56:28 [error] 27#0: *62200 stream [lua] handler.lua:960: before(): no Route found with those values while prereading client data, client: 172.17.0.4, server: 0.0.0.0:9443

It looks it’s because my upstream service was using http as its protocol by default. This can be seen by checking the service information using admin API:

{
  "next": null,
  "data": [
    {
      "host": "mysvc.default.443.svc",
      "created_at": 1589208910,
      "connect_timeout": 60000,
      "id": "cfc3d65a-baa8-532c-b07d-99794425e593",
      "protocol": "http",
      "name": "default.mysvc.443",
      "read_timeout": 60000,
      "port": 80,
      "path": "/",
      "updated_at": 1589208910,
      "client_certificate": null,
      "tags": null,
      "write_timeout": 60000,
      "retries": 5
    }
  ]
}

However, when I was trying to modify this by annotating the service mysvc as below:

apiVersion: v1
kind: Service
metadata:
  annotations:
    konghq.com/protocol: tcp

It always fails and there’s error messages appeared in ingress controller’s pod:

E0511 15:09:30.473950       1 controller.go:127] unexpected failure updating Kong configuration:
posting new config to /config: 400 Bad Request {"fields":{"services":[{"path":"value must be null","@entity":["failed conditional validation given value of field 'protocol'"]}]},"name":"invalid declarative configuration","code":14,"message":"declarative config is invalid: {services={{[\"@entity\"]={\"failed conditional validation given value of field 'protocol'\"},path=\"value must be null\"}}}"}
W0511 15:09:30.473985       1 queue.go:112] requeuing default/kubernetes, err posting new config to /config: 400 Bad Request {"fields":{"services":[{"path":"value must be null","@entity":["failed conditional validation given value of field 'protocol'"]}]},"name":"invalid declarative configuration","code":14,"message":"declarative config is invalid: {services={{[\"@entity\"]={\"failed conditional validation given value of field 'protocol'\"},path=\"value must be null\"}}}"}

Is it because the tcp value does not support by the community edition? Or, is there anything I missed?

Thank you for the detailed post here. Having the entire flow helps understand the use-case easily.

Kong doesn’t support TLS pass-through in the way you are trying to implement.

Kong can do either of the two things:

  • Send all traffic on on a specific port to an upstream service
  • Terminate TLS connection on a specific port, and then route traffic to different upstream services based on the SNI

Today, Kong cannot sniff SNI, not terminate the TLS and route traffic to an usptream service.
I’ve come across this feature request a few times, but I’m not sure where it is not our roadmap currently. It is something we would like to add support for but is not urgent enough right now.

Thanks @hbagdi for your prompt reply and this valuable information as we are evaluating the technical feasibility of TLS passthrough for HTTPs upstream service using Kong. Since it’s not supported yet, I’d be happy to contribute something if it’s not very big changes introduced, given that our team will be proactively using Kong in our product strategically in the coming releases:-)

Back to the issue, I was not able to fully understand what it means:

Terminate TLS connection on a specific port, and then route traffic to different upstream services based on the SNI

Please correct me if any. Usually the reason we need TLS termination for incoming request is because we need to decrypt the payload so that we can touch it, e.g. to parse or modify HTTP headers as needed. On the other hand, if people cannot touch the payload, while still want to route traffic to specific target using shared IP/port, here comes the SNI. And that’s appears to me the exact scenario of TLS passthrough. So, I do not understand why Kong needs TLS termination in order to support SNI. When it can terminate TLS, it can extract HOST from HTTP headers. Or, anything I missed here?

Also, as I know, to support TLS passthrough against HTTPs backend services is straightforward for nginx. So, I’m wondering what’s the real difficulties behind for Kong to support this since Kong is built on top of OpenResty, hence based on nginx as well.

Looking forward your guidance!

Of course. Let’s start with a Github issue for a feature request and continue the conversation there.

It needs to terminate TLS because the feature of TLS passthrough is missing.

The reason is that the route by sni is a module inside nginx. Openresty doesn’t have these capabilities (yet). I could be wrong here.

Are you looking for SNI based routing for some services while terminating TLS for some other services? Or is that all traffic to different services should be routed based on SNI?

Hi @hbagdi, sorry for my late response. I created a git issue to track the TLS passthrough discussion at: https://github.com/Kong/kong/issues/5902

Regarding your question:

Our ultimate goal is to look for ways of mixed use for both TLS termination and passthrough. We are going to put Kong as an API Gateway centrally, for those need authentication and data transformation, we need to terminate, others can passthrough. It could have different SNI so that can share the same IP and port.


© 2019 Kong Inc.    Terms  •  Privacy  •  FAQ