HTTP to HTTPS redirect with kong-ingress and TLS-offloading

thecodingrobot · October 31, 2019, 9:26pm

Hi all.

I’m having issues setting up HTTP to HTTPS redirect on the kong ingress while using AWS ELB (classic) and HTTPS listener for TLS offloading. With the current configuration no matter whether the request arrives via HTTP or HTTPS kong issues 302 redirect. By the looks of it Kong is not handling the X-Forward-Proto header supplied by the ELB.

I have followed the posts in the forum about similar issue (Redirecting HTTP to HTTPS - #23 by Sothy_Lorn) but so far it doesn’t seem that anyone had success implementing proper HTTP to HTTPS redirect with TLS offloading or am I configuring Kong incorrectly?

Thanks in advance for any hints!

The Kong ingress is provisioned using the stable/kong Helm chart.
Some relevant bits from values.yaml:

proxy:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-2:XXXX
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: 443
  http:
    enabled: true
    servicePort: 80
    containerPort: 8000
  tls:
    enabled: true
    servicePort: 443
    containerPort: 8443
    overrideServiceTargetPort: 8000 # <=

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: adminer
  annotations:
    kubernetes.io/ingress.class: kong
    configuration.konghq.com: ingress-api
spec:
  rules:
    - host: XXXX
      http:
        paths:
          - path: /
            backend:
              serviceName: adminer
              servicePort: 80

apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: ingress-api
route:
  protocols:
    - https
  https_redirect_status_code: 302

hbagdi · October 31, 2019, 9:54pm

You need to set KONG_TRUSTED_IPS environment variable so that Kong trusts the headers sent by ELB:

mnombrado · February 11, 2020, 4:19pm

Hi @hbagdi @thecodingrobot

I have the same issue. Were you able to resolve it?

Ingress and KongIngress definitions:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "2048-ingress"
  namespace: "2048-game"
  annotations:
    kubernetes.io/ingress.class: kong
    kubernetes.io/tls-acme: "true"
    configuration.konghq.com: https-only
  labels:
    app: 2048-ingress
spec:
  rules:
    - host: 2048.test.com
      http:
        paths:
          - path: /
            backend:
              serviceName: "service-2048"
              servicePort: 80
---
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
    name: https-only
    namespace: 2048-game
route:
  protocols:
  - https
  https_redirect_status_code: 302

Both HTTP and HTTPS are redirected causing loop.

sh-4.2$ curl -I http://2048.test.com
HTTP/1.1 302 Moved Temporarily
Date: Tue, 11 Feb 2020 16:07:26 GMT
Content-Type: text/html
Content-Length: 110
Connection: keep-alive
Location: https://2048.test.com
X-Kong-Response-Latency: 0
Server: kong/1.4.3

sh-4.2$ curl -I https://2048.test.com
HTTP/1.1 302 Moved Temporarily
Date: Tue, 11 Feb 2020 16:07:33 GMT
Content-Type: text/html
Content-Length: 110
Connection: keep-alive
Location: https://2048.test.com
X-Kong-Response-Latency: 1
Server: kong/1.4.3

hbagdi · February 11, 2020, 4:55pm

Can you set KONG_TRUSTED_IPS=0.0.0.0/0 and try?

mnombrado · February 12, 2020, 3:19am

@hbagdi, thanks for reply. I have updated the ingress-kong deployment env variables to add the KONG_TRUSTED_IPS. Is that what you mean? The result is the same, both http and https is being redirected to https causing loop.

spec:
  containers:
  - env:
    - name: KONG_DATABASE
      value: "off"
    - name: KONG_NGINX_WORKER_PROCESSES
      value: "1"
    - name: KONG_NGINX_HTTP_INCLUDE
      value: /kong/servers.conf
    - name: KONG_ADMIN_ACCESS_LOG
      value: /dev/stdout
    - name: KONG_ADMIN_ERROR_LOG
      value: /dev/stderr
    - name: KONG_ADMIN_LISTEN
      value: 127.0.0.1:8444 ssl
    - name: KONG_PROXY_LISTEN
      value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
    - name: KONG_TRUSTED_IPS
      value: 0.0.0.0/0

hbagdi · February 12, 2020, 5:11pm

KONG_TRUSTED_IPS=0.0.0.0/0,::/0

Can you change trusted ips to that?

To make sure, are you using ELB or NLB? ELB supports x-forwarded-proto and this should work with that configuration.

Kong correctly parses forwarded-proto and matches the route accordingly. I tested this locally to make sure of it.

mnombrado · February 12, 2020, 6:08pm

It worked! I’m using ELB. Only the http is redirected to https now, so loop issue was resolved. Thanks a lot @hbagdi.

hbagdi · February 13, 2020, 7:11pm

We should add this to docs somewhere. I’m not sure where though.

jl-gogovapps · February 19, 2020, 5:06pm

I’ve been able to confirm this works in my use case. However, I’m wondering how this would work in the context of a Helm chart. Looking through the templates, it doesn’t seem supported.

Would it be useful if I created a PR to allow this as an env parameter?

jl-gogovapps · February 19, 2020, 5:40pm

Looks like this is already handled here

gael · June 16, 2020, 10:21pm

Hi,
What about NLB ? Is there a way to configure HTTP to HTTPS redirection with that load balancer ?
Thanks,

hbagdi · June 17, 2020, 7:05pm

The same approach should work if you are terminating HTTP at NLB.

VLam · January 10, 2021, 10:00am

Hi @hbagdi,

I’m using nlb and facing The plain HTTP request was sent to HTTPS port, i tried adding KONG_TRUSTED_IPS to kong deployment but no luck.
below is my kong-proxy service

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:...
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: nlb

  ports:
  - name: proxy
    nodePort: 31193
    port: 80
    protocol: TCP
    targetPort: 8000
  - name: proxy-ssl
    nodePort: 30102
    port: 443
    protocol: TCP
    targetPort: 8443

and my ingress be like:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    konghq.com/override: https-only
    konghq.com/strip-path: "true"

and ingress-kong pod variables:

Environment:
  KONG_PROXY_LISTEN:            0.0.0.0:8000, 0.0.0.0:8443 ssl http2
  KONG_PORT_MAPS:               80:8000, 443:8443
  KONG_ADMIN_LISTEN:            127.0.0.1:8444 ssl
  KONG_STATUS_LISTEN:           0.0.0.0:8100
  KONG_DATABASE:                off
  KONG_NGINX_WORKER_PROCESSES:  2
  KONG_ADMIN_ACCESS_LOG:        /dev/stdout
  KONG_ADMIN_ERROR_LOG:         /dev/stderr
  KONG_PROXY_ERROR_LOG:         /dev/stderr
  KONG_TRUSTED_IPS:             0.0.0.0/0, ::/0

If i change proxy-ssl target port to 8000 then got a redirect loop

  - name: proxy-ssl
    nodePort: 30102
    port: 443
    protocol: TCP
    targetPort: 8000

Not sure what i’m missing here. Thank you in advance !

Sergey_S · July 21, 2021, 4:41pm

Hi team and @hbagdi
We have we same issue on our AWS NLB. We use overrideServiceTargetPort to resolve “The plain HTTP request was sent to HTTPS port”.
If the use

konghq.com/protocols: "https"
konghq.com/https-redirect-status-code: "301"

it makes too many redirections(loop) and this looks impossible now to configure http-https redirection with overrideServiceTargetPort

vothanhbinhlt · September 8, 2021, 2:59pm

Hello Team,

I have had the same issue. I am using AWS NLB. If we point the target group of port 443 to HTTPS port of KIC (Kong Ingress Controller), we will see an error message “The plain HTTP request was sent to HTTPS port”. However, If we point the target group of port 443 to the HTTP port of KIC, everything is worked fine, but we can not redirect HTTP to HTTPS because the x-forwarding-port is always HTTP. It makes too many redirections

VLam · September 8, 2021, 3:32pm

it’s because you send unencrypted data to a SSL encrypted port, try adjusting your target port to http port will help, or switch to ELB which is easier.

donalddw · September 23, 2021, 10:52am

Any help here would be appreciated. I have exactly the same problem as the guys above.

If we do TLS on a NLB, we need to set the ports for the service in kong-proxy to this

spec:
  ports:
    - name: proxy
      port: 80
      protocol: TCP
      targetPort: 8000
    - name: proxy-ssl
      port: 443
      protocol: TCP
      targetPort: 8000

Otherwise we’re getting The plain HTTP request was sent to HTTPS port

On our ingress I put

  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/protocols: "https"
    konghq.com/https-redirect-status-code: "301"

But this causes an endless redirect loop

This is deployment with the environment variables

containers:
        - env:
            - name: KONG_PROXY_LISTEN
              value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
            - name: KONG_PORT_MAPS
              value: 80:8000, 443:8443
            - name: KONG_ADMIN_LISTEN
              value: 127.0.0.1:8444 ssl
            - name: KONG_STATUS_LISTEN
              value: 0.0.0.0:8100
            - name: KONG_DATABASE
              value: "off"
            - name: KONG_NGINX_WORKER_PROCESSES
              value: "2"
            - name: KONG_ADMIN_ACCESS_LOG
              value: /dev/stdout
            - name: KONG_ADMIN_ERROR_LOG
              value: /dev/stderr
            - name: KONG_PROXY_ERROR_LOG
              value: /dev/stderr
            - name: KONG_TRUSTED_IPS
              value: 0.0.0.0/0,::/0
          image: kong:2.5

@hbagdi

Enzos23 · December 20, 2021, 1:41am

Hello, after having encountered the same problem as you ‘too many redirects’ I debugged and found a solution. The redirect problem is caused by this part of the conf:

  tls:
# Use it for ELB load balancer type not NLB
    overrideServiceTargetPort: 8000

so I removed this overrideServiceTargetPort which avoids the rewriting of port 8443 on port 8000.

In addition you have to use this annotation:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: “ssl”
not
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: “tcp”
at the risk of encountering a plain text error on an HTTPS connection.

Moreover, the kong doc clearly indicates that for an L4 load balancer like NLB you have to use its var env:

  trusted_ips: "0.0.0.0/0,::/0"
  proxy_listen: "0.0.0.0:8000 proxy_protocol, 0.0.0.0:8443 ssl proxy_protocol"
  real_ip_header: "proxy_protocol"

But this is not enough, since the proxy listens on port 8443 with the SSL protocol, the healthcheck executed by the load balancer uses the TCP protocol which generates an error of the type:

"broken header: "" while reading PROXY protocol".

The solution I found to this is to add a var env:

status_listen: "0.0.0.0:8100"

you have to add this annotation too:

service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "8100"

This allows the load balancer to get the status on port 8100 via TCP without generating an error.
hoping to have helped you !

Vishnu_Kadavarath · December 25, 2021, 8:24pm

Hi @Enzos23 your method didn’t work for me . It returns

* TLSv1.2 (IN), TLS alert, close notify (256):
* Empty reply from server
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
curl: (52) Empty reply from server

ELB with TLS is not able to direct my request to Kong when i remove the overrideServiceTargetPort

jrowinski3d · April 14, 2022, 10:41pm

I too am getting empty reply with the solution posted from @Enzos23 . I also get a redirect loop if I try to enforce https redirect. I am using a NLB with the following configurations.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ingress-kong
  name: ingress-kong
  namespace: kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-kong
  template:
    metadata:
      annotations:
        kuma.io/gateway: enabled
        traffic.sidecar.istio.io/includeInboundPorts: ""
      labels:
        app: ingress-kong
    spec:
      containers:
      - env:
        - name: KONG_PROXY_LISTEN
          value: 0.0.0.0:8000 proxy_protocol, 0.0.0.0:8443 ssl proxy_protocol
        - name: KONG_PORT_MAPS
          value: 80:8000, 443:8443
        - name: KONG_ADMIN_LISTEN
          value: 127.0.0.1:8444 ssl
        - name: KONG_STATUS_LISTEN
          value: 0.0.0.0:8100
        - name: KONG_DATABASE
          value: "off"
        - name: KONG_NGINX_WORKER_PROCESSES
          value: "2"
        - name: KONG_KIC
          value: "on"
        - name: KONG_ADMIN_ACCESS_LOG
          value: /dev/stdout
        - name: KONG_ADMIN_ERROR_LOG
          value: /dev/stderr
        - name: KONG_PROXY_ERROR_LOG
          value: /dev/stderr
        - name: KONG_PLUGINS
          value: bundled,custom-auth
        - name: KONG_LUA_PACKAGE_PATH
          value: /opt/?.lua;;
        - name: KONG_TRUSTED_IPS
          value: 0.0.0.0/0,::/0
        - name: KONG_REAL_IP_HEADER
          value: proxy_protocol

Service

apiVersion: v1
kind: Service
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/hostname: xxxx
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "8100"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxxx
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  name: kong-proxy
  namespace: kong
spec:
  ports:
  - name: proxy
    port: 80
    protocol: TCP
    targetPort: 8000
  - name: proxy-ssl
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: ingress-kong
  type: LoadBalancer

Topic		Replies	Views
Redirecting HTTP to HTTPS Questions kubernetes	35	21909	March 29, 2023
SSL Redirect from http to https Questions kubernetes	2	2903	September 9, 2020
HTTP to HTTPS Redirection Using Kong Ingress on Kubernete Questions kubernetes	0	19	October 1, 2024
Is there any way to redirect http to https in kong	3	2723	December 17, 2020
X-Forwarded-Proto forcing HTTPS redirect Questions kubernetes	1	793	January 20, 2023

HTTP to HTTPS redirect with kong-ingress and TLS-offloading

Related topics