How to expose kong-proxy with own TLS certificates and custom domain name

Kubernetes
1

Hi I am using the latest kong GW ingress from the github it is installed using the kubectl apply -f https://bit.ly/k4k8s method from the github page.

However instead of deploying everything in the kong namespace I have them deployed in a namespace called “prod”

I would like to expose my api (the kong-proxy) on a custom domain that I have using a wildcard certificate.

For example https://api.mycompany.com/v1/my-service should hit a pod called my-service on a GET route. The pod “my-service” should be protected by kong and have whatever plugins applied to it

If i deploy kong-proxy service with type loadbalancer (reproduced below for clarity)

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  name: kong-proxy
  namespace: prod
spec:
  externalTrafficPolicy: Local
  ports:
  - name: proxy
    port: 80
    protocol: TCP
    targetPort: 8000
  - name: proxy-ssl
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: ingress-kong
  type: LoadBalancer

Standard ingress-kong deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ingress-kong
  name: ingress-kong
  namespace: prod
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-kong
  template:
    metadata:
      annotations:
        prometheus.io/port: "9542"
        prometheus.io/scrape: "true"
        traffic.sidecar.istio.io/includeInboundPorts: ""
      labels:
        app: ingress-kong
    spec:
      containers:
      - env:
        - name: KONG_DATABASE
          value: "off"
        - name: KONG_NGINX_WORKER_PROCESSES
          value: "1"
        - name: KONG_NGINX_HTTP_INCLUDE
          value: /kong/servers.conf
        - name: KONG_ADMIN_ACCESS_LOG
          value: /dev/stdout
        - name: KONG_ADMIN_ERROR_LOG
          value: /dev/stderr
        - name: KONG_ADMIN_LISTEN
          value: 127.0.0.1:8444 ssl
        - name: KONG_PROXY_LISTEN
          value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
        image: kong:1.3
        lifecycle:
          preStop:
            exec:
              command:
              - /bin/sh
              - -c
              - kong quit
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /health
            port: 9001
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: proxy
        ports:
        - containerPort: 8000
          name: proxy
          protocol: TCP
        - containerPort: 8443
          name: proxy-ssl
          protocol: TCP
        - containerPort: 9542
          name: metrics
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /health
            port: 9001
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        securityContext:
          runAsUser: 1000
        volumeMounts:
        - mountPath: /kong
          name: kong-server-blocks
      - args:
        - /kong-ingress-controller
        - --kong-url=https://localhost:8444
        - --admin-tls-skip-verify
        - --publish-service=prod/kong-proxy
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.6.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: ingress-controller
        ports:
        - containerPort: 8080
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
      serviceAccountName: kong-serviceaccount
      volumes:
      - configMap:
          name: kong-server-blocks
        name: kong-server-blocks

I can see in k8s that I get a cluster IP and I can visit the created cluster IP in my browser http://cluster-IP or https://cluster-IP

In order to associate the domain name api.company.com with the cluster IP I have to update my DNS settings with an A record point to it is this enough?

Now if I surf to https://cluster-IP I get a kong localhost generated certificate, but I don’t want this certificate I want to use my own wildcard certificate.

I have my cert file and key file, and I create a k8s secret as usual, now how do I set this with the kong-proxy?

I tried the following as a quick test to see if i can associate the kong-proxy service with the custom domain and the tls secret but the k8s ingress does not route my request to the kong-proxy on port 80 due to backend services being unhealthy?

Perhaps I am using the Ingress object wrong and it should only be used to configure my routes (i.e path should be /v1/my-service pointing to serviceName: my-service) and not attempt to expose the kong-proxy? - I was trying to associate the TLS secret with kong-proxy here

apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: kong-proxy-ingress
  namespace: prod
route:
  methods:
  - GET
  - POST
  strip_path: false
-------
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: platform2-ingress
  namespace: prod
  annotations:
    configuration.konghq.com: kong-proxy-ingress
    kubernetes.io/ingress.class: "kong"
spec:
  tls:
  - hosts:
    - api.company.com
    secretName: company-tls
  rules:
    - host: api.company.com
      http:
        paths:
        - path: /
          backend:
            serviceName: kong-proxy
            servicePort: 80

If i create a Ingress object, It creates a Http Load balancer, now I have 2 External IPs - one for kong-proxy and one for the Ingress object, which one do I have to set in my DNS records?

Any guidance is appreciated in these matters.

Thanks,
Sevren