Cannot get Kong Manager to talk to Admin over HTTPS

Hi all - new to Kong and hoping to get a little direction when setting up the manager UI via Kubernetes/Helm and getting it to communicate over HTTPS to the admin API’s. I think it comes down to a typo somewhere that I am missing or just inexperience :slight_smile:

Below is my values.yml file that I am using, a mix from a couple of different tutorials, including Victor’s Youtube video(s) https://www.youtube.com/watch?v=xbFXvNUtfFQ&t=3121s&ab_channel=Kong and some others found around Kong website/github for examples.

I believe I’ve set all the variables (and added extra annotations for https everywhere I think it would matter to resolve). The issue I am experiencing is that HTTPS works mostly. But the communication between the manager UI and the admin API is being transmitted over HTTP instead of HTTPS. Everything works over HTTP just fine.

Any help would be appreciated.

# Kong for Kubernetes with Kong Enterprise with Enterprise features enabled and
# exposed via TLS-enabled Ingresses. Before installing:
# * Several settings (search for the string "CHANGEME") require user-provided
#   Secrets. These Secrets must be created before installation.
# * Ingresses reference example "<service>.kong.CHANGEME.example" hostnames. These must
#   be changed to an actual hostname that resolve to your proxy.
# * Ensure that your session configurations create cookies that are usable
#   across your services. The admin session configuration must create cookies
#   that are sent to both the admin API and Kong Manager, and any Dev Portal
#   instances with authentication must create cookies that are sent to both
#   the Portal and Portal API.

# Sections:
# - Deployment parameters
# - Kong parameters
# - Ingress Controller parameters
# - Postgres sub-chart parameters
# - Miscellaneous parameters
# - Kong Enterprise parameters

# -----------------------------------------------------------------------------
# Deployment parameters
# -----------------------------------------------------------------------------

deployment:
  kong: 
    # Enable or disable Kong itself
    # Setting this to false with ingressController.enabled=true will create a
    # controller-only release.
    # enabled: true
    ## Minimum number of seconds for which a newly created pod should be ready without any of its container crashing,
    ## for it to be considered available.
    # minReadySeconds: 60
    ## Specify the service account to create and to be assigned to the deployment / daemonset and for the migrations
    serviceAccount:
        create: true
        # Automount the service account token. By default, this is disabled, and the token is only mounted on the controller
        # container. Some sidecars require enabling this. Note that enabling this exposes Kubernetes credentials to Kong
        # Lua code, increasing potential attack surface.
        automountServiceAccountToken: false
        ## Optionally specify the name of the service account to create and the annotations to add.
        name: kong-sa
        #  annotations: {}


# Override namepsace for Kong chart resources. By default, the chart creates resources in the release namespace.
# This may not be desirable when using this chart as a dependency.
# namespace: "example"

# -----------------------------------------------------------------------------
# Kong parameters
# -----------------------------------------------------------------------------

# Specify Kong configuration
# This chart takes all entries defined under `.env` and transforms them into into `KONG_*`
# environment variables for Kong containers.
# Their names here should match the names used in https://github.com/Kong/kong/blob/master/kong.conf.default
# See https://docs.konghq.com/latest/configuration also for additional details
# Values here take precedence over values from other sections of values.yaml,
# e.g. setting pg_user here will override the value normally set when postgresql.enabled
# is set below. In general, you should not set values here if they are set elsewhere.
env:
    database: 'off'
    prefix: /kong_prefix/
    pg_user: 'kong'
    pg_password: 'kong'
    pg_database: 'kong'
    # pg_host: 'pg-zoe-ha-ha.postgres-operator.svc.cluster.local'
    # These should have no trailing slashes. This will cause bugs
    admin_api_uri: 'https://admin.kong.d.zoerml.zlocal'
    admin_gui_url: 'https://manager.kong.d.zoerml.zlocal'
    # password:
    #     valueFrom:
    #         secretKeyRef:
    #             name: kong-enterprise-superuser-password #CHANGEME
    #             key: password #CHANGEME
image:
    repository: kong/kong-gateway
    tag: "3.4"
    pullPolicy: IfNotPresent


admin:
    enabled: true
    type: LoadBalancer
    loadBalancerIP: 10.20.3.8
    annotations: 
        cert-manager.io/cluster-issuer: vault-issuer
        konghq.com/plugins: zoe-cors
        konghq.com/protocol: "https"
    http:
        enabled: true
        servicePort: 8001
        containerPort: 8001
    tls:
        enabled: true
        servicePort: 8444
        containerPort: 8444
        parameters:
            - http2

    ingress:
        enabled: true
        ingressClassName: kong
        # TLS secret name.
        # tls: kong-kong-admin-cert
        # Ingress hostname
        hostname: admin.kong.d.zoerml.zlocal
        # Map of ingress annotations.
        annotations: 
            cert-manager.io/cluster-issuer: vault-issuer
            # konghq.com/plugins: zoe-cors
            konghq.com/https-redirect-status-code: "308"
            konghq.com/protocols: https
            konghq.com/protocol: "https"
            konghq.com/strip-path: "true"
            nginx.ingress.kubernetes.io/backend-protocol: HTTPS
            
        # Ingress path.
        path: /
        # Each path in an Ingress is required to have a corresponding path type. (ImplementationSpecific/Exact/Prefix)
        pathType: ImplementationSpecific

# Specify Kong proxy service configuration
proxy:
    # Enable creating a Kubernetes service for the proxy        
    enabled: true
    type: LoadBalancer
    loadBalancerIP: 10.20.3.6
    annotations: 
        konghq.com/plugins: zoe-cors
        konghq.com/protocol: "https"
        # If terminating TLS at the ELB, the following annotations can be used
        # "service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "*",
        # "service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled": "true",
        # "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:REGION:ACCOUNT:certificate/XXXXXX-XXXXXXX-XXXXXXX-XXXXXXXX",
        # "service.beta.kubernetes.io/aws-load-balancer-ssl-ports": "kong-proxy-tls",
        # "service.beta.kubernetes.io/aws-load-balancer-type": "elb"
    # labels:
    #     enable-metrics: "true"
    http:
        # Enable plaintext HTTP listen for the proxy
        enabled: true
        servicePort: 80
        containerPort: 8000
        parameters: []

    tls:
        # Enable HTTPS listen for the proxy
        enabled: true
        servicePort: 443
        containerPort: 8443
        parameters:
            - http2

    # stream: {}

    ingress:
        enabled: true
        annotations: 
            cert-manager.io/cluster-issuer: vault-issuer
            konghq.com/plugins: zoe-cors
            konghq.com/protocols: https
            konghq.com/protocol: "https"
            konghq.com/strip-path: "true"
            nginx.ingress.kubernetes.io/backend-protocol: HTTPS
            konghq.com/https-redirect-status-code: 308
           
        path: /
        # Ingress hosts. Use this instead of or in combination with hostname to specify multiple ingress host configurations
        # hostname: manager.kong.d.zoerml.zlocal
        hosts: 
        # - host: kong-proxy.example.com
        #   paths:
        #   # Ingress path.
        #   - path: /*
        #   # Each path in an Ingress is required to have a corresponding path type. (ImplementationSpecific/Exact/Prefix)
        #     pathType: ImplementationSpecific
        - host: admin.kong.d.zoerml.zlocal
          paths:
          # Ingress path.
          - path: /
          # Each path in an Ingress is required to have a corresponding path type. (ImplementationSpecific/Exact/Prefix)
            pathType: ImplementationSpecific
            backend:
              service:
                name: kong-kong-admin
                port:
                  number: 8444
        - host: manager.kong.d.zoerml.zlocal
          paths:
          # Ingress path.
          - path: /
          # Each path in an Ingress is required to have a corresponding path type. (ImplementationSpecific/Exact/Prefix)
            pathType: ImplementationSpecific
            backend:
              service:
                name: kong-kong-manager
                port:
                  number: 8445
        # TLS secret(s)
        # tls: kong-proxy.example.com-tls
        # Or if multiple hosts/secrets needs to be configured:
        # tls:
        # - secretName: kong-proxy.zoerml.zlocal-tls
        #   hosts:
        #   - api.d.zoerml.zlocal
        #   - manager.kong.d.zoerml.zlocal
        #   - admin.kong.d.zoerml.zlocal

    externalIPs: []


manager:
    enabled: true
    type: LoadBalancer
    loadBalancerIP: 10.20.3.7
    annotations: 
        cert-manager.io/cluster-issuer: vault-issuer
        konghq.com/plugins: zoe-cors
        konghq.com/protocol: "https"
    http:
        enabled: true
        servicePort: 8002
        containerPort: 8002
        # Set a nodePort which is available if service type is NodePort
        # nodePort: 32080
        # Additional listen parameters, e.g. "reuseport", "backlog=16384"
        parameters: []

    tls:
        enabled: true
        servicePort: 8445
        containerPort: 8445
        parameters:
            - http2

    ingress:
        enabled: true
        ingressClassName: kong
        # tls: CHANGEME-manager-tls-secret
        hostname: manager.kong.d.zoerml.zlocal
        annotations: 
            cert-manager.io/cluster-issuer: vault-issuer
            konghq.com/plugins: zoe-cors
            konghq.com/protocol: "https"
            konghq.com/protocols: https
            konghq.com/https-redirect-status-code: "301"
            nginx.ingress.kubernetes.io/backend-protocol: HTTPS
        path: /

    externalIPs: []

enterprise:
    enabled: false
    # CHANGEME: https://github.com/Kong/charts/blob/main/charts/kong/README.md#kong-enterprise-license
    license_secret: kong-enterprise-license
    vitals:
        enabled: false
    portal:
        enabled: false
    rbac:
        enabled: false
        admin_gui_auth: basic-auth
        # session_conf_secret: kong-session-config
        # admin_gui_auth_conf_secret: CHANGEME-admin-gui-auth-conf-secret
        admin_gui_session_conf:
            secret: 'thisisnotasecret'
            cookie_secure: true
            cookie_samesite: false
    smtp:
        enabled: false
        portal_emails_from: none@example.com
        portal_emails_reply_to: none@example.com
        admin_emails_from: none@example.com
        admin_emails_reply_to: none@example.com
        smtp_admin_emails: none@example.com
        smtp_host: smtp.example.com
        smtp_port: 587
        smtp_auth_type: ''
        smtp_ssl: nil
        smtp_starttls: true
        auth:
            smtp_username: '' # e.g. postmaster@example.com
            smtp_password_secret: CHANGEME-smtp-password

portal:
    enabled: false
    type: ClusterIP
    # annotations:
    #     konghq.com/protocol: 'https'

    http:
        enabled: true
        servicePort: 8003
        containerPort: 8003
        parameters: []

    tls:
        enabled: true
        servicePort: 8446
        containerPort: 8446
        parameters:
            - http2

    ingress:
        enabled: false
        # tls: CHANGEME-portal-tls-secret
        # hostname: portal.kong.CHANGEME.example
        annotations:
            kubernetes.io/ingress.class: 'kong'
        path: /

    externalIPs: []

portalapi:
    enabled: false
    type: ClusterIP
    # annotations:
    #     konghq.com/protocol: 'https'

    http:
        enabled: true
        servicePort: 8004
        containerPort: 8004
        parameters: []

    tls:
        enabled: false
        servicePort: 8447
        containerPort: 8447
        parameters:
            - http2

    ingress:
        enabled: true
        # tls: CHANGEME-portalapi-tls-secret
        # hostname: portalapi.kong.CHANGEME.example
        annotations:
            kubernetes.io/ingress.class: 'kong'
        path: /

    externalIPs: []

certificates:
  enabled: true

  # Set either `issuer` or `clusterIssuer` to the name of the desired cert manager issuer
  # If left blank a built in self-signed issuer will be created and utilized
  clusterIssuer: vault-issuer

  # Set proxy.enabled to true to issue default kong-proxy certificate with cert-manager
  proxy:
    enabled: true
    # Set `issuer` or `clusterIssuer` to name of alternate cert-manager clusterIssuer to override default
    # self-signed issuer.
    issuer: ""
    clusterIssuer: vault-issuer
    # Use commonName and dnsNames to set the common name and dns alt names which this
    # certificate is valid for. Wildcard records are supported by the included self-signed issuer.
    commonName: "api.d.zoerml.zlocal"
    # Remove the "[]" and uncomment/change the examples to add SANs
    dnsNames: 
     - "api.d.zoerml.zlocal"
     - "admin.kong.d.zoerml.zlocal"
     - "manager.kong.d.zoerml.zlocal"
     - "kong-kong-proxy.kong.svc.cluster.local"
    # - "*.kong.example"

  # Set admin.enabled true to issue kong admin api and manager certificate with cert-manager
  admin:
    enabled: true
    # Set `issuer` or `clusterIssuer` to name of alternate cert-manager clusterIssuer to override default
    # self-signed issuer.
    issuer: ""
    clusterIssuer: vault-issuer
    # Use commonName and dnsNames to set the common name and dns alt names which this
    # certificate is valid for. Wildcard records are supported by the included self-signed issuer.
    commonName: "admin.kong.d.zoerml.zlocal"
    
    # Remove the "[]" and uncomment/change the examples to add SANs
    dnsNames: 
        - "admin.kong.d.zoerml.zlocal" 
        - "manager.kong.d.zoerml.zlocal"

  # Set portal.enabled to true to issue a developer portal certificate with cert-manager
  portal:
    enabled: true
    # Set `issuer` or `clusterIssuer` to name of alternate cert-manager clusterIssuer to override default
    # self-signed issuer.
    issuer: ""
    clusterIssuer: vault-issuer
    # Use commonName and dnsNames to set the common name and dns alt names which this
    # certificate is valid for. Wildcard records are supported by the included self-signed issuer.
    commonName: "portal.developer.d.zoerml.zlocal"
    # Remove the "{}" and uncomment/change the examples to add SANs
    dnsNames: []
    # - "manager.kong.example"

  # Set cluster.enabled true to issue kong hybrid mtls certificate with cert-manager
  cluster:
    enabled: true
    # Issuers used by the control and data plane releases must match for this certificate.
    issuer: ""
    clusterIssuer: vault-issuer
    commonName: "cluster.kong.d.zoerml.zlocal"
    dnsNames: []

postgresql:
    enabled: false
    auth:
        username: kong
        database: kong
        password: kong
        postgresPassword: kong
    image:
        # use postgres < 14 until is https://github.com/Kong/kong/issues/8533 resolved and released
        # enterprise (kong-gateway) supports postgres 14
        tag: 13.11.0-debian-11-r20
    service:
        ports:
            postgresql: "5432"

# Custom Kong plugins can be loaded into Kong by mounting the plugin code
# into the file-system of Kong container.
# The plugin code should be present in ConfigMap or Secret inside the same
# namespace as Kong is being installed.
# The `name` property refers to the name of the ConfigMap or Secret
# itself, while the pluginName refers to the name of the plugin as it appears
# in Kong.
# Subdirectories (which are optional) require separate ConfigMaps/Secrets.
# "path" indicates their directory under the main plugin directory: the example
# below will mount the contents of kong-plugin-rewriter-migrations at "/opt/kong/rewriter/migrations".
plugins: {}
  # configMaps:
  # - pluginName: rewriter
  #   name: kong-plugin-rewriter
  #   subdirectories:
  #   - name: kong-plugin-rewriter-migrations
  #     path: migrations
  # secrets:
  # - pluginName: rewriter
  #   name: kong-plugin-rewriter
# Inject specified secrets as a volume in Kong Container at path /etc/secrets/{secret-name}/
# This can be used to override default SSL certificates.
# Be aware that the secret name will be used verbatim, and that certain types
# of punctuation (e.g. `.`) can cause issues.
# Example configuration
# secretVolumes:
# - kong-proxy-tls
# - kong-admin-tls
secretVolumes: []