Kong 0.14 installation on GKE 1.14.6 fails due to missing volumes

Hi,

I am a newbie in both Kong and GCP, so sorry if I made any silly mistake.
When I try do install Kong 0.14 either from Marketplace or following the steps from Kong via Manifest Files I get the error below in kong-control-plane POD’s Container Events:

MountVolume.SetUp failed for volume “api-server-cert” : secret “kong-control-plane.kong.svc” not found

Unable to mount volumes for pod “kong-control-plane-5ccfd6574d-mbq8g_kong(1a773f89-f752-11e9-9b7a-42010a80011a)”: timeout expired waiting for volumes to attach or mount for pod “kong”/“kong-control-plane-5ccfd6574d-mbq8g”. list of unmounted volumes=[api-server-cert]. list of unattached volumes=[api-server-cert kong-token-87df9]

At the Kubernetes > Storage tab there are no such volumes:

The installation works fine on a GKE 1.13.10-gke-0.

Thank you in advance for your attention.
Souto.

1 Like

It seems like api-server-cert volume is needed in the default Deployment, you can safely remove the volume mount and then your pods will be created correctly.

1 Like

Hi Harry,

I followed your advice and removed the volume mount and the variables KONG_ADMIN_SSL_CERT and KONG_ADMIN_SSL_CERT_KEY from the kong-control-plane-postgres.yaml file, and Kong installed successfully.
Thanks!

Which feature do I miss with that?
Will this be fixed/documented it so people can install Kong seamlessly on GKE version 1.14.6 and later?

Best Regards,
Souto.

Those settings are used for the default TLS certificate used by Kong when you have no certificate defined in Kong and Kong receives an HTTPS request.

We’re in the process of deprecating Kong from the GCP marketplace and instead are going to recommend users to use our Helm chart to deploy Kong onto Kubernetes.

Hi Harry,

Without any changes on k8s or Kong, Kong stopped working with the errors below:

kong-control-plane container logs:
Error: Cannot run migrations: database needs bootstrapping; run ‘kong migrations bootstrap’

kong-ingress-data-plane container logs:
2019/11/06 18:31:02 [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/cmd/utils/migrations.lua:20: Database needs bootstrapping; run ‘kong migrations bootstrap’

I tried scaling the job to zero and back to 1 but it didn’t solve the issue.

While trying to figure it out I found at https://docs.konghq.com/install/kubernetes/#using-a-database that the recommended way to run Kong on Kubernetes is DB-less. In this case is the declarative configuration supported in a cluster of Kong nodes?

Side note: I tried to install Kong with Helm with no success:
helm repo update Hang tight while we grab the latest from your chart repositories... ...Skip local chart repository ...Successfully got an update from the "stable" chart repository Update Complete. helm install stable/kong
Error: failed to download “stable/kong” (hint: running helm repo update may help)

Regards,
Souto

This happens when the backing Persistent Volume for the Postgres pod goes away. You want to ensure that the Postgres pod’s storage is stable.

If you would like to use Kong’s DB-less mode, you should check out Kong for Kubernetes:

It allows you to manage all your configuration in a Kubernetes native way.

Hi Harry,

I was able to successfully install Kong DB-less on GKE and configure ingresses, services, user and authorizations for servives running inside the GKE cluster.
Is it possible to configure a Ingress that routes to an external legacy service?
I tried the below example configuration borrowed from https://github.com/Kong/kubernetes-ingress-controller/issues/39 without success:

  • ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: proxy-from-k8s-to-mockbin
  namespace: kong
  annotations:
    request-transformer.plugin.konghq.com: |
      transform-request-to-mockbin
    jwt.plugin.konghq.com: |
      wechat-jwt
spec:
  rules:
  - host: foo.bar
    http:
      paths:
      - path: /request
        backend:
          serviceName: proxy-to-mockbin
          servicePort: 80
  • svc.yaml
kind: Service
apiVersion: v1
metadata:
  name: proxy-to-mockbin
  namespace: kong
spec:
  ports:
  - protocol: TCP
    port: 80
  type: ExternalName
  externalName: mockbin.org
  • request-transformer.yaml
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: transform-request-to-mockbin
  namespace: kong
config:
  remove:
    headers: host
```.
The foo.bar url is unreachable .
If I remove the 'foo.bar' host I get  {"message":"no Route matched with those values"}

Regards,
Souto

@Antonio_Souto What error did you run into? External name services are indeed supported by Kong’s Ingress Controller.