Kong gateway doesn't start on GKE when using external PostgresDB instance

Questions
1

Hi,

I am trying to install kong/kong-gateway (tag 3.1) on GCP GKE K8s cluster with external PostgresDB instance running on GCP with public IP and all SSL settings off. I have bootstrapped Postgres database externally using Virtual machine installed with Kong 3.1 after I got error earlier. (Am I doing this wrong ?? )

Now that I try to install Kong, init pod completes with logs,

% kubectl logs -n kong kong-kong-init-migrations-lrmhc | tail -5
Database already bootstrapped
%

but kong pod itself doesn’t come up.
% kubectl get pods -n kong
NAME READY STATUS RESTARTS AGE
kong-kong-64b9984454-crmg4 1/2 CrashLoopBackOff 20 (64s ago) 79m
kong-kong-init-migrations-lrmhc 0/1 Completed 0 23h
kong-postgresql-0 0/1 Pending 0 23h

% kubectl logs -n kong kong-kong-64b9984454-crmg4 | tail -20
Defaulted container “ingress-controller” out of: ingress-controller, proxy, clear-stale-pid (init), wait-for-db (init)
time=“2023-03-26T08:11:22Z” level=info msg=“diagnostics server disabled”
time=“2023-03-26T08:11:22Z” level=info msg=“starting controller manager” commit=c6ee6c4828881b374925755f48284fb21f46ef8d logger=setup release=2.8.1 repo=“GitHub - Kong/kubernetes-ingress-controller: Kong for Kubernetes: The official Ingress Controller for Kubernetes.”
time=“2023-03-26T08:11:22Z” level=info msg=“getting enabled options and features” logger=setup
time=“2023-03-26T08:11:22Z” level=info msg=“getting the kubernetes client configuration” logger=setup
W0326 08:11:22.901115 1 client_config.go:618] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
time=“2023-03-26T08:11:22Z” level=info msg=“getting the kong admin api client configuration” logger=setup
Error: unable to build kong api client: looking up workspace: HTTP status 401 (message: “Invalid RBAC credentials”)
Error: unable to build kong api client: looking up workspace: HTTP status 401 (message: “Invalid RBAC credentials”)
%

Here is snippet from values.yml used while installing with Helm
…
pg_port: 5432
pg_database: kong
pg_host:
valueFrom:
secretKeyRef:
key: pg_host
name: kong-config-secret
pg_password:
valueFrom:
secretKeyRef:
key: pg_password
name: kong-config-secret
pg_ssl: “off”
pg_ssl_verify: “off”
pg_user: kong
…
…
postgresql:
enabled: false

auth:

username: kong

database: kong

…

Please help me with this setup.

Regards,
Prasad

2

Hi @hgprasad -

As per error message you mentioned “Invalid RBAC credentials” it looks like error is related to RBAC configurations in the values.yaml.

When we enable RBAC, in the below example it is basic auth, we have to mention below values in the yaml.

enterprise.rbac.enabled=true 
enterprise.rbac.admin_gui_auth=basic-auth 
enterprise.rbac.session_conf_secret=kong-session-config 

env.password.valueFrom.secretKeyRef.name=kong-enterprise-superuser-password 
env.password.valueFrom.secretKeyRef.key=password

Try with above and see if your error disappears.

3

I tried different approach, i deployed kong on k8s using Helm and inbuilt postgres DB subchart.
Now I am trying to connect to external PostgresDB instance but I get this error as external DB is not bootstrapped.
How do I bootstrap it ?

**kubectl logs -n kong quickstart-kong-pre-upgrade-migrations-9g5sq **
Error: Cannot run migrations: Database needs bootstrapping or is older than Kong 1.0.

To start a new installation from scratch, run ‘kong migrations bootstrap’.

**To migrate from a version older than 1.0, migrated to Kong 1.5.0 first. **
If you still have ‘apis’ entities, you can convert them to Routes and Services
using the ‘kong migrations migrate-apis’ command in Kong 1.5.0.

I know in docker env we do this, whats the alternate to k8s installation.
docker run --rm --name kong-boot *
** --network=kong *
** -e “KONG_DATABASE=postgres” **
** -e “KONG_PG_HOST=kong-database” **
** kong/kong-gateway:3.0.1.0 **
** kong migrations bootstrap**

thanks,
Prasad

4

Please refer attached file and run below command:

kubectl apply -f migrationFile.yml -n YOUR_NAMESPACE

image
image963×700 19.4 KB

5

Thanks for reply,

I did fresh helm install with external PostgresDB and it worked.