When creating HTTPS Kong Services and Routes via the Ingress Controller, my services are always defaulted to protocol=HTTP and port=80 in postgres.
However, the hostname is correctly configured as namespace.example-api.https.
Once I run this update in postgres, HTTPS works:
UPDATE services SET port = 80, protocol = 'https'
This is quite problematic for my use case, as we MUST use e2e TLS encryption, and any deployment requires a manual update to the postgres service records. This also has to be run after any k8 deployment to an existing k8 deployment resource, which leads me to believe that this is either related to certificates/TLS, or (more likely) kong’s parsing of ingress resources’ https configuration.
I’ve also tried using 443 rather than https as the value for servicePort, to no avail.
Here’s an example ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
namespace: example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: "kong"
configuration.konghq.com: example-kongingress
spec:
tls:
- hosts:
- "example.ingress.com"
secretName: example-cert
rules:
- host: "example.ingress.com"
http:
paths:
- path: "/v1/example"
backend:
serviceName: example-svc
servicePort: https
---
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
name: example-kongingress
namespace: example
route:
methods:
- POST
- PUT
- DELETE
- PATCH
- GET
- OPTIONS
protocols:
- https
- http
strip_path: true
preserve_host: true
---
Here’s the ingress controller log for this event (which doesn’t tell us much about what’s going on with the https configuration):
Event(v1.ObjectReference{Kind:"Ingress", Namespace:"example", Name:"example-ingress", UID:"some-uuid", APIVersion:"extensions", ResourceVersion:"2549274", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress example/example-ingress
and the resulting postgres service record:
I’ve dug into the go src for the kong ingress controller, and have found that this issue is probably occurring within
kubernetes-ingress-controller/internal/ingress/controller/store/store.go
ingEventHandler := cache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
ing := obj.(*extensions.Ingress)
if !class.IsValid(&ing.ObjectMeta) {
a, _ := parser.GetStringAnnotation(class.IngressKey, &ing.ObjectMeta)
glog.Infof("ignoring add for ingress %v based on annotation %v with value %v", ing.Name, class.IngressKey, a)
return
}
recorder.Eventf(ing, corev1.EventTypeNormal, "CREATE", fmt.Sprintf("Ingress %s/%s", ing.Namespace, ing.Name))
recorder.Eventf(ing, corev1.EventTypeNormal, "CREATE", fmt.Sprintf("Ingress %s/%s", ing.Namespace, ing.Name))
updateCh.In() <- Event{
Type: CreateEvent,
Obj: obj,
}
},
(skipping some irrelevant lines here...)
epEventHandler := cache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
updateCh.In() <- Event{
Type: CreateEvent,
Obj: obj,
}
},
DeleteFunc: func(obj interface{}) {
updateCh.In() <- Event{
Type: DeleteEvent,
Obj: obj,
}
},
UpdateFunc: func(old, cur interface{}) {
oep := old.(*corev1.Endpoints)
ocur := cur.(*corev1.Endpoints)
if !reflect.DeepEqual(ocur.Subsets, oep.Subsets) {
updateCh.In() <- Event{
Type: UpdateEvent,
Obj: cur,
}
}
},
}
serviceEventHandler := cache.ResourceEventHandlerFuncs{
UpdateFunc: func(old, cur interface{}) {
glog.Infof("UPDATING SERVICE, OBJECT IS:%+v\n", cur)
updateCh.In() <- Event{
Type: ConfigurationEvent,
Obj: cur,
}
},
DeleteFunc: func(obj interface{}) {
updateCh.In() <- Event{
Type: DeleteEvent,
Obj: obj,
}
},
}
I can see that we’re sending events to a go channel, and somewhere on the other end, that channel is being listened to, and updating postgres accordingly.
I can see that there is an add endpoint function, but only an update service function, and I believe that kong uses endpoints rather services to proxy requests to, but I’m a bit shaky on how that works.
I’m looking into rebuilding in the ingress controller image from scratch, and including better logging to view the data prior to being written to postgres.
In lieu of that, has anyone else run into this problem and found the root of the issue?
Thanks!
- Dylan
