Adding a client cert to an upstream service

An upstream service is protected by certificate authentication and It expects a client certificare, What is the best way to add cert in kong when Kong forwards the request to the upstream service?

Hi @rsvenkatesh ,

As of today, the only way to make Kong use a client certificate when connecting to an upstream is to rely on the ngx_http_proxy_module’s proxy_ssl_certificate and friends directives.

If you are using Kong 0.14, you can make use of dynamically injected nginx directives to achieve this. Otherwise, you’ll have to roll-out your own custom nginx configuration template if you are using an older version of Kong.
The downside of this approach is that every upstream connection will use this certificate, and this isn’t configurable. You could duplicate the Kong location / block in a custom nginx configuration template to work around this, such that 2 location blocks are defined, both running Kong, but one of them sets a client certificate, and the other doesn’t.

In the future, we have plans to natively support dynamic client certificates, and related work is undergoing at this moment. Please stay tuned for a future release with this feature!

Best,

Hi @thibaultcha,
Thank you for your quick response, we are currently actively evaluating Kong and istio and I hope Kong will get this feature out soon, do you have any rough eta on the this feature rollout?

Thank you.
Venki

@rsvenkatesh A very rough ETA for this feature would be sometime in the next couple of months. Could be sooner. Pinging @James_Callahan into the conversation as well, as he is heavily involved in the groundwork for this feature.

@thibaultcha Thank you

@thibaultcha Hi … is dynamic client certificates supported in kong 1.3 ?

See: Kong upstream SSL mTLS - #15 by datong.sun , give upstream mSSL a try in Kong 1.4 . There were some fixes that went in (if your looking to present certs to help w mSSL handshakes that is). Otherwise 1.3 does have the client cert presentation indeed but with a caveat of not being able to validate the upstream certificate.

Hi @jeremyjpj0916

Please guide how to use certificate through “lua_ssl_trusted_certificate”. I have deployed to k8s through helm chart and I have .pem on my local machine.

I have added certificates using end point /certificate and then PATCH services with certificate ID. I have configured routes for this services. But still it give 503, service temporarily unavailable

My Use Case : my upstream server has certificate authentication. So I need to make call using client certificate. Hi @jeremyjpj0916 Please guide steps to achieve this.

Hi All,

Can someone please guide me on this?

I am using Kong 1.4.2 and one of my upstream server has certificate authentication. and I want to proxy using client certificate. How can I do this?
Steps I am following is :

  1. I have added .crt and .key using /certificate API
  2. I have configured upstream server with the certificate id created in first step
  3. I have configured route
    Now making call using routes, but getting 503 in return. Am I missing something? Does anyone know how to achieve this?

@kumarbijendra Are you using Kong or Kong Ingress Controller for this?

If Kong Ingress Controller, then you can’t do that yet. The support for it is coming in 0.7
You can use a build from the master branch to do this as well.

IF you are using Kong, then you need to specify the certificate’s ID in the Service entity in Kong. The service entity has client_certificate property for this.

hi @hbagdi I am using Kong. And, that’s what I explained - while configuring service I have actually specified client_certificate. Still I am getting 503. Do I need to set any environment variable ? or any other suggestions?

HTTP 503 is service unavailable and not an authentication problem.
You should make sure that the response is coming from upstream service or Kong and based on that do further debugging.

That’s all I can say based on the info you’ve provided.

Hi @hbagdi,

I am giving some information here:

GET call on /certificates :
{
“next”: null,
“data”: [
{
“created_at”: 1576440335,
“cert”: “-----BEGIN CERTIFICATE-----
\n-----END CERTIFICATE-----\n”,
“id”: “866814d8-e0ab-4353-a8db-00cff0b7da57”,
“tags”: null,
“key”: “-----BEGIN PRIVATE KEY-----
\n-----END PRIVATE KEY-----\n”,
“snis”: [
“some snis”
]
}
]
}

GET call on /services :
{
“next”: null,
“data”: [
{
“host”: “some-host”,
“created_at”: 1576440685,
“connect_timeout”: 60000,
“id”: “559723b4-1d97-4acb-8dbf-418abc6c4e8c”,
“protocol”: “https”,
“name”: “config-svc-health”,
“read_timeout”: 60000,
“port”: 443,
“path”: “/bifrost-configurator/api/health”,
“updated_at”: 1576441105,
“retries”: 5,
“write_timeout”: 60000,
“tags”: null,
“client_certificate”: {
“id”: “866814d8-e0ab-4353-a8db-00cff0b7da57”
}
}
]
}
GET On /routes
{
“id”: “0a081087-b68e-47e6-84b6-46945a535e66”,
“tags”: null,
“updated_at”: 1576444412,
“destinations”: null,
“headers”: null,
“protocols”: [
“http”,
“https”
],
“created_at”: 1576444412,
“snis”: null,
“service”: {
“id”: “559723b4-1d97-4acb-8dbf-418abc6c4e8c”
},
“name”: null,
“preserve_host”: false,
“regex_priority”: 0,
“strip_path”: true,
“sources”: null,
“paths”: [
“/config-svc-health”
],
“https_redirect_status_code”: 426,
“hosts”: [
“some host”
],
“methods”: null
}

curl -i -X GET https://host/kong/config-svc-health

HTTP/2 503
server: openresty/1.15.8.2
date: Mon, 16 Dec 2019 18:06:02 GMT
content-type: text/html; charset=UTF-8
content-length: 203
strict-transport-security: max-age=15724800; includeSubDomains
x-kong-upstream-latency: 11
x-kong-proxy-latency: 41
via: kong/1.4.2

503 Service Temporarily Unavailable

503 Service Temporarily Unavailable


openresty/1.15.8.2

Kong logs:
10.244.2.59 - - [16/Dec/2019:19:44:24 +0000] “GET /config-svc-health HTTP/1.1” 503 203 “-” “curl/7.61.1”

Please make sure that Kong can talk to the upstream server.

HTTP 503 means that your upstream service is not available. It doesn’t mean that Kong can’t authenticate itself.

Please make sure that Kong can at least connect to your upstream service.

Thanks, Harry! I tested with mock service with SSL authentication. It worked! Many thanks for your quick response and help.

I am using Kong Konnect to configure my Gateways running on Azure AKS. Everything works fine in terms of service and routes. I created a service to the backend IDRIX TLS Client Authentication Test. This API responds with the certificate details presented by the client. I created a certificate and uploaded it on Konnect under Certificates and updated the certificate ID under the client certificate in the Service. However, the certificate is not being sent to the API. Every time the API responds that no TLS Client certificate is sent. I used curl to reach the backend by providing the same certificate and the response works correctly.

I checked the pod logs and I see the error
invalid reference ‘client_certificate: {“id”:“59af4051-cada-4224-8418-60b252841959”}’ (no such entry in ‘certificates’), context: ngx.timer

the certificate with this id exists in the Certificates section on Konnect