Jwt and auth0 not working properly

Greetings,
I am trying to get Auth0 integration with the Kong Jwt plugin on Kubernetes. I have confirmed the plugin is operating and preventing requests without a token from being accepted, so that is good. However, when I pass a valid token (taken from the Auth0 API Explorer) I get a message: “No credentials found for given ‘iss’”.

Googling has yielded several examples and in each of them I appear to be doing this correct so I am not sure what is going on. Here is my resources definition:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: weather-jwt
  namespace: weather-api

consumers:
  - username: apiUser

jwt_secrets:
  - consumer: apiUser
    key: "https://mycompany.auth0.com/"
    algorithm: RS256
    rsa_public_key: |-
      -----BEGIN PUBLIC KEY-----
      -----END PUBLIC KEY-----
plugin: jwt

Nothing I google seems to indicate what is causing this issue. Thanks

Can you provide an example of one of the JWTs you’re trying to verify and instructions for how you obtain it. Essentially that means it couldn’t find https://mycompany.auth0.com/ (or some other JWT credential key) in the iss field of the JWT (as that’s the default key name)

https://github.com/Kong/kong/blob/2.0.4/kong/plugins/jwt/handler.lua#L157-L167 calls load_credential(CLAIM_VALUE) using the value of the key_claim_name claim (i.e. by default the value of iss).

That then runs a query that’s effectively SELECT FROM jwt_secrets WHERE key = 'CLAIM_VALUE' in https://github.com/Kong/kong/blob/2.0.4/kong/plugins/jwt/handler.lua#L70-L76

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IlFrVXdOREl4UmpsR1JEYzNSalU1UVVaRE9UVkRSalUyTURJek9FTkdNRFJCTlRBeVJFUkNOUSJ9.eyJpc3MiOiJodHRwczovL2ZhcnJlbGxzb2Z0LmF1dGgwLmNvbS8iLCJzdWIiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YkBjbGllbnRzIiwiYXVkIjoiaHR0cHM6Ly9mYXJyZWxsc29mdC5hdXRoMC5jb20vYXBpL3YyLyIsImlhdCI6MTU5MTY1NjA2NiwiZXhwIjoxNTkxNzQyNDY2LCJhenAiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YiIsInNjb3BlIjoicmVhZDpjbGllbnRfZ3JhbnRzIGNyZWF0ZTpjbGllbnRfZ3JhbnRzIGRlbGV0ZTpjbGllbnRfZ3JhbnRzIHVwZGF0ZTpjbGllbnRfZ3JhbnRzIHJlYWQ6dXNlcnMgdXBkYXRlOnVzZXJzIGRlbGV0ZTp1c2VycyBjcmVhdGU6dXNlcnMgcmVhZDp1c2Vyc19hcHBfbWV0YWRhdGEgdXBkYXRlOnVzZXJzX2FwcF9tZXRhZGF0YSBkZWxldGU6dXNlcnNfYXBwX21ldGFkYXRhIGNyZWF0ZTp1c2Vyc19hcHBfbWV0YWRhdGEgcmVhZDp1c2VyX2N1c3RvbV9ibG9ja3MgY3JlYXRlOnVzZXJfY3VzdG9tX2Jsb2NrcyBkZWxldGU6dXNlcl9jdXN0b21fYmxvY2tzIGNyZWF0ZTp1c2VyX3RpY2tldHMgcmVhZDpjbGllbnRzIHVwZGF0ZTpjbGllbnRzIGRlbGV0ZTpjbGllbnRzIGNyZWF0ZTpjbGllbnRzIHJlYWQ6Y2xpZW50X2tleXMgdXBkYXRlOmNsaWVudF9rZXlzIGRlbGV0ZTpjbGllbnRfa2V5cyBjcmVhdGU6Y2xpZW50X2tleXMgcmVhZDpjb25uZWN0aW9ucyB1cGRhdGU6Y29ubmVjdGlvbnMgZGVsZXRlOmNvbm5lY3Rpb25zIGNyZWF0ZTpjb25uZWN0aW9ucyByZWFkOnJlc291cmNlX3NlcnZlcnMgdXBkYXRlOnJlc291cmNlX3NlcnZlcnMgZGVsZXRlOnJlc291cmNlX3NlcnZlcnMgY3JlYXRlOnJlc291cmNlX3NlcnZlcnMgcmVhZDpkZXZpY2VfY3JlZGVudGlhbHMgdXBkYXRlOmRldmljZV9jcmVkZW50aWFscyBkZWxldGU6ZGV2aWNlX2NyZWRlbnRpYWxzIGNyZWF0ZTpkZXZpY2VfY3JlZGVudGlhbHMgcmVhZDpydWxlcyB1cGRhdGU6cnVsZXMgZGVsZXRlOnJ1bGVzIGNyZWF0ZTpydWxlcyByZWFkOnJ1bGVzX2NvbmZpZ3MgdXBkYXRlOnJ1bGVzX2NvbmZpZ3MgZGVsZXRlOnJ1bGVzX2NvbmZpZ3MgcmVhZDpob29rcyB1cGRhdGU6aG9va3MgZGVsZXRlOmhvb2tzIGNyZWF0ZTpob29rcyByZWFkOmFjdGlvbnMgdXBkYXRlOmFjdGlvbnMgZGVsZXRlOmFjdGlvbnMgY3JlYXRlOmFjdGlvbnMgcmVhZDplbWFpbF9wcm92aWRlciB1cGRhdGU6ZW1haWxfcHJvdmlkZXIgZGVsZXRlOmVtYWlsX3Byb3ZpZGVyIGNyZWF0ZTplbWFpbF9wcm92aWRlciBibGFja2xpc3Q6dG9rZW5zIHJlYWQ6c3RhdHMgcmVhZDp0ZW5hbnRfc2V0dGluZ3MgdXBkYXRlOnRlbmFudF9zZXR0aW5ncyByZWFkOmxvZ3MgcmVhZDpzaGllbGRzIGNyZWF0ZTpzaGllbGRzIHVwZGF0ZTpzaGllbGRzIGRlbGV0ZTpzaGllbGRzIHJlYWQ6YW5vbWFseV9ibG9ja3MgZGVsZXRlOmFub21hbHlfYmxvY2tzIHVwZGF0ZTp0cmlnZ2VycyByZWFkOnRyaWdnZXJzIHJlYWQ6Z3JhbnRzIGRlbGV0ZTpncmFudHMgcmVhZDpndWFyZGlhbl9mYWN0b3JzIHVwZGF0ZTpndWFyZGlhbl9mYWN0b3JzIHJlYWQ6Z3VhcmRpYW5fZW5yb2xsbWVudHMgZGVsZXRlOmd1YXJkaWFuX2Vucm9sbG1lbnRzIGNyZWF0ZTpndWFyZGlhbl9lbnJvbGxtZW50X3RpY2tldHMgcmVhZDp1c2VyX2lkcF90b2tlbnMgY3JlYXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgZGVsZXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgcmVhZDpjdXN0b21fZG9tYWlucyBkZWxldGU6Y3VzdG9tX2RvbWFpbnMgY3JlYXRlOmN1c3RvbV9kb21haW5zIHVwZGF0ZTpjdXN0b21fZG9tYWlucyByZWFkOmVtYWlsX3RlbXBsYXRlcyBjcmVhdGU6ZW1haWxfdGVtcGxhdGVzIHVwZGF0ZTplbWFpbF90ZW1wbGF0ZXMgcmVhZDptZmFfcG9saWNpZXMgdXBkYXRlOm1mYV9wb2xpY2llcyByZWFkOnJvbGVzIGNyZWF0ZTpyb2xlcyBkZWxldGU6cm9sZXMgdXBkYXRlOnJvbGVzIHJlYWQ6cHJvbXB0cyB1cGRhdGU6cHJvbXB0cyByZWFkOmJyYW5kaW5nIHVwZGF0ZTpicmFuZGluZyBkZWxldGU6YnJhbmRpbmcgcmVhZDpsb2dfc3RyZWFtcyBjcmVhdGU6bG9nX3N0cmVhbXMgZGVsZXRlOmxvZ19zdHJlYW1zIHVwZGF0ZTpsb2dfc3RyZWFtcyBjcmVhdGU6c2lnbmluZ19rZXlzIHJlYWQ6c2lnbmluZ19rZXlzIHVwZGF0ZTpzaWduaW5nX2tleXMgcmVhZDpsaW1pdHMgdXBkYXRlOmxpbWl0cyIsImd0eSI6ImNsaWVudC1jcmVkZW50aWFscyJ9.A8BWCihsazeHasnD2KeP8YNO08uwhjZzo2yxvGLH1CGS9VymjtroGAM4ZFo4XELjZnobK6WzplGiYDUZz9H1m8A5jRWOfPbixXOG69pLUYka5NZR153F-EmqrXQ81vM25kEPsyHLJBqadvaDow-XOAN_t8KbTuetjP0e7ya8C8JY30uLWgYrQODwhQK03qu0P9hkutknVvKIzQChKL8l0b-pThJMu65-6b5yUr57giVITXtljshXFiUsYTRTsi_6Sgun70RyTTYmpx7DgjWif_mlra2bpSpHnJFVT8gSPkj2bntwiq3KzFicP4yld4OqD0Rc0DnknFRr3emINud4-Q

This token is taken from the API Explorer page in Auth0. I have also tried a token generated from a login in an Angular application. Both resulted in the same error message.

My guess is that I am not specifying jwt_secrets properly. There is no example of using them in a jwt_plugin so I am wondering if I do need to create a KongConsumer. Right now I have only defined a KongPlugin and Ingress

Please advise.
Thanks for your response

Here is example with DB:

  1. Create Consumer
$ http put :8001/consumers/apiUser
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 119
Content-Type: application/json; charset=utf-8
Date: Tue, 09 Jun 2020 10:49:50 GMT
Server: kong/2.0.4
X-Kong-Admin-Latency: 261

{
    "created_at": 1591699790,
    "custom_id": null,
    "id": "6ad39431-9f2f-4f49-9624-6c8c4ea36af2",
    "tags": null,
    "username": "apiUser"
}
  1. Create JWT for Consumer:
$ http post :8001/consumers/apiUser/jwt algorithm=RS256 key=https://farrellsoft.auth0.com/ rsa_public_key@a.pem -f
HTTP/1.1 201 Created
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 728
Content-Type: application/json; charset=utf-8
Date: Tue, 09 Jun 2020 10:55:42 GMT
Server: kong/2.0.4
X-Kong-Admin-Latency: 13

{
    "algorithm": "RS256",
    "consumer": {
        "id": "6ad39431-9f2f-4f49-9624-6c8c4ea36af2"
    },
    "created_at": 1591700142,
    "id": "bb74037e-3919-4224-bb2b-c971173304a8",
    "key": "https://farrellsoft.auth0.com/",
    "rsa_public_key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3PYgeiVyURLhqAkkUOfL\nroY281upGVWgBTZKZu6rIMPCiyzuZU8Rnlc1k+cHkbov0uRZIVmwrhMLTr6E9ZwD\nF2S2CY/K9yo7ZfSc1nc2uHYHJWkPBDKzjwPhkMhBKZ5hS1PXWQpgLdgDo4OrnscS\nfWTE1V82Mxv43LF4z32XiVtc5+vR59srgScQRXxQ4ghe35oCEtzr72LN08ypVKFm\nN5aARF1ifBUuom8SiyWJoL8cNfMre0wWNG23M2QC1fUtHPuob6K+Wnwyd//Re0n3\nAqyHfN71b+/pV5xDTde005nl08WU2g64D6LosH9TGk0hBNsj1u3mZNAPGdcVHY8J\nNQIDAQAB\n-----END PUBLIC KEY-----\n",
    "secret": "w0R5MVu5yRo7OyggrDBqh74LZDXbsi0O",
    "tags": null
}
  1. Create Service:
$ http put :8001/services/bin url=http://bin.test/anything
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 289
Content-Type: application/json; charset=utf-8
Date: Tue, 09 Jun 2020 10:56:18 GMT
Server: kong/2.0.4
X-Kong-Admin-Latency: 4

{
    "client_certificate": null,
    "connect_timeout": 60000,
    "created_at": 1591700178,
    "host": "bin.test",
    "id": "69f7ca9b-4f83-442c-bc15-8a59c2e882d3",
    "name": "bin",
    "path": "/anything",
    "port": 80,
    "protocol": "http",
    "read_timeout": 60000,
    "retries": 5,
    "tags": null,
    "updated_at": 1591700178,
    "write_timeout": 60000
}
  1. Create Route:
$ http put  :8001/services/bin/routes/bin paths=/ -f
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 421
Content-Type: application/json; charset=utf-8
Date: Tue, 09 Jun 2020 10:56:41 GMT
Server: kong/2.0.4
X-Kong-Admin-Latency: 5

{
    "created_at": 1591700201,
    "destinations": null,
    "headers": null,
    "hosts": null,
    "https_redirect_status_code": 426,
    "id": "8f5f179d-b57d-4f6f-afcb-b22e5862e620",
    "methods": null,
    "name": "bin",
    "path_handling": "v0",
    "paths": [
        "/"
    ],
    "preserve_host": false,
    "protocols": [
        "http",
        "https"
    ],
    "regex_priority": 0,
    "service": {
        "id": "69f7ca9b-4f83-442c-bc15-8a59c2e882d3"
    },
    "snis": null,
    "sources": null,
    "strip_path": true,
    "tags": null,
    "updated_at": 1591700201
}
  1. Apply JWT Plugin to Service
$ http post :8001/services/bin/plugins name=jwt -f
HTTP/1.1 201 Created
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 462
Content-Type: application/json; charset=utf-8
Date: Tue, 09 Jun 2020 10:57:11 GMT
Server: kong/2.0.4
X-Kong-Admin-Latency: 5

{
    "config": {
        "anonymous": null,
        "claims_to_verify": null,
        "cookie_names": [],
        "header_names": [
            "authorization"
        ],
        "key_claim_name": "iss",
        "maximum_expiration": 0,
        "run_on_preflight": true,
        "secret_is_base64": false,
        "uri_param_names": [
            "jwt"
        ]
    },
    "consumer": null,
    "created_at": 1591700231,
    "enabled": true,
    "id": "b6014f1d-04e3-4f61-973a-073da7bc3ece",
    "name": "jwt",
    "protocols": [
        "grpc",
        "grpcs",
        "http",
        "https"
    ],
    "route": null,
    "service": {
        "id": "69f7ca9b-4f83-442c-bc15-8a59c2e882d3"
    },
    "tags": null
}
  1. Call without Token
$ http :8000
HTTP/1.1 401 Unauthorized
Connection: keep-alive
Content-Length: 26
Content-Type: application/json; charset=utf-8
Date: Tue, 09 Jun 2020 10:57:15 GMT
Server: kong/2.0.4
X-Kong-Response-Latency: 1

{
    "message": "Unauthorized"
}
  1. Call with Token that you provided:
$ http :8000 Authorization:"Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IlFrVXdOREl4UmpsR1JEYzNSalU1UVVaRE9UVkRSalUyTURJek9FTkdNRFJCTlRBeVJFUkNOUSJ9.eyJpc3MiOiJodHRwczovL2ZhcnJlbGxzb2Z0LmF1dGgwLmNvbS8iLCJzdWIiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YkBjbGllbnRzIiwiYXVkIjoiaHR0cHM6Ly9mYXJyZWxsc29mdC5hdXRoMC5jb20vYXBpL3YyLyIsImlhdCI6MTU5MTY1NjA2NiwiZXhwIjoxNTkxNzQyNDY2LCJhenAiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YiIsInNjb3BlIjoicmVhZDpjbGllbnRfZ3JhbnRzIGNyZWF0ZTpjbGllbnRfZ3JhbnRzIGRlbGV0ZTpjbGllbnRfZ3JhbnRzIHVwZGF0ZTpjbGllbnRfZ3JhbnRzIHJlYWQ6dXNlcnMgdXBkYXRlOnVzZXJzIGRlbGV0ZTp1c2VycyBjcmVhdGU6dXNlcnMgcmVhZDp1c2Vyc19hcHBfbWV0YWRhdGEgdXBkYXRlOnVzZXJzX2FwcF9tZXRhZGF0YSBkZWxldGU6dXNlcnNfYXBwX21ldGFkYXRhIGNyZWF0ZTp1c2Vyc19hcHBfbWV0YWRhdGEgcmVhZDp1c2VyX2N1c3RvbV9ibG9ja3MgY3JlYXRlOnVzZXJfY3VzdG9tX2Jsb2NrcyBkZWxldGU6dXNlcl9jdXN0b21fYmxvY2tzIGNyZWF0ZTp1c2VyX3RpY2tldHMgcmVhZDpjbGllbnRzIHVwZGF0ZTpjbGllbnRzIGRlbGV0ZTpjbGllbnRzIGNyZWF0ZTpjbGllbnRzIHJlYWQ6Y2xpZW50X2tleXMgdXBkYXRlOmNsaWVudF9rZXlzIGRlbGV0ZTpjbGllbnRfa2V5cyBjcmVhdGU6Y2xpZW50X2tleXMgcmVhZDpjb25uZWN0aW9ucyB1cGRhdGU6Y29ubmVjdGlvbnMgZGVsZXRlOmNvbm5lY3Rpb25zIGNyZWF0ZTpjb25uZWN0aW9ucyByZWFkOnJlc291cmNlX3NlcnZlcnMgdXBkYXRlOnJlc291cmNlX3NlcnZlcnMgZGVsZXRlOnJlc291cmNlX3NlcnZlcnMgY3JlYXRlOnJlc291cmNlX3NlcnZlcnMgcmVhZDpkZXZpY2VfY3JlZGVudGlhbHMgdXBkYXRlOmRldmljZV9jcmVkZW50aWFscyBkZWxldGU6ZGV2aWNlX2NyZWRlbnRpYWxzIGNyZWF0ZTpkZXZpY2VfY3JlZGVudGlhbHMgcmVhZDpydWxlcyB1cGRhdGU6cnVsZXMgZGVsZXRlOnJ1bGVzIGNyZWF0ZTpydWxlcyByZWFkOnJ1bGVzX2NvbmZpZ3MgdXBkYXRlOnJ1bGVzX2NvbmZpZ3MgZGVsZXRlOnJ1bGVzX2NvbmZpZ3MgcmVhZDpob29rcyB1cGRhdGU6aG9va3MgZGVsZXRlOmhvb2tzIGNyZWF0ZTpob29rcyByZWFkOmFjdGlvbnMgdXBkYXRlOmFjdGlvbnMgZGVsZXRlOmFjdGlvbnMgY3JlYXRlOmFjdGlvbnMgcmVhZDplbWFpbF9wcm92aWRlciB1cGRhdGU6ZW1haWxfcHJvdmlkZXIgZGVsZXRlOmVtYWlsX3Byb3ZpZGVyIGNyZWF0ZTplbWFpbF9wcm92aWRlciBibGFja2xpc3Q6dG9rZW5zIHJlYWQ6c3RhdHMgcmVhZDp0ZW5hbnRfc2V0dGluZ3MgdXBkYXRlOnRlbmFudF9zZXR0aW5ncyByZWFkOmxvZ3MgcmVhZDpzaGllbGRzIGNyZWF0ZTpzaGllbGRzIHVwZGF0ZTpzaGllbGRzIGRlbGV0ZTpzaGllbGRzIHJlYWQ6YW5vbWFseV9ibG9ja3MgZGVsZXRlOmFub21hbHlfYmxvY2tzIHVwZGF0ZTp0cmlnZ2VycyByZWFkOnRyaWdnZXJzIHJlYWQ6Z3JhbnRzIGRlbGV0ZTpncmFudHMgcmVhZDpndWFyZGlhbl9mYWN0b3JzIHVwZGF0ZTpndWFyZGlhbl9mYWN0b3JzIHJlYWQ6Z3VhcmRpYW5fZW5yb2xsbWVudHMgZGVsZXRlOmd1YXJkaWFuX2Vucm9sbG1lbnRzIGNyZWF0ZTpndWFyZGlhbl9lbnJvbGxtZW50X3RpY2tldHMgcmVhZDp1c2VyX2lkcF90b2tlbnMgY3JlYXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgZGVsZXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgcmVhZDpjdXN0b21fZG9tYWlucyBkZWxldGU6Y3VzdG9tX2RvbWFpbnMgY3JlYXRlOmN1c3RvbV9kb21haW5zIHVwZGF0ZTpjdXN0b21fZG9tYWlucyByZWFkOmVtYWlsX3RlbXBsYXRlcyBjcmVhdGU6ZW1haWxfdGVtcGxhdGVzIHVwZGF0ZTplbWFpbF90ZW1wbGF0ZXMgcmVhZDptZmFfcG9saWNpZXMgdXBkYXRlOm1mYV9wb2xpY2llcyByZWFkOnJvbGVzIGNyZWF0ZTpyb2xlcyBkZWxldGU6cm9sZXMgdXBkYXRlOnJvbGVzIHJlYWQ6cHJvbXB0cyB1cGRhdGU6cHJvbXB0cyByZWFkOmJyYW5kaW5nIHVwZGF0ZTpicmFuZGluZyBkZWxldGU6YnJhbmRpbmcgcmVhZDpsb2dfc3RyZWFtcyBjcmVhdGU6bG9nX3N0cmVhbXMgZGVsZXRlOmxvZ19zdHJlYW1zIHVwZGF0ZTpsb2dfc3RyZWFtcyBjcmVhdGU6c2lnbmluZ19rZXlzIHJlYWQ6c2lnbmluZ19rZXlzIHVwZGF0ZTpzaWduaW5nX2tleXMgcmVhZDpsaW1pdHMgdXBkYXRlOmxpbWl0cyIsImd0eSI6ImNsaWVudC1jcmVkZW50aWFscyJ9.A8BWCihsazeHasnD2KeP8YNO08uwhjZzo2yxvGLH1CGS9VymjtroGAM4ZFo4XELjZnobK6WzplGiYDUZz9H1m8A5jRWOfPbixXOG69pLUYka5NZR153F-EmqrXQ81vM25kEPsyHLJBqadvaDow-XOAN_t8KbTuetjP0e7ya8C8JY30uLWgYrQODwhQK03qu0P9hkutknVvKIzQChKL8l0b-pThJMu65-6b5yUr57giVITXtljshXFiUsYTRTsi_6Sgun70RyTTYmpx7DgjWif_mlra2bpSpHnJFVT8gSPkj2bntwiq3KzFicP4yld4OqD0Rc0DnknFRr3emINud4-Q"
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 3990
Content-Type: application/json
Date: Tue, 09 Jun 2020 10:57:39 GMT
Server: gunicorn/19.9.0
Via: kong/2.0.4
X-Kong-Proxy-Latency: 15
X-Kong-Upstream-Latency: 7

{
    "args": {},
    "data": "",
    "files": {},
    "form": {},
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate",
        "Authorization": "Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IlFrVXdOREl4UmpsR1JEYzNSalU1UVVaRE9UVkRSalUyTURJek9FTkdNRFJCTlRBeVJFUkNOUSJ9.eyJpc3MiOiJodHRwczovL2ZhcnJlbGxzb2Z0LmF1dGgwLmNvbS8iLCJzdWIiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YkBjbGllbnRzIiwiYXVkIjoiaHR0cHM6Ly9mYXJyZWxsc29mdC5hdXRoMC5jb20vYXBpL3YyLyIsImlhdCI6MTU5MTY1NjA2NiwiZXhwIjoxNTkxNzQyNDY2LCJhenAiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YiIsInNjb3BlIjoicmVhZDpjbGllbnRfZ3JhbnRzIGNyZWF0ZTpjbGllbnRfZ3JhbnRzIGRlbGV0ZTpjbGllbnRfZ3JhbnRzIHVwZGF0ZTpjbGllbnRfZ3JhbnRzIHJlYWQ6dXNlcnMgdXBkYXRlOnVzZXJzIGRlbGV0ZTp1c2VycyBjcmVhdGU6dXNlcnMgcmVhZDp1c2Vyc19hcHBfbWV0YWRhdGEgdXBkYXRlOnVzZXJzX2FwcF9tZXRhZGF0YSBkZWxldGU6dXNlcnNfYXBwX21ldGFkYXRhIGNyZWF0ZTp1c2Vyc19hcHBfbWV0YWRhdGEgcmVhZDp1c2VyX2N1c3RvbV9ibG9ja3MgY3JlYXRlOnVzZXJfY3VzdG9tX2Jsb2NrcyBkZWxldGU6dXNlcl9jdXN0b21fYmxvY2tzIGNyZWF0ZTp1c2VyX3RpY2tldHMgcmVhZDpjbGllbnRzIHVwZGF0ZTpjbGllbnRzIGRlbGV0ZTpjbGllbnRzIGNyZWF0ZTpjbGllbnRzIHJlYWQ6Y2xpZW50X2tleXMgdXBkYXRlOmNsaWVudF9rZXlzIGRlbGV0ZTpjbGllbnRfa2V5cyBjcmVhdGU6Y2xpZW50X2tleXMgcmVhZDpjb25uZWN0aW9ucyB1cGRhdGU6Y29ubmVjdGlvbnMgZGVsZXRlOmNvbm5lY3Rpb25zIGNyZWF0ZTpjb25uZWN0aW9ucyByZWFkOnJlc291cmNlX3NlcnZlcnMgdXBkYXRlOnJlc291cmNlX3NlcnZlcnMgZGVsZXRlOnJlc291cmNlX3NlcnZlcnMgY3JlYXRlOnJlc291cmNlX3NlcnZlcnMgcmVhZDpkZXZpY2VfY3JlZGVudGlhbHMgdXBkYXRlOmRldmljZV9jcmVkZW50aWFscyBkZWxldGU6ZGV2aWNlX2NyZWRlbnRpYWxzIGNyZWF0ZTpkZXZpY2VfY3JlZGVudGlhbHMgcmVhZDpydWxlcyB1cGRhdGU6cnVsZXMgZGVsZXRlOnJ1bGVzIGNyZWF0ZTpydWxlcyByZWFkOnJ1bGVzX2NvbmZpZ3MgdXBkYXRlOnJ1bGVzX2NvbmZpZ3MgZGVsZXRlOnJ1bGVzX2NvbmZpZ3MgcmVhZDpob29rcyB1cGRhdGU6aG9va3MgZGVsZXRlOmhvb2tzIGNyZWF0ZTpob29rcyByZWFkOmFjdGlvbnMgdXBkYXRlOmFjdGlvbnMgZGVsZXRlOmFjdGlvbnMgY3JlYXRlOmFjdGlvbnMgcmVhZDplbWFpbF9wcm92aWRlciB1cGRhdGU6ZW1haWxfcHJvdmlkZXIgZGVsZXRlOmVtYWlsX3Byb3ZpZGVyIGNyZWF0ZTplbWFpbF9wcm92aWRlciBibGFja2xpc3Q6dG9rZW5zIHJlYWQ6c3RhdHMgcmVhZDp0ZW5hbnRfc2V0dGluZ3MgdXBkYXRlOnRlbmFudF9zZXR0aW5ncyByZWFkOmxvZ3MgcmVhZDpzaGllbGRzIGNyZWF0ZTpzaGllbGRzIHVwZGF0ZTpzaGllbGRzIGRlbGV0ZTpzaGllbGRzIHJlYWQ6YW5vbWFseV9ibG9ja3MgZGVsZXRlOmFub21hbHlfYmxvY2tzIHVwZGF0ZTp0cmlnZ2VycyByZWFkOnRyaWdnZXJzIHJlYWQ6Z3JhbnRzIGRlbGV0ZTpncmFudHMgcmVhZDpndWFyZGlhbl9mYWN0b3JzIHVwZGF0ZTpndWFyZGlhbl9mYWN0b3JzIHJlYWQ6Z3VhcmRpYW5fZW5yb2xsbWVudHMgZGVsZXRlOmd1YXJkaWFuX2Vucm9sbG1lbnRzIGNyZWF0ZTpndWFyZGlhbl9lbnJvbGxtZW50X3RpY2tldHMgcmVhZDp1c2VyX2lkcF90b2tlbnMgY3JlYXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgZGVsZXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgcmVhZDpjdXN0b21fZG9tYWlucyBkZWxldGU6Y3VzdG9tX2RvbWFpbnMgY3JlYXRlOmN1c3RvbV9kb21haW5zIHVwZGF0ZTpjdXN0b21fZG9tYWlucyByZWFkOmVtYWlsX3RlbXBsYXRlcyBjcmVhdGU6ZW1haWxfdGVtcGxhdGVzIHVwZGF0ZTplbWFpbF90ZW1wbGF0ZXMgcmVhZDptZmFfcG9saWNpZXMgdXBkYXRlOm1mYV9wb2xpY2llcyByZWFkOnJvbGVzIGNyZWF0ZTpyb2xlcyBkZWxldGU6cm9sZXMgdXBkYXRlOnJvbGVzIHJlYWQ6cHJvbXB0cyB1cGRhdGU6cHJvbXB0cyByZWFkOmJyYW5kaW5nIHVwZGF0ZTpicmFuZGluZyBkZWxldGU6YnJhbmRpbmcgcmVhZDpsb2dfc3RyZWFtcyBjcmVhdGU6bG9nX3N0cmVhbXMgZGVsZXRlOmxvZ19zdHJlYW1zIHVwZGF0ZTpsb2dfc3RyZWFtcyBjcmVhdGU6c2lnbmluZ19rZXlzIHJlYWQ6c2lnbmluZ19rZXlzIHVwZGF0ZTpzaWduaW5nX2tleXMgcmVhZDpsaW1pdHMgdXBkYXRlOmxpbWl0cyIsImd0eSI6ImNsaWVudC1jcmVkZW50aWFscyJ9.A8BWCihsazeHasnD2KeP8YNO08uwhjZzo2yxvGLH1CGS9VymjtroGAM4ZFo4XELjZnobK6WzplGiYDUZz9H1m8A5jRWOfPbixXOG69pLUYka5NZR153F-EmqrXQ81vM25kEPsyHLJBqadvaDow-XOAN_t8KbTuetjP0e7ya8C8JY30uLWgYrQODwhQK03qu0P9hkutknVvKIzQChKL8l0b-pThJMu65-6b5yUr57giVITXtljshXFiUsYTRTsi_6Sgun70RyTTYmpx7DgjWif_mlra2bpSpHnJFVT8gSPkj2bntwiq3KzFicP4yld4OqD0Rc0DnknFRr3emINud4-Q",
        "Connection": "keep-alive",
        "Host": "bin.test",
        "User-Agent": "HTTPie/2.1.0",
        "X-Consumer-Id": "6ad39431-9f2f-4f49-9624-6c8c4ea36af2",
        "X-Consumer-Username": "apiUser",
        "X-Credential-Identifier": "https://farrellsoft.auth0.com/",
        "X-Forwarded-Host": "localhost"
    },
    "json": null,
    "method": "GET",
    "origin": "127.0.0.1",
    "url": "http://localhost/anything"
}

Seems to be working just fine.

Thank you so much for your reply, it pointed me in the general direction. You were right, the iss error was due to the fact that the credentials were never being created with the YAML I provided. It turns out that I needed to use kong specific Yaml (file with _formatVersion) and pass it through /config on the Admin API - not quite what I was hoping for since I started the process with the aim to use the custom CRDs and Kubernetes Resources to execute my operation - it would seem this is not possible, I will explain.

Per the documentation:

Calls to /config for db-less wipe out all residing Kong objects in memory and replace them with the resources represented in the incoming file

This is key because the way Kong interacts with Kubernetes is it monitors new resources and “syncs” them. So, when an Ingress resource is created, under the hood Kong will create a corresponding KongIngress.

Since, the only way to get the consumer and its credentials into Kong is via /config this operation wipes out the created KongIngress created via the method above. Following the operation, I can see the consumers and credentials via the Admin API but since the KongIngress is wiped out Kong cannot route traffic.

If I recreate the Ingress, it seems to tell Kong to reset the in-memory store. I can route again but the Admin API reveals the consumer and its credentials have been removed.

What I end up with is a chicken and egg problem that seems like it will be solved by specifying KongIngress manually in my YAML file rather than allowing Kong to create it from Ingress.

The goal of my experiment was to leverage existing Kubernetes resources (Service, Ingress) and still get the intended functionality. While this worked with the Rate Limit plugin it does not work with Jwt verification operation, though I can get the functionality to check the token is present.

Tonight I will try with KongIngress, I expect it will work however, it means that my overall experiment is partially a failure.

Thanks for the reply, unfortunately, I was aiming to avoid using the Admin API in this way for my experiment. For this reason, I intentionally did not leverage this approach during my experimentation.

Cheers.

I should work the same without db.

Here is the declarative that I exported:

_transform: false
_format_version: '2.1'
consumers:
- created_at: 1591699790
  id: 6ad39431-9f2f-4f49-9624-6c8c4ea36af2
  username: apiUser
jwt_secrets:
- rsa_public_key: |
    -----BEGIN PUBLIC KEY-----
    MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3PYgeiVyURLhqAkkUOfL
    roY281upGVWgBTZKZu6rIMPCiyzuZU8Rnlc1k+cHkbov0uRZIVmwrhMLTr6E9ZwD
    F2S2CY/K9yo7ZfSc1nc2uHYHJWkPBDKzjwPhkMhBKZ5hS1PXWQpgLdgDo4OrnscS
    fWTE1V82Mxv43LF4z32XiVtc5+vR59srgScQRXxQ4ghe35oCEtzr72LN08ypVKFm
    N5aARF1ifBUuom8SiyWJoL8cNfMre0wWNG23M2QC1fUtHPuob6K+Wnwyd//Re0n3
    AqyHfN71b+/pV5xDTde005nl08WU2g64D6LosH9TGk0hBNsj1u3mZNAPGdcVHY8J
    NQIDAQAB
    -----END PUBLIC KEY-----
  created_at: 1591700142
  consumer: 6ad39431-9f2f-4f49-9624-6c8c4ea36af2
  id: bb74037e-3919-4224-bb2b-c971173304a8
  key: https://farrellsoft.auth0.com/
  secret: w0R5MVu5yRo7OyggrDBqh74LZDXbsi0O
  algorithm: RS256
services:
- host: bin.test
  created_at: 1591700178
  connect_timeout: 60000
  id: 69f7ca9b-4f83-442c-bc15-8a59c2e882d3
  protocol: http
  name: bin
  read_timeout: 60000
  port: 80
  path: /anything
  updated_at: 1591700178
  retries: 5
  write_timeout: 60000
routes:
- id: 8f5f179d-b57d-4f6f-afcb-b22e5862e620
  path_handling: v0
  paths:
  - /
  protocols:
  - http
  - https
  service: 69f7ca9b-4f83-442c-bc15-8a59c2e882d3
  name: bin
  strip_path: true
  preserve_host: false
  regex_priority: 0
  updated_at: 1591700201
  https_redirect_status_code: 426
  created_at: 1591700201
plugins:
- created_at: 1591700231
  config:
    secret_is_base64: false
    run_on_preflight: true
    uri_param_names:
    - jwt
    key_claim_name: iss
    header_names:
    - authorization
    maximum_expiration: 0
    cookie_names: {}
  id: b6014f1d-04e3-4f61-973a-073da7bc3ece
  service: 69f7ca9b-4f83-442c-bc15-8a59c2e882d3
  enabled: true
  protocols:
  - grpc
  - grpcs
  - http
  - https
  name: jwt

Then:

KONG_DECLARATIVE_CONFIG=<above.yml> KONG_DATABASE=off ./bin/kong start

Then:

$ http :8000 Authorization:"Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IlFrVXdOREl4UmpsR1JEYzNSalU1UVVaRE9UVkRSalUyTURJek9FTkdNRFJCTlRBeVJFUkNOUSJ9.eyJpc3MiOiJodHRwczovL2ZhcnJlbGxzb2Z0LmF1dGgwLmNvbS8iLCJzdWIiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YkBjbGllbnRzIiwiYXVkIjoiaHR0cHM6Ly9mYXJyZWxsc29mdC5hdXRoMC5jb20vYXBpL3YyLyIsImlhdCI6MTU5MTY1NjA2NiwiZXhwIjoxNTkxNzQyNDY2LCJhenAiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YiIsInNjb3BlIjoicmVhZDpjbGllbnRfZ3JhbnRzIGNyZWF0ZTpjbGllbnRfZ3JhbnRzIGRlbGV0ZTpjbGllbnRfZ3JhbnRzIHVwZGF0ZTpjbGllbnRfZ3JhbnRzIHJlYWQ6dXNlcnMgdXBkYXRlOnVzZXJzIGRlbGV0ZTp1c2VycyBjcmVhdGU6dXNlcnMgcmVhZDp1c2Vyc19hcHBfbWV0YWRhdGEgdXBkYXRlOnVzZXJzX2FwcF9tZXRhZGF0YSBkZWxldGU6dXNlcnNfYXBwX21ldGFkYXRhIGNyZWF0ZTp1c2Vyc19hcHBfbWV0YWRhdGEgcmVhZDp1c2VyX2N1c3RvbV9ibG9ja3MgY3JlYXRlOnVzZXJfY3VzdG9tX2Jsb2NrcyBkZWxldGU6dXNlcl9jdXN0b21fYmxvY2tzIGNyZWF0ZTp1c2VyX3RpY2tldHMgcmVhZDpjbGllbnRzIHVwZGF0ZTpjbGllbnRzIGRlbGV0ZTpjbGllbnRzIGNyZWF0ZTpjbGllbnRzIHJlYWQ6Y2xpZW50X2tleXMgdXBkYXRlOmNsaWVudF9rZXlzIGRlbGV0ZTpjbGllbnRfa2V5cyBjcmVhdGU6Y2xpZW50X2tleXMgcmVhZDpjb25uZWN0aW9ucyB1cGRhdGU6Y29ubmVjdGlvbnMgZGVsZXRlOmNvbm5lY3Rpb25zIGNyZWF0ZTpjb25uZWN0aW9ucyByZWFkOnJlc291cmNlX3NlcnZlcnMgdXBkYXRlOnJlc291cmNlX3NlcnZlcnMgZGVsZXRlOnJlc291cmNlX3NlcnZlcnMgY3JlYXRlOnJlc291cmNlX3NlcnZlcnMgcmVhZDpkZXZpY2VfY3JlZGVudGlhbHMgdXBkYXRlOmRldmljZV9jcmVkZW50aWFscyBkZWxldGU6ZGV2aWNlX2NyZWRlbnRpYWxzIGNyZWF0ZTpkZXZpY2VfY3JlZGVudGlhbHMgcmVhZDpydWxlcyB1cGRhdGU6cnVsZXMgZGVsZXRlOnJ1bGVzIGNyZWF0ZTpydWxlcyByZWFkOnJ1bGVzX2NvbmZpZ3MgdXBkYXRlOnJ1bGVzX2NvbmZpZ3MgZGVsZXRlOnJ1bGVzX2NvbmZpZ3MgcmVhZDpob29rcyB1cGRhdGU6aG9va3MgZGVsZXRlOmhvb2tzIGNyZWF0ZTpob29rcyByZWFkOmFjdGlvbnMgdXBkYXRlOmFjdGlvbnMgZGVsZXRlOmFjdGlvbnMgY3JlYXRlOmFjdGlvbnMgcmVhZDplbWFpbF9wcm92aWRlciB1cGRhdGU6ZW1haWxfcHJvdmlkZXIgZGVsZXRlOmVtYWlsX3Byb3ZpZGVyIGNyZWF0ZTplbWFpbF9wcm92aWRlciBibGFja2xpc3Q6dG9rZW5zIHJlYWQ6c3RhdHMgcmVhZDp0ZW5hbnRfc2V0dGluZ3MgdXBkYXRlOnRlbmFudF9zZXR0aW5ncyByZWFkOmxvZ3MgcmVhZDpzaGllbGRzIGNyZWF0ZTpzaGllbGRzIHVwZGF0ZTpzaGllbGRzIGRlbGV0ZTpzaGllbGRzIHJlYWQ6YW5vbWFseV9ibG9ja3MgZGVsZXRlOmFub21hbHlfYmxvY2tzIHVwZGF0ZTp0cmlnZ2VycyByZWFkOnRyaWdnZXJzIHJlYWQ6Z3JhbnRzIGRlbGV0ZTpncmFudHMgcmVhZDpndWFyZGlhbl9mYWN0b3JzIHVwZGF0ZTpndWFyZGlhbl9mYWN0b3JzIHJlYWQ6Z3VhcmRpYW5fZW5yb2xsbWVudHMgZGVsZXRlOmd1YXJkaWFuX2Vucm9sbG1lbnRzIGNyZWF0ZTpndWFyZGlhbl9lbnJvbGxtZW50X3RpY2tldHMgcmVhZDp1c2VyX2lkcF90b2tlbnMgY3JlYXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgZGVsZXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgcmVhZDpjdXN0b21fZG9tYWlucyBkZWxldGU6Y3VzdG9tX2RvbWFpbnMgY3JlYXRlOmN1c3RvbV9kb21haW5zIHVwZGF0ZTpjdXN0b21fZG9tYWlucyByZWFkOmVtYWlsX3RlbXBsYXRlcyBjcmVhdGU6ZW1haWxfdGVtcGxhdGVzIHVwZGF0ZTplbWFpbF90ZW1wbGF0ZXMgcmVhZDptZmFfcG9saWNpZXMgdXBkYXRlOm1mYV9wb2xpY2llcyByZWFkOnJvbGVzIGNyZWF0ZTpyb2xlcyBkZWxldGU6cm9sZXMgdXBkYXRlOnJvbGVzIHJlYWQ6cHJvbXB0cyB1cGRhdGU6cHJvbXB0cyByZWFkOmJyYW5kaW5nIHVwZGF0ZTpicmFuZGluZyBkZWxldGU6YnJhbmRpbmcgcmVhZDpsb2dfc3RyZWFtcyBjcmVhdGU6bG9nX3N0cmVhbXMgZGVsZXRlOmxvZ19zdHJlYW1zIHVwZGF0ZTpsb2dfc3RyZWFtcyBjcmVhdGU6c2lnbmluZ19rZXlzIHJlYWQ6c2lnbmluZ19rZXlzIHVwZGF0ZTpzaWduaW5nX2tleXMgcmVhZDpsaW1pdHMgdXBkYXRlOmxpbWl0cyIsImd0eSI6ImNsaWVudC1jcmVkZW50aWFscyJ9.A8BWCihsazeHasnD2KeP8YNO08uwhjZzo2yxvGLH1CGS9VymjtroGAM4ZFo4XELjZnobK6WzplGiYDUZz9H1m8A5jRWOfPbixXOG69pLUYka5NZR153F-EmqrXQ81vM25kEPsyHLJBqadvaDow-XOAN_t8KbTuetjP0e7ya8C8JY30uLWgYrQODwhQK03qu0P9hkutknVvKIzQChKL8l0b-pThJMu65-6b5yUr57giVITXtljshXFiUsYTRTsi_6Sgun70RyTTYmpx7DgjWif_mlra2bpSpHnJFVT8gSPkj2bntwiq3KzFicP4yld4OqD0Rc0DnknFRr3emINud4-Q"
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 4022
Content-Type: application/json
Date: Tue, 09 Jun 2020 12:13:04 GMT
Server: gunicorn/19.9.0
Via: kong/2.0.4
X-Kong-Proxy-Latency: 1
X-Kong-Upstream-Latency: 5

{
    "args": {},
    "data": "",
    "files": {},
    "form": {},
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate",
        "Authorization": "Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IlFrVXdOREl4UmpsR1JEYzNSalU1UVVaRE9UVkRSalUyTURJek9FTkdNRFJCTlRBeVJFUkNOUSJ9.eyJpc3MiOiJodHRwczovL2ZhcnJlbGxzb2Z0LmF1dGgwLmNvbS8iLCJzdWIiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YkBjbGllbnRzIiwiYXVkIjoiaHR0cHM6Ly9mYXJyZWxsc29mdC5hdXRoMC5jb20vYXBpL3YyLyIsImlhdCI6MTU5MTY1NjA2NiwiZXhwIjoxNTkxNzQyNDY2LCJhenAiOiJsQ1A3dHYzZDZ2cHJJbTBmcW5zOURpOWVwNmI1Z3A5YiIsInNjb3BlIjoicmVhZDpjbGllbnRfZ3JhbnRzIGNyZWF0ZTpjbGllbnRfZ3JhbnRzIGRlbGV0ZTpjbGllbnRfZ3JhbnRzIHVwZGF0ZTpjbGllbnRfZ3JhbnRzIHJlYWQ6dXNlcnMgdXBkYXRlOnVzZXJzIGRlbGV0ZTp1c2VycyBjcmVhdGU6dXNlcnMgcmVhZDp1c2Vyc19hcHBfbWV0YWRhdGEgdXBkYXRlOnVzZXJzX2FwcF9tZXRhZGF0YSBkZWxldGU6dXNlcnNfYXBwX21ldGFkYXRhIGNyZWF0ZTp1c2Vyc19hcHBfbWV0YWRhdGEgcmVhZDp1c2VyX2N1c3RvbV9ibG9ja3MgY3JlYXRlOnVzZXJfY3VzdG9tX2Jsb2NrcyBkZWxldGU6dXNlcl9jdXN0b21fYmxvY2tzIGNyZWF0ZTp1c2VyX3RpY2tldHMgcmVhZDpjbGllbnRzIHVwZGF0ZTpjbGllbnRzIGRlbGV0ZTpjbGllbnRzIGNyZWF0ZTpjbGllbnRzIHJlYWQ6Y2xpZW50X2tleXMgdXBkYXRlOmNsaWVudF9rZXlzIGRlbGV0ZTpjbGllbnRfa2V5cyBjcmVhdGU6Y2xpZW50X2tleXMgcmVhZDpjb25uZWN0aW9ucyB1cGRhdGU6Y29ubmVjdGlvbnMgZGVsZXRlOmNvbm5lY3Rpb25zIGNyZWF0ZTpjb25uZWN0aW9ucyByZWFkOnJlc291cmNlX3NlcnZlcnMgdXBkYXRlOnJlc291cmNlX3NlcnZlcnMgZGVsZXRlOnJlc291cmNlX3NlcnZlcnMgY3JlYXRlOnJlc291cmNlX3NlcnZlcnMgcmVhZDpkZXZpY2VfY3JlZGVudGlhbHMgdXBkYXRlOmRldmljZV9jcmVkZW50aWFscyBkZWxldGU6ZGV2aWNlX2NyZWRlbnRpYWxzIGNyZWF0ZTpkZXZpY2VfY3JlZGVudGlhbHMgcmVhZDpydWxlcyB1cGRhdGU6cnVsZXMgZGVsZXRlOnJ1bGVzIGNyZWF0ZTpydWxlcyByZWFkOnJ1bGVzX2NvbmZpZ3MgdXBkYXRlOnJ1bGVzX2NvbmZpZ3MgZGVsZXRlOnJ1bGVzX2NvbmZpZ3MgcmVhZDpob29rcyB1cGRhdGU6aG9va3MgZGVsZXRlOmhvb2tzIGNyZWF0ZTpob29rcyByZWFkOmFjdGlvbnMgdXBkYXRlOmFjdGlvbnMgZGVsZXRlOmFjdGlvbnMgY3JlYXRlOmFjdGlvbnMgcmVhZDplbWFpbF9wcm92aWRlciB1cGRhdGU6ZW1haWxfcHJvdmlkZXIgZGVsZXRlOmVtYWlsX3Byb3ZpZGVyIGNyZWF0ZTplbWFpbF9wcm92aWRlciBibGFja2xpc3Q6dG9rZW5zIHJlYWQ6c3RhdHMgcmVhZDp0ZW5hbnRfc2V0dGluZ3MgdXBkYXRlOnRlbmFudF9zZXR0aW5ncyByZWFkOmxvZ3MgcmVhZDpzaGllbGRzIGNyZWF0ZTpzaGllbGRzIHVwZGF0ZTpzaGllbGRzIGRlbGV0ZTpzaGllbGRzIHJlYWQ6YW5vbWFseV9ibG9ja3MgZGVsZXRlOmFub21hbHlfYmxvY2tzIHVwZGF0ZTp0cmlnZ2VycyByZWFkOnRyaWdnZXJzIHJlYWQ6Z3JhbnRzIGRlbGV0ZTpncmFudHMgcmVhZDpndWFyZGlhbl9mYWN0b3JzIHVwZGF0ZTpndWFyZGlhbl9mYWN0b3JzIHJlYWQ6Z3VhcmRpYW5fZW5yb2xsbWVudHMgZGVsZXRlOmd1YXJkaWFuX2Vucm9sbG1lbnRzIGNyZWF0ZTpndWFyZGlhbl9lbnJvbGxtZW50X3RpY2tldHMgcmVhZDp1c2VyX2lkcF90b2tlbnMgY3JlYXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgZGVsZXRlOnBhc3N3b3Jkc19jaGVja2luZ19qb2IgcmVhZDpjdXN0b21fZG9tYWlucyBkZWxldGU6Y3VzdG9tX2RvbWFpbnMgY3JlYXRlOmN1c3RvbV9kb21haW5zIHVwZGF0ZTpjdXN0b21fZG9tYWlucyByZWFkOmVtYWlsX3RlbXBsYXRlcyBjcmVhdGU6ZW1haWxfdGVtcGxhdGVzIHVwZGF0ZTplbWFpbF90ZW1wbGF0ZXMgcmVhZDptZmFfcG9saWNpZXMgdXBkYXRlOm1mYV9wb2xpY2llcyByZWFkOnJvbGVzIGNyZWF0ZTpyb2xlcyBkZWxldGU6cm9sZXMgdXBkYXRlOnJvbGVzIHJlYWQ6cHJvbXB0cyB1cGRhdGU6cHJvbXB0cyByZWFkOmJyYW5kaW5nIHVwZGF0ZTpicmFuZGluZyBkZWxldGU6YnJhbmRpbmcgcmVhZDpsb2dfc3RyZWFtcyBjcmVhdGU6bG9nX3N0cmVhbXMgZGVsZXRlOmxvZ19zdHJlYW1zIHVwZGF0ZTpsb2dfc3RyZWFtcyBjcmVhdGU6c2lnbmluZ19rZXlzIHJlYWQ6c2lnbmluZ19rZXlzIHVwZGF0ZTpzaWduaW5nX2tleXMgcmVhZDpsaW1pdHMgdXBkYXRlOmxpbWl0cyIsImd0eSI6ImNsaWVudC1jcmVkZW50aWFscyJ9.A8BWCihsazeHasnD2KeP8YNO08uwhjZzo2yxvGLH1CGS9VymjtroGAM4ZFo4XELjZnobK6WzplGiYDUZz9H1m8A5jRWOfPbixXOG69pLUYka5NZR153F-EmqrXQ81vM25kEPsyHLJBqadvaDow-XOAN_t8KbTuetjP0e7ya8C8JY30uLWgYrQODwhQK03qu0P9hkutknVvKIzQChKL8l0b-pThJMu65-6b5yUr57giVITXtljshXFiUsYTRTsi_6Sgun70RyTTYmpx7DgjWif_mlra2bpSpHnJFVT8gSPkj2bntwiq3KzFicP4yld4OqD0Rc0DnknFRr3emINud4-Q",
        "Connection": "keep-alive",
        "Host": "bin.test",
        "User-Agent": "HTTPie/2.1.0",
        "X-Consumer-Id": "6ad39431-9f2f-4f49-9624-6c8c4ea36af2",
        "X-Consumer-Username": "apiUser",
        "X-Credential-Identifier": "https://farrellsoft.auth0.com/",
        "X-Forwarded-Host": "localhost",
        "X-Forwarded-Prefix": "/"
    },
    "json": null,
    "method": "GET",
    "origin": "127.0.0.1",
    "url": "http://localhost/anything"
}

Works fine as well.

Yes, I already created the top half of that file. I was trying to avoid creating the bottom half as my goal was to leverage jwt WITHOUT delving into KongService, KongRoute, and KongIngress since I wanted to stick with the standard Kubernetes resources (Service, Ingress).

The bigger question I have here is why. I can understand why a call to /config effectively resets the in memory store for Kong but, I am confused why running kubectl apply on a Kubernetes Ingress Yaml file, and thereby causing a sync to happen with Kong, would eliminate my KongConsumer and KongCredential created via a Yaml file passed to /config.

To restate - I am not implying that what I am doing cannot be done with Kong, I know that it can. I am trying to approach this in a very certain way with the following aims:

  • Leverages as much of standard Kubernetes resources as possible
  • Enables creation of declaration files that can be version controlled

Thanks.

@hbagdi, can you help with Kubernetes side of things?

I can make time for a Zoom call or Google Meeting if appropriate. I have been working this for a few days now and its been slow going

Thanks

Using either is possible. When the controller is enabled, it will scan Kubernetes resources, render the config file, and send it to /config on your behalf.

https://github.com/Kong/kubernetes-ingress-controller/blob/master/docs/guides/using-consumer-credential-resource.md goes through the resources needed to apply an auth plugin. The KongConsumer section is where that credential (the iss or other key_claim_name value) is bound to a specific consumer.

While you cannot modify the config directly with the controller in place, GET requests will still work normally–if you find that you’re no longer able to access the route as intended after creating resources for the controller, comparing GET /consumers/apiUser/jwt (and similar for routes/plugins/etc.) output between the controller configuration and manually-added config files may provide clues as to what’s missing in the controller/Kubernetes resources.

I am still not following. Let walk through my complete example:

First I create my deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2020-05-31T22:04:51Z"
  generation: 1
  labels:
    app: weather-deployment
  name: weather-deployment
  namespace: weather-api
  resourceVersion: "202906"
  selfLink: /apis/apps/v1/namespaces/weather-api/deployments/weather-deployment
  uid: 76ea373a-301d-4adc-8363-405babb2c0de
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: weather-deployment
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: weather-deployment
    spec:
      containers:
      - image: xximjasonxx/weatherapi-basic:v2
        imagePullPolicy: IfNotPresent
        name: weatherapi-basic
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2020-05-31T22:04:51Z"
    lastUpdateTime: "2020-05-31T22:04:53Z"
    message: ReplicaSet "weather-deployment-6d8789b8fc" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2020-06-02T23:27:08Z"
    lastUpdateTime: "2020-06-02T23:27:08Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

Next, I create the Service:

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2020-05-31T22:13:14Z"
  labels:
    app: weather-service
  name: weather-service
  namespace: weather-api
  resourceVersion: "3696"
  selfLink: /api/v1/namespaces/weather-api/services/weather-service
  uid: 38574bc4-1e29-44ee-8b80-e1a05e4a1e67
spec:
  clusterIP: 10.0.25.75
  ports:
  - name: 80-80
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: weather-deployment
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Next, I create my plugins:

#
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: weather-rate-limit
  namespace: weather-api
config:
  minute: 5
  policy: local
plugin: rate-limiting

---

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: weather-jwt
  namespace: weather-api
plugin: jwt

Next, I create the Ingress

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: weather-ingress
  namespace: weather-api
  annotations:
    kubernetes.io/ingress.class: kong
    plugins.konghq.com: weather-rate-limit, weather-jwt
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: weather-service
              servicePort: 80

Once created, I browse the endpoint and received 401 Unauthorized as expected. Based on the link you sent it seems I need to create a Consumer. So I do that - I include my credentials as instructed based on what I read

apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
  name: api-consumer
  namespace: weather-api
username: apiUser

jwt_secrets:
  - consumer: apiUser
    key: https://mycompany.auth0.com/
    algorithm: RS256
    rsa_public_key: |-
      -----BEGIN PUBLIC KEY-----
      -----END PUBLIC KEY-----

I call the admin API /consumers/apiUser and I receive the following JSON response

{
    "custom_id": null,
    "created_at": 1591755695,
    "id": "182d87e8-9e41-5dee-8623-89007c2a0cdc",
    "tags": null,
    "username": "apiUser"
}

When I call /consumers/apiUser/jwt I get the following:

{
    "next": null,
    "data": []
}

This is the issue, the credentials are not being created. I need to figure out why this is happening. Any thoughts?

The consumer-credential mapping works by listing Secrets in the credentials array of the KongConsumer, e.g.

apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
  name: api-consumer
  namespace: weather-api
username: apiUser
credentials:
- apiuser-apikey

And then to store the credential itself, something like:

apiVersion: v1
kind: Secret
metadata:
  name: apiuser-apikey
type: Opaque
stringData:
  key: https://mycompany.auth0.com/
  algorithm: RS256
  rsa_public_key: |-
    -----BEGIN PUBLIC KEY-----
    -----END PUBLIC KEY-----

Unfortuantely, this has no effect on the outcome, I am still getting the same error with regard to iss and I can confirm via the Admin API that /jwt returns the result as stated above. I tried creating the secret in both default and weather-api to no avail.

Might it be necessary to set up a call to dive deeper into this? I know this can be done, but I am bemused with why its proving so difficult.

For reference:
Secret:

apiVersion: v1
kind: Secret
metadata:
  name: apiuser-apikey
  namespace: weather-api
type: Opaque
stringData:
  key: https://mycompany.auth0.com/
  algorithm: RS256
  rsa_public_key: |-
    -----BEGIN PUBLIC KEY-----
    MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3PYgeiVyURLhqAkkUOfL
    -----END PUBLIC KEY-----

Consumer

apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
  name: api-consumer
  namespace: weather-api
username: apiUser
credentials:
- apiuser-apikey

Thanks

Any ideas on the above?

Apologies, forgot the kongCredType earlier in the secret. In the future, know that the controller will often report issues with configuration you’ve added in its logs:

$ kubectl logs test-kong-5c844d47fd-d57cc -c ingress-controller
E0611 23:21:13.468398       1 parser.go:388] invalid credType in secret 'helmgress/apiuser-apikey'

The corrected version is:

apiVersion: v1
kind: Secret
metadata:
  name: apiuser-apikey
  namespace: weather-api
type: Opaque
stringData:
  kongCredType: jwt
  key: https://mycompany.auth0.com/
  algorithm: RS256
  rsa_public_key: |-
    -----BEGIN PUBLIC KEY-----
    ...
    -----END PUBLIC KEY-----

I actually tried that already and still had no effect. It does not seem to matter what I do, the /jwt does not get created for the consumer via the KongConsumer resource def. Consumer is getting created fine

Got it!!! Of course it was something stupid. Yes, I was missing the kongCredType but I realized that before you replied. No, the issue was I forgot the namespace data for my secret.

Its in the sample but I mustve deleted it at some point. Everything is working.
I can now write up my blog post on this. The feedback I would give is, it is pretty clear there needs to be some better clarity created around using Kong in db-less with Kubernetes.

Cheers. And thanks for the help

Complete blog post on this experience and the solution here: https://jfarrell.net/2020/06/11/kong-jwt-and-auth0/