Stale cache is being resurrected by kong.singletons even when resurrect_ttl param is absent


#1

kong.singletons is resurrecting a stale item from the cache from shm even there is no resurrect_ttl is specified.

How to avoid this behaviour?

singletons.cache:get(sessionId, { ttl = tl }, session_client.getSessionSrvcWithClientCrt, session_service_url,sessionId,clientCrt,clientKey,caCrtLoc)

I have also tried with resurrect_ttl value of 0, still no chnage in the behaviour

2019/01/15 19:23:31 [warn] 65#0: *8992 [lua] mlcache.lua:780: get(): callback returned an error (table: 0x42aeb230) but stale value found in shm will be resurrected for 30s (resurrect_ttl), client: 100.124.0.10, server: kong, request: “GET /api/gateway/admin/healthcheck HTTP/1.1”, host: “health.xxxx-xxx-kongsrvr.us-xxxx-2.csp.xxxx.com
2019/01/15 19:23:31 [info] 65#0: *8992 [lua] csp_session_client.lua:236: getSessionWithClientCrt(): Cached Session data type string, client: 100.124.0.10, server: kong, request: “GET /api/gateway/


#2

The Kong singletons cache is kongs main cache, and if you look in the core code you will see where they do initialize it with a resurrect_ttl of 30 seconds. You will need to declare your own mlcache if you don’t want that functionality.


#3

Thank you very much for the quick response.

Venki


#4

@jeremyjpj0916
My understanding is the kong cache is instantiated in init, If I create a custom cache using mlcache what is is the best way to make it available across all the workers in Kong?

Thank you.
Venki,


#5

Example from my plugin:

function OidcHandler:init_worker()
  OidcHandler.super.init_worker(self)
  
  local oidccache, err = mlcache.new("kong_oidc_cache", "kong_oidc_cache", {
    shm_miss         = "kong_oidc_cache_miss",
    shm_locks        = "kong_oidc_cache_locks",
    shm_set_retries  = 3,
    lru_size         = 1000,
    ttl              = 43200,
    neg_ttl          = 30,
    resty_lock_opts  = {exptime = 10, timeout = 5,},
  })
  
  if not oidccache then
    ngx.log(ngx.ERR, "failed to instantiate oidc mlcache: " .. err)
    return
  end
  
  kong.oidc_cache = oidccache
end

In your custom template you would need something like this too:

lua_shared_dict kong_oidc_cache      10m;
lua_shared_dict kong_oidc_cache_miss  6m;
lua_shared_dict kong_oidc_cache_locks 3m;

#6

@jeremyjpj0916,

Thank you for the example really appreciate your help, a couple of quick questions

  1. How does kong resolve “kong.oidc_cache” ?
  2. Do you have the code fragment on where you use oidccache:get?

Thank you.
Venki


#7

@jeremyjpj0916,

When I add lua_shared_dict to the nginx-kong.conf an drestart Kong for some reason Kong overrides my change and set the nginx-kong.conf back to the old unchanged version. Have you ever encountered this issue?

Thank you for your help.


#8

If building/deploying w docker be sure your dropping in the right file to the right place prior to launching kong:

COPY templates/optum_nginx.template /usr/local/kong/optum_nginx.template

Then we launch like(ignore the dynatrace bit):

CMD sh -c "/opt/dynatrace/oneagent/dynatrace-agent64.sh /usr/local/openresty/nginx/sbin/nginx -c '/usr/local/kong/nginx.conf' -p '/usr/local/kong/'"