Nil entity returned when using dao select statement on Kong 3.3 Postgres

Questions
1

After upgrading plugins from Kong 2.2.1 to Kong 3.3, under the following conditions:

  1. I have a sample piece of lua code in a custom plugin that utilizes the kong cache function
local credential_cache_key = kong.db.oauth2_credentials:cache_key(client_id)
        client, err = kong.cache:get(credential_cache_key, nil,
            load_oauth2_credential_by_client_id,
            client_id)

where the callback function to retrieve the entity to be loaded into the cache is a dao select function

local function load_oauth2_credential_by_client_id(client_id)
    local credential, err = kong.db.oauth2_credentials:select_by_client_id(client_id)
    if err then
        return nil, err
    end

    return credential
end
  1. Kong is deployed with the custom plugin in Kubernetes with more than one instance.

More often than not, on the initial start up, kong will hit the following error

[error] 1261#0: *4864420 [kong] init.lua:359 [partner-custom-oauth2] /opt/kong/plugins/partner-custom-oauth2/access.lua:1071: attempt to index local 'client' (a nil value)

indicating that the value in the cache/entity returned from the select statement is nil, even though a manual db query indicates that the entity exists but ws_id is null.

image
image1653×87 30.9 KB

After a few kong restarts, it is noticed that the error goes away and when checking the oauth2_credentials table the ws_id is populated.

When turning on OpenTelemetry logs, it is noted that the dao select statement queries ws_id as well, and having ws_id as null could then be an issue

SELECT
  id,
  EXTRACT(EPOCH FROM created_at AT TIME ZONE 'UTC') AS created_at,
  expires_in,
  access_token,
  ws_id,
  FLOOR(EXTRACT(EPOCH FROM (ttl AT TIME ZONE 'UTC' - CURRENT_TIMESTAMP AT TIME ZONE 'UTC'))) AS ttl
.... (columns scrubbed to remove sensitive information)
FROM oauth2_tokens
WHERE
  access_token = 'jkCFYnyzN2gumQI9lbKgYTuHE63XNXDO'
  AND (ttl IS NULL OR ttl >= CURRENT_TIMESTAMP AT TIME ZONE 'UTC')
  AND (ws_id = '6c431e2e-020c-4e4c-af58-a25f21b2d5de')
LIMIT 1;

Anyone has had a similar issue or know the cause? What causes the ws_id to be null in the first place