Consumer value is empty in Prometheus kong_http_requests_total

1

I am using Kong 3.1. I have set per_consumer flag to true in my Prometheus global plugin but still in Prometheus metrics logs I cannot see consumer value.
For some services log, it is shown with correct consumer value but not for all services’ kong_http_requests_total log. Example below:

kong_http_requests_total{service=“myservice”,route=“myroute”,code=“200”,source=“service”,consumer=“”}

2

@Sachin_Ghumbre Do you have authentication plugin set on service/route level?

3

No @trustworthygoblin . I am using separate OAuth token endpoint with Global OAuth plugin (client credentials grant type) to generate the token. Generated token will be passed in authorization header while making a call to other APIs. Here, sometimes consumer value is captured in the prometheus log and sometimes not.

Interestingly, I can see all 4xx/5xx API calls do not have consumer value in prometheus log kong_http_requests_total . Also, my separate Token Endpoint (OAuth) and couple of other APIs with 2xx are also not showing consumer value. Rest all successful APIs show correct consumer value.