Rate-limit plugin

Questions
,
1

This is a query related to the rate-limiting plugin. I am using Kong 3.6 OSS.

I have a consumer configured in my Kong gateway, with four gateway services under that consumer. Two services route to OpenAI endpoints, and the other two route to Gemini endpoints. Each service has its own rate-limiting plugin. My goal is to enforce a consumer-level rate limit of 10 requests per day, but I don’t know in advance how many requests each service will handle. I’ve set a rate limit of 10 requests per day on each service’s rate-limiting plugin. This means that each service is allowed 10 requests per day, correct?

The real challenge is that, among the four services, two are OpenAI and two are Gemini, and each group requires a different combined rate limit. Specifically, I want the total number of requests across both OpenAI services to be capped at 10 per day, regardless of how the requests are distributed between the two endpoints. For example, one OpenAI endpoint might receive 6 requests and the other 4, or the split could be different, but together they should not exceed 10 requests per day.

Same as Gemini has a rate-limit value of 15 combined for both of the Gemini endpoints.

All 4 endpoints are under the same consumer.
How can I set the limit like this using the rate-limiting plugin?

I’ve tried header-based rate-limiting by adding a custom header like
limit_by: header
header: x-model
This was created for OpenAI under the consumer level, and it’s working. But for Gemini, we need a separate rate-limiting plugin, right? But we can add only one rate-limiting plugin under a single consumer.

I’ve also tried path-based routing. Created rate-limiting plugin globally
limit_by: path
path: /openai

Again, this is working for OpenAI.
But I am not able to create a second path-based rate-limiting plugin globally.

Is there any way to set a rate limit like the above?

2

Hi, if your goal is to limit by consumer then I believe you will want to configure the rate-limit plugin to limit_by: consumer (which is also the default value)

3

Hi @jdd
As I mentioned, I currently have four gateway services under the same consumer, and I’ve added separate rate-limit plugins for each.
Yes, by default, limit_by is consumer.
But if I add the request/day as 10 to all the rate-limiting plugins associated with each service, the combined requests/month will be 40 in the consumer level. I’ve tested this case. At the service level, it can only raise 10 requests. That is working. But if all the 4 endpoints raise 10 requests, at the consumer level, it can reach 40 requests. Consumer is not blocking the API requests when the combined number of request-limit reaches 10.
I guess in order to work the solution mentioned by you, we should add the rate-limiting plugin in the consumer level, and it’s not suited for my requirement.