Are there any pointers for tuning the kong_cache size and the size of the separate cache used for the ee_proxy_cache plugin?
If rate-limiting is done per-user and there are a LOT of users (say >25K) I’m assuming you’d need more cache space than for an API where rate-limiting is done per service or route?
I’m also seeing that lua might have upper memory limits, so even if you had a lot of memory on the instance (16GB) you really couldn’t assign more than half a GB or so to each cache?
Are there any pointers for tuning the kong_cache size
The kong_cache size can be tuned via the mem_cache_size configuration property. (See how the template injects it here).
If you happen to be customizing the underlying Nginx configuration template, be aware that this shared memory zone was renamed to kong_db_cache to better reflect its role and avoid misleading users into believing this was something else than a database cache. This is just a friendly warning in case you are customizing the Nginx config template, otherwise, you have nothing to worry about
And even if you were to modify it, when such changes to the configuration occur, they are documented in the changelog (as breaking changes), and a diff of the Nginx config changes is included in the Upgrade Path (unfold the “Click here to see the Nginx configuration changes” fold).
You can read more about database caching (and clustering, as it is what this ultimately relies to) here (and I am only now realizing that this section of the documentation does not mention the mem_cache_size configuration property at all… )
and the size of the separate cache used for the ee_proxy_cache plugin?
You may already be aware: this plugin proposes a config.memory.dictionary_name configuration property. By default, this will be kong_cache (for pre 0.14 cores). (Source: Proxy Cache | Kong Docs). If you wish to use a separate memory zone (which can avoid running into cache churning scenarios depending on your specific workload and how busy your Kong nodes are), you can define your own shared memory zone. In pre 0.14, you’d have to customize the Nginx configuration, and in 0.14 and above, you could inject the directive via an environment variable or a minor edit to your kong.conf file
I’m also seeing that lua might have upper memory limits, so even if you had a lot of memory on the instance (16GB) you really couldn’t assign more than half a GB or so to each cache?
Luckily, the shared memory zones allocated by Nginx are not bounded by this limit, and you are free to assign very large areas of memory! More details on the ngx_lua documentation for lua_shared_dict.
If rate-limiting is done per-user and there are a LOT of users (say >25K) I’m assuming you’d need more cache space than for an API where rate-limiting is done per service or route?
It depends on how you configured rate limiting:
If you are using the rate-limiting plugin withconfig.policy = local, then yes.
If you are using rate-limiting-advanced (in any configured fashion), then yes as well. This plugin allows you to customize the underlying shared memory zone as well with config.dictionary_name.
There should be a way for us to estimate the amount of memory needed to support N concurrent clients in each implementation, that is something that we should communicate better on the documentation (unless I missed it).
Or else, if you are looking at refining your solution, there exists tools to monitor shared memory usage for OpenResty that you can leverage to fine-tune your implementation: GitHub - openresty/stapxx: Simple macro language extentions to systemtap. This is something that Kong aims at doing on its own, eventually, but this has yet to be implemented.
For the rate-limiting-advanced plugin, would setting the config.sync_rate to a sufficiently high number make it behave essentially like the local setting of the rate-limiting plugin?
Where would one set the size of the config.dictionary_name (kong_rate_limiting_counters)? In the NGINX template?
Are there pointers for a good value for config.sync_rate? Should it be set to correspond to the rate-limit window? e.g. hourly for an hourly window?
You can set it to -1, in which case, it would never synchronize to the datastore, and thus mimic the behavior of rate-limiting with local policy:
a value of -1 ignores sync behavior entirely and only stores counters in node memory.
The size of the default shared dict is hard-coded to 12m. To increase it, you’d have to edit the file (easy but not recommended because you’d have to not forget to do so between Kong upgrades). You can edit it in /usr/local/share/lua/5.1/kong/templates/nginx_kong.lua (I believe). A find can help you there too. Or else, you can maintain your own Nginx configuration template (as described above), define your own lua_shared_dict, and instruct the plugin to use your shared memory zone instead. When the underlying Kong version to EE gets bumped to 0.14, a simple environment variable will be able to define the shared memory zone for you (see my previous link).
It mostly depends on how frequently you want your Kong nodes to get in sync. If you have a load balancing with consistent hashing in front of Kong, you can on in-memory only counters. If not, then the sync_rate will take care of synchronizing them for you, but you have to decide what is the level of eventual consistency you are fine with. It’s a tradeoff
One pitfall to avoid with this plugin is to define hundreds (or more) of instances of it (e.g. one per Consumer) and use auto-generated config.namespace with sync_rate >= 0. It is fine if you just configure the plugin on a Route/Service, or even a few dozen of them. That is because all plugin instances will sync to the DB based on their config.namespace value, so if you have too many auto-generated (random) namespaces, you may overwhelm your datastore with requests to synchronize each namespace.
So just to be sure I understand, let’s say (just as a hypothetical ) that a Kong configuration has the rate-limiting plugin enabled on one service, and also enabled on some consumers who have a higher limit.
Even if >25K separate credentials are active on that service (with the rate-limiting plugin) it will still only have one namespace (with 25K values assuming all the credentials are actively hitting that service).
Only the consumers that have the rating-limiting plugin will have their own namespaces.
And in the worst/most naïve possible case, given two Kong nodes, even if the nodes never sync and there’s no consistent hashing load-balancing, a consumer would only be able to make 2x calls their limit. (And couldn’t rely on that, as they couldn’t count on their calls being perfectly evenly distributed )
The namespace is determined by the config.namespace parameter (source):
The rate limiting library namespace to use for this plugin instance. Counter data and sync configuration is shared in a namespace.
By default, a namespace is auto-generated for each configuration of the plugin, which is most likely the desired behavior. The plugin, when configured globally, or on a Service/Route, will rate-limiting authenticated consumers just fine (as long as config.identifier is set appropriately), all equally.
Some of our users however, need very fine-grained customizations for each Consumer (with different limits). To the point that they may create a few hundreds configurations of this plugin (one for each “special” Consumer). In this case, we recommend using the namepace property to federate those plugins together, so that they share the same sync-rate (which is much more efficient), but simply apply different limits. If you do not create hundreds of instances of this plugin, then you have nothing to worry about.
An interesting resource to consult on the above topic is the order of plugins precedence, which explains how one may “override” a plugin configured on, say, a Service, by re-configuring it on a Consumer, with slightly different values. Effectively, Kong will execute the Consumer’s plugin when the Consumer is authenticated, instead of the standard plugin it would execute for all other requests/Consumers. A practical use-case for this is to grant a specific Consumer a more generous usage policy, with higher rate limiting counters.
Correct. In any case, the use-case behind the rate-limiting plugins is not really to provide accurate monitoring, but the protection of your upstream services against aggressive usage
)
) that a Kong configuration has the rate-limiting plugin enabled on one service, and also enabled on some consumers who have a higher limit.
)