Well I thought up a plugin I wanted, and began questioning on the gitter about it, @James_Callahan gave me some ideas. Thought I would not write it today but l am just one of those people when I get an idea I have to scratch it like a bad itch lol. Before I open source this to the community I wanted to go over the core code to make sure its ship shape before anyone uses it in the wild so I don’t rek your Kong instance, so posting it here for any Kong devs or community members to point out possible flaws!
The purpose of this plugin will be to limit the size of the responses coming back to Kong from backend on a proxy call and to stop the transaction and return to original caller that the response was too big if api providers try to push something back through the gateway that was defined as too large. For instance say a backend api provider tried to push 1 gig through a response we were not anticipating, we gotta stop them before they hog all that precious mem and just bog down the gateway!
handler.lua:
local BasePlugin = require "kong.plugins.base_plugin"
local responses = require "kong.tools.responses"
local tonumber = tonumber
local MB = 2^20
local KongResponseSizeLimitingHandler = BasePlugin:extend()
KongResponseSizeLimitingHandler.PRIORITY = 802
function KongResponseSizeLimitingHandler:new()
KongResponseSizeLimitingHandler.super.new(self, "kong-response-size-limiting")
end
local function check_size(length, allowed_size)
local allowed_bytes_size = allowed_size * MB
if length > allowed_bytes_size then
return responses.send(413, "Response size limit exceeded")
end
end
function KongResponseSizeLimitingHandler:header_filter(conf)
KongResponseSizeLimitingHandler.super.header_filter(self)
local cl = kong.response.get_header("content-length")
if cl and tonumber(cl) then
check_size(tonumber(cl), conf.allowed_payload_size)
end
end
function KongResponseSizeLimitingHandler:body_filter(conf)
KongResponseSizeLimitingHandler.super.body_filter(self)
kong.ctx.plugin.responseDataSize = kong.ctx.plugin.responseDataSize or 0
kong.ctx.plugin.responseDataSize = kong.ctx.plugin.responseDataSize + #ngx.arg[1] -- ngx.arg[1] Chunk
if ngx.arg[2] then --EOF marker
check_size(kong.ctx.plugin.responseDataSize, conf.allowed_payload_size)
end
end
return KongResponseSizeLimitingHandler
One thing I know for sure I could not decide on was the HTTP Status to respond with this. 413 is supposed to be for request payloads too large, but there is no such status for a response too large. Maybeee 417? I really don’t know .
Thanks ahead of time for any input you have or ways to improve it, a lot of the logic I was able to piece together from existing plugins made by Kong which was a huge help!
I’m not sure what we are trying to protect here; since nginx streams the response back to the client, there typically isn’t any memory concern. To the exception of plugins that buffer the whole response in the Lua land, in which case I believe such a plugin could make sense, but the clients would receive invalid bodies…
One thing I know for sure I could not decide on was the HTTP Status to respond with this.
Well, the more important takeaway here is that once the body has started streaming through Nginx, there is no way to change the status code, since the headers have already been sent to the client. The same way, we cannot update, say, the Content-Length or Content-Type headers (hence my previous commit about invalid bodies), nor can we return an error message to the client. We would just suddenly stop streaming the response back.
Now, we could reject all chunked encoding responses, and rely exclusively on the Content-Length header as per your implementation, so this way we can send any status code, headers and payload we may want, but there isn’t much we can do once we reach the body filtering phase. I am not sure if rejecting all responses without a Content-Length header really is reasonable either…
That is how I thought things worked initially but then I was hopeful based on gitter chat it could still work with a counter of the chunks, but it makes total sense, nginx does not wait on the whole response to buffer in before talking with the client unless you explicitly force it to by reading in the resp body(like Kong does in the response transformer). And I think the argument there would be that this functionality is not worth reading in the whole response body, so I believe with that in mind it makes sense to shorten this plugin to rely on content-length header with the caveat that if the backend does not send the header, then the plugin won’t stop it(I hate tech debt like that but in this circumstance the value gain by the plugin would be lost because you have introduced reading in the whole body every time to mem which is not good).
Back on the point of what we are trying to protect, this was an ask from a provider internally and I simply thought I could make it work:
Client → Gateway → Some provider → Some other service provider relies on
Where client calls are being protected to the provider but the provider currently just calls some other backend service right now and has no protections between the two(api call)
He wanted:
Client → Gateway → Some provider → Gateway → Some other service provider relies on
His reasoning was “Some other service provider relies on” was sending responses back too large to their application and was causing issue, he wants a way to stop those responses from reaching their middleware and to error out. He also wanted auth between their provider app and the service the provider relies on because currently there seems to be no security(sad times) . Does that make sense?
The last unknown is how often does content-length not get set? Just depends on the app? I notice the request-transformer uses it if present then falls back on checking the size of the request data itself if not present(which we can’t really do in the response limiter). Overall I think this code makes the most sense:
local BasePlugin = require "kong.plugins.base_plugin"
local responses = require "kong.tools.responses"
local tonumber = tonumber
local MB = 2^20
local KongResponseSizeLimitingHandler = BasePlugin:extend()
KongResponseSizeLimitingHandler.PRIORITY = 802
function KongResponseSizeLimitingHandler:new()
KongResponseSizeLimitingHandler.super.new(self, "kong-response-size-limiting")
end
local function check_size(length, allowed_size)
local allowed_bytes_size = allowed_size * MB
if length > allowed_bytes_size then
return responses.send(413, "Response size limit exceeded")
end
end
function KongResponseSizeLimitingHandler:header_filter(conf)
KongResponseSizeLimitingHandler.super.header_filter(self)
local cl = kong.response.get_header("content-length")
if cl and tonumber(cl) then
check_size(tonumber(cl), conf.allowed_payload_size)
else
ngx.log(ngx.DEBUG, "Upstream response lacks Content-Length header!")
end
end
return KongResponseSizeLimitingHandler
Tested the above minimal code, seems I can’t do what I want to do during the header_filter phase.
2018/09/13 06:26:43 [error] 38#0: *8051 failed to run header_filter_by_lua*: /usr/local/share/lua/5.1/kong/tools/responses.lua:160: API disabled in the context of header_filter_by_lua*
stack traceback:
[C]: in function 'say'
/usr/local/share/lua/5.1/kong/tools/responses.lua:160: in function 'check_size'
...5.1/kong/plugins/kong-response-size-limiting/handler.lua:24: in function 'header_filter'
/usr/local/share/lua/5.1/kong/init.lua:499: in function 'header_filter'
header_filter_by_lua:2: in function <header_filter_by_lua:1> while reading response header from upstream, client: 10.xxx.xx.x, server: kong, request: "GET /api/fasthttp/v1.0 HTTP/1.1", upstream: "http://server:8080/ ", host: "gateway.company.com"
So I guess you cannot use the API in the header_filter context, I see Kong has acknowledged the limitation here as well once:
Also reading here:
Does mention some directives are disabled (which may disable doing the kong.response(413, “stuff here”) call too, but honestly the header_filter_by_lua phase seems like the perfect phase in the flow to be doing this kind of logic, I get back my response headers from backend and can process and take instant action based on header information to the calling client.
Seems I can’t take action during the header_filter phase. Is there an alternative? Anything that happens in body filter phase won’t work because I can’t change headers and http status on whats already going back to the client .
Edit:
Potentially taking a look at the Kong response transformer plugin, maybe I can use the header filter phase to change the status code header if it calculates size is bigger, then in the body filter phase i transform the response body to my message if a flag i can somehow use between phases is set in the header filter phase?
It seems like you are grabbing the Content-Length header from kong.response, instead of kong.service.response. The former is the response produced by Kong, the latter is the response received from the upstream service. The former would prevent Kong from sending, e.g. a response produced by another plugin if it were bigger than the allowed size. But the response would already be in memory anyway, so it depends if the intent of this plugin is to protect clients as well or no.
Regarding the other issue, how about:
local str = "Response size limit exceeded"
local function check_size(length, allowed_size)
local allowed_bytes_size = allowed_size * MB
if length > allowed_bytes_size then
kong.response.set_status(413)
kong.response.header_header("Content-Length", #str)
end
end
function KongResponseSizeLimitingHandler:body_filter(conf)
if kong.response.get_status() == 413 then
ngx.arg[1] = str
ngx.arg[2] = true
end
end
The PDK does not yet allow for much control over the proxied response (i.e. within the body_filter phase). One caveat of the above code is that an upstream response with a 413 status would be interpreted as if this plugin had just triggered. One way around it would be to use kong.response.get_source(), to guarantee that the status code we just checked came from the service, and not from Kong itself, but get_source() only recognizes calls to kong.response.exit(), and not kong.response.set_stauts(), unfortunately. One workaround would be to set a ctx variable like so for now:
-- check_size:
if length > allowed_bytes_size then
kong.ctx.plugin.limited = true
kong.response.set_status(413)
kong.response.header_header("Content-Length", #str)
end
-- body_filter:
if kong.response.get_status() == 413 and kong.ctx.plugin.limited then
end
Yep, everything you covered here and your follow up with that ctx flag was exactly how I was thinking to drive this plugin(as I realized also that we needed to know if the 413 was kong plugin driven or backend driven(If backend driven then the “size too big” error is just the request is too large and not even about the response size, and if backend driven then kong should not change the response body and let the backend respond as expected)! Going to test this out right now as a matter of fact. Glad we came to similar conclusion, it feels a little less elegant to have to do it like so but I am glad to see its not impossible (Hopefully testing will prove so!).
Current to be tested code(definitely want to be using the size of the upstream response so change to kong.service.response as you suggested!) :
local BasePlugin = require "kong.plugins.base_plugin"
local tonumber = tonumber
local MB = 2^20
local str = "Response size limit exceeded"
local KongResponseSizeLimitingHandler = BasePlugin:extend()
KongResponseSizeLimitingHandler.PRIORITY = 802
function KongResponseSizeLimitingHandler:new()
KongResponseSizeLimitingHandler.super.new(self, "kong-response-size-limiting")
end
local function check_size(length, allowed_size)
local allowed_bytes_size = allowed_size * MB
if length > allowed_bytes_size then
kong.ctx.plugin.limited = true
kong.response.set_status(413)
kong.response.set_header("Content-Length", #str)
end
end
function KongResponseSizeLimitingHandler:header_filter(conf)
KongResponseSizeLimitingHandler.super.header_filter(self)
local cl = kong.service.response.get_header("content-length")
if cl and tonumber(cl) then
check_size(tonumber(cl), conf.allowed_payload_size)
else
ngx.log(ngx.DEBUG, "Upstream response lacks Content-Length header!")
end
end
function KongResponseSizeLimitingHandler:body_filter(conf)
KongResponseSizeLimitingHandler.super.body_filter(self)
if kong.response.get_status() == 413 and kong.ctx.plugin.limited then
ngx.arg[1] = str
ngx.arg[2] = true
end
end
return KongResponseSizeLimitingHandler
One thing I see you added was the
ngx.arg[2] = true
Does this basically tell the body_filter phase no more work to do and to exit out essentially since it triggers the EOF? I would not have known to do that .
EDIT: And it works, thanks again for the input. Will get this out in the public shortly as an easy to add plugin for any to utilize!
.
.
(Hopefully testing will prove so!).