Not able to upload files to server through Kong API Gateway

1

Hello,
I am observing an error on our Kong setup https://paste.fedoraproject.org/paste/cgbYVO2MU1s47gNqnaKlGQ , while uploading file to our servers via Kong Gateway.

2018/07/20 13:49:04 [info] 26817#0: *12337 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 10.100.10.231, server: kong, request: "POST /media-service/api/news/5/media HTTP/1.1", upstream: "http://10.1.0.34:80/api/news/5/media", host: "10.100.10.128:8000", referrer: "http://admin-app.mystaff.hexad.cloud/news/5/media"

nginx-kong.conf

As I have seen it, this is mostly happening if the files (mostly images) are larger than 20 KB in size.

I found this SO answer: https://stackoverflow.com/a/13953869/1125893 which says to to set proxy_ignore_client_abort on http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_client_abort.

But I am not sure about it, as we don’t have proxy enabled.

Is there any Kong configuration that can avoid the premature closing of connection?

2

Hi there,

But I am not sure about it, as we don’t have proxy enabled

Kong uses the proxy module to act as a proxy, so this could in fact affect the result. In order to include that variable in your nginx config, you can add this to your Kong config file:

nginx_http_proxy_ignore_client_abort=on

See the docs for more info about how to include specific Nginx directives in your Kong config.

After adding it, you will have to execute kong prepare and kong reload.

Notice that in the same SO thread you linked there is another anwer that could be happening as well: there is a person which had something before their NGINX instance (a load balancer) and it had a too small timeout.

Also, I would look at the logs closely to make sure the dropped requests are coming from real users and not from things like crawlers or bots testing your network - they can cancel requests immediately after making them. If that’s the case, these logs can be a good way to find them out. Reminder that Kong comes with a bot detection plugin that could potentially be useful to you.

3

Hi,

Thanks for replying :slight_smile:

I tried setting the directive in kong.conf

nginx_http_proxy_ignore_client_abort=on

but it’s not taking effect.

I checked in /usr/local/kong/nginx-kong.conf and /usr/local/kong/nginx.conf no entry of proxy_ignore_client_abort=on

This is my kong.conf(/etc/kong/kong.conf) https://paste.fedoraproject.org/paste/Ra230HXVtfUi-rP5HmQcZQ

Also, I tried with env variable

export KONG_NGINX_HTTP_PROXY_IGNORE_CLIENT_ABORT=on

no luck.

Is this the correct way to set the http directive?

4

Hi, did you get any errors when you did kong prepare or kong reload?

5

Nope. Just some locale warning(which I suppose are harmless)

root@kong-mystaff:/usr/local/kong# kong prepare
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_CTYPE = "UTF-8",
	LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
root@kong-mystaff:/usr/local/kong# kong reload
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_CTYPE = "UTF-8",
	LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
Kong reloaded
root@kong-mystaff:/usr/local/kong# kong check
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_CTYPE = "UTF-8",
	LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
configuration at /etc/kong/kong.conf is valid
6

Without locale warning:

root@kong-mystaff:/usr/local/kong# kong check
configuration at /etc/kong/kong.conf is valid
root@kong-mystaff:/usr/local/kong# kong prepare
root@kong-mystaff:/usr/local/kong# kong reload
Kong reloaded
root@kong-mystaff:/usr/local/kong#
7

Is your Kong version up to date? The nginx directives changes where releases in Kong 0.14

8

Ah. You are right.
Kong version is: 0.13.0

root@kong-mystaff:/usr/local/kong# kong version
0.13.0

I’ll update the kong version to latest: https://github.com/Kong/kong/releases/tag/0.14.0 and then try setting the directive.

Thanks!

1 Like
9

@kikito
After upgrading kong version to 0.14.0 the directive nginx_http_proxy_ignore_client_abort=on got added.
Also, I had to increase the ulimit to 4096 for the file uploads to work.

ulimit -n 4096

I am no longer getting client closed connection and file uploads are working fine now. Thank you so much for the help.

1 Like