How does kong handle long duration requests?

1

Hello All,

I have a local setup of an application which has a delayed response of 60 seconds (This is my upstream on kong)

When try to access the application via curl command, I get the expected output in 60 seconds.

Whereas when I try to access the same via kong (curl -o /dev/null -s -w “%{http_code}\n” api_gateway_url/my_custom_app), my curl command doesn’t terminates. I have to explicitly cancel the request (pressing control + C)

I check the error logs and see the below error
2020/02/26 14:29:31 [error] 9352#0: *1097844 upstream timed out (110: Connection timed out) while reading response header from upstream, client: X.X.X.X

Is this an expected behavior, that if the client doesn’t set a default timeout for the connection, that kong will not return any exit code after getting a connection timeout from the upstream?

My kong/nginx keepalive_timeout is set to 60 seconds

Regards
Ashish

2

Please increase the timeouts defined in the service object. The default timeouts are 60s for connect, read and write. You can increase them dynamically via the admin api on a per service basis.

3

Hello @hbagdi,
Thanks for the reply. But why I do not get any http code in the response on the client side. Is this an expected behavior ?

Regards
Ashish

4

Yes.
This happens because Kong closes the upstream closes the TCP connection.

5

@hbagdi : Sorry I didn’t get your sentence. Can you please re-phrase or elaborate it ?
Also, shouldn’t kong http send error code to the client?

6

My curl output :

date && curl -vk http://kongurl:8000/some-app
Fri Feb 28 14:30:01 CET 2020
* About to connect() to kongurl port 8000 (#0)
* Trying x.x.x.x…
* Connected to kongurl (x.x.x.x) port 8000 (#0)
> GET /some-app HTTP/1.1
> User-Agent: curl/7.29.0
> Host: kongurl:8000
> Accept: */*
>
< HTTP/1.1 504 Gateway Time-out
< Date: Fri, 28 Feb 2020 13:36:01 GMT
< Content-Type: text/plain; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: kong/1.4.0
< X-Kong-Upstream-Latency: 60001
< X-Kong-Proxy-Latency: 300003
< Via: kong/1.4.0
<
The upstream server is timing out
* Connection #0 to host kongurl left intact

I notice that kong does sends the client a timeout but only after 6 minutes, as well as from error logs I see that kong received 6 timeouts.
Setup
Upstream application has a 60 sec delay and kong configuration for this upstream has timeout value of 60 secs (referring your previous comment)

7

That’s because by default the number of retries is set to 5 minutes.

Everything here is working as expected. What is your question now?

8

Hello @hbagdi,

This means that once the request from kong is sent to the upstream then, incase of http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 or a timeout , kong will retry to send the same request 5 times right to the same upstream(as I do not have proxy_next_upstream setup)?

Service is created with defaults :
{
“host”: “service-xxxxxxxxxxxxxxxxxxxx”,
“created_at”: 1582898310,
“connect_timeout”: 60000,
“id”: “60d38a0e-e4df-4d46-8590-6a2e007f8b69”,
“protocol”: “http”,
“name”: “service-xxxxxxxxxxxxxxxx”,
“read_timeout”: 60000,
“port”: 80,
“path”: “/”,
“updated_at”: 1582898310,
“retries”: 5,
“write_timeout”: 60000,
“tags”: null,
“client_certificate”: null
}
Route is also created with defaults :
{
“id”: “009b9971-9f48-47a7-b072-f99fcac11108”,
“tags”: null,
“updated_at”: 1582898310,
“destinations”: null,
“headers”: null,
“protocols”: [
“http”,
“https”
],
“created_at”: 1582898310,
“snis”: null,
“service”: {
“id”: “60d38a0e-e4df-4d46-8590-6a2e007f8b69”
},
“name”: “route-yyyyyyyyyyyyyyyyy”,
“preserve_host”: false,
“regex_priority”: 0,
“strip_path”: true,
“sources”: null,
“paths”: [
“/fpm-app”
],
“https_redirect_status_code”: 426,
“hosts”: null,
“methods”: null
}

Regards
Ashish

9

I’m not sure but most probably not. Kong retries based on TCP layer errors and timeouts.