TL;DR: Three consecutive HTTPS requests from an AWS ap-southeast-2 client to a Konnect Serverless Gateway (us-east-1) timed out at the client with ECONNABORTED and no HTTP status. Nothing appeared in Kong’s HTTP Log output — no 499, no error, no entry at all — while the same gateway served other clients normally throughout. It self-resolved after about four minutes. I’ve ruled out six explanations and would like to know if anyone has seen this pattern, or knows of a failure mode on a managed gateway that produces no log line.
Setup
- Kong Konnect Serverless Gateway, us-east-1,
3.15.0.3-enterprise-edition - Custom domain CNAME’d to the
*.serverless.gateways.konggateway.comhostname - One route proxying to an external SaaS API over HTTPS
- HTTP Log plugin shipping to New Relic
- Client: Node.js +
axios/0.27.2in ap-southeast-2,Connection: close(no keep-alive), 20 s timeout, 3 attempts with exponential backoff
Normal latency through this path is 400–500 ms end to end (Sydney → us-east-1 gateway → upstream), and it’s been stable for months.
What happened
All times UTC.
06:52:10 PATCH → 200, 480 ms
06:52:34 PATCH → 200, 454 ms <-- last known good
~06:52:47 attempt 1 → ECONNABORTED after 20,000 ms, status null
~06:53:08 attempt 2 → ECONNABORTED after 20,000 ms, status null
~06:53:31 attempt 3 → ECONNABORTED after 20,000 ms, status null
06:53:52 client gives up
06:58:10 PATCH → 200, 637 ms <-- recovered on its own
All three attempt errors were byte-identical: same code, status: null, same message.
Attempt start times are reconstructed from the retry config working backwards from the error timestamp, so they’re accurate to about a second.
What the gateway logs show
Nothing. Specifically:
- Faceting all production Kong logs on
response.statusfor 06:45–07:00 returns only 200, 204 and 201. No 499s from any client. - Querying by the client’s source IP returns the two successful requests at 06:52 and the next one at 06:58, with nothing in between — despite three requests demonstrably having been attempted across that window.
- Total gateway throughput held at 5–10 req/min throughout, from other source IPs, all 2xx.
So the gateway was healthy and serving traffic. It just has no record that these three requests happened.
What I’ve ruled out
Stale keep-alive sockets / NAT idle timeout. The client sends Connection: close on every request. Every attempt was a fresh DNS → TCP → TLS → request sequence. There was no pooled socket to have gone stale.
Client-side network failure. The error was successfully delivered to Sentry — a different destination — 107 ms after the third attempt aborted, and an audit record was written to another system immediately after. General egress from that runtime was healthy at the exact moment of failure. This also weakens DNS-throughput limits and SNAT port exhaustion, since both would have affected those calls too.
Client aborts that Kong recorded. If the requests had reached a worker and I’d hung up at 20 s, Kong would log 499. There are none, from any client, in the window.
A gateway version rollover. The data plane went 3.15.0.3 → 3.15.0.5 about an hour after the incident. Everything through 07:15 was served by 3.15.0.3. I initially thought this was the answer and it wasn’t.
Upstream slowness. Kong’s read timeout is 60 s, well past my 20 s client timeout, so a slow upstream would produce a 499 at the gateway. See above.
DNS pool churn. The gateway hostname resolves to five A records with a 300 s TTL. The set is identical from my local resolver and from 1.1.1.1, and unchanged across repeated sampling over several minutes. I floated a stale-record theory and the 300 s TTL doesn’t support it — a cached dead address would have persisted longer than the 65-second outage.
Where I’m stuck
Since all three failures produced no HTTP status at all, the failure looks like it was in connection establishment rather than while awaiting a response. That would put it below the layer the HTTP Log plugin can see.
On a self-managed data plane I’d read error.log — TLS handshake failures, worker_connections are not enough, accept4() failed all land there and never touch the access log. On Serverless Gateways there’s no shell, no error.log, and no load balancer of my own in front, so there’s no second log source to consult.
Questions for anyone who’s been here:
- Is there a way to get connection-level diagnostics out of a Serverless Gateway — handshake failures, accept-queue drops, anything below the HTTP layer?
- During a node lifecycle event on Serverless Gateways, can a node stay in the DNS pool after it stops accepting connections? A 300 s TTL would make that window fairly wide.
- Has anyone seen a request accepted at TCP level and then never serviced — no log line either way — on a managed gateway?
- Is there any client-side pattern people use to distinguish “gateway never saw it” from “gateway saw it and dropped it,” short of packet capture?