# Latency spikes under nominal load

**URL:** <https://discuss.konghq.com/t/latency-spikes-under-nominal-load/1254>\
**Category:** General\
**Tags:** kong-gateway\
**Created:** [June 11, 2018, 4:37pm UTC](https://discuss.konghq.com/t/latency-spikes-under-nominal-load/1254 "2018-06-11T16:37:11Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![jeremyjpj0916](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/jeremyjpj0916/32/1388_2.png) [@jeremyjpj0916](https://discuss.konghq.com/u/jeremyjpj0916)\
**Post date:** [June 15, 2018, 8:45am UTC](https://discuss.konghq.com/t/latency-spikes-under-nominal-load/1254/6 "2018-06-15T08:45:14Z")

</div>

Currently investigating our Prod Environment a bit more to see actual impact before I make changes to the default Kong DNS settings, I also don’t know a good way to really debug this, I don’t want to throw kong in debug mode because it prints soooo many logs when really the best way to validate root cause here would be a way to isolate DNS related logging mechanisms and just alert on anomalies found there(like a DNS lookup taking longer than 2s if this is indeed the root cause for bringking or myself), results are as follows thus far from my digging:

Queries I used to lookup real prod traffic:

```auto
index=cba_kong host="gateway" RoutingURL!="*localhost*" TotalLatency >= 2000 BackendLatency <= 1000
index=cba_kong host="gateway" RoutingURL!="*localhost*"

```

Results are as followed over a 7 day period:

 ![FurtherInvestigation](https://canada1.discourse-cdn.com/flex036/uploads/konghq/original/1X/0e9871c4adaed2684e74851532369e66dd6044c4.png)

As you can see, a little over 4% of traffic is impacted by whatever is causing this. Will try some changes to see if this can be resolved. @bringking not sure how deep your analytics go, but I am interested to see the % impact to your services if its around the same % I see.

Edit - Maybe to further prove out if its DNS or not I could turn off DNS caching all together in Stage env and see if doing look-ups every-time yield more frequent latency spikes but I don’t see an option in the kong.conf. If anyone knows what lua file and lines I need to modify(only if its a trivial small snippit change) before starting kong to basically disable all dns caching I will apply that patch in Stage and run some tests.

Edit Edit - I researched our Stage env too and noticed 4% latency spikes from the Kong side of things as well, go figure 😃 . I modified our Stage env to the same DNS STALE TTL 60 value you have tried, will watch it next week and see if any improvements there and 4% goes to 0%. If not back to the drawing board on my side.

---

_[View the full topic](https://discuss.konghq.com/t/latency-spikes-under-nominal-load/1254)._