Http Log Performance


Hello! I am using Kong as an API gateway in front of some restful API’s. As I am doing some load testing, I have found that the Http Log plugin is fairly slow. For example - If I make 1000 requests through the Kong API to my backend and they are processed around 50 TPS (So approx 20 seconds total time) - the Kong Http Log plugin works - however it takes upwards of 15 minutes for the plugin to process and send the 1000 messages to my http server (Simple Node.JS/Express app). I have verified it is not an issue with the Node application - looking at the load balancer logs in front of my NodeJS application - we are getting less than 1 TPS from the Http Log Plugin. I spent time looking into the code of the Http Log Plugin - it looks like it is using a queuing mechanism to send the logs async which is great, however I need much higher throughput. I also looked into the TCP log plugin - it looks like it fires off the messages right away (no queuing). Should I move to the TCP log plugin?

Any help is greatly appreciated!


Care to post the JSON settings you ran with for the plugin config? How many Nginx worker processes you run with? I use a variant of the HTTP Log plugin as well and like the functionality of a batch queue then send design but I can’t speak much to noticing a hindrance on throughput @ 50 TPS.


I am using the default values for nginx.

Here is the JSON config:

“created_at”: 1557758803,
“config”: {
“flush_timeout”: 2,
“http_endpoint”: “http://host.docker.internal:6000/gateway/audit”,
“retry_count”: 10,
“timeout”: 100,
“queue_size”: 1,
“keepalive”: 60000,
“content_type”: “application/json”,
“method”: “POST”
“id”: “62f5fb8c-8f40-4de4-aeb5-6bd17c4bb873”,
“service”: null,
“name”: “http-log”,
“protocols”: [“http”, “https”],
“enabled”: true,
“run_on”: “first”,
“consumer”: null,
“route”: null,
“tags”: null


bump. Looking for recommendations here!