Http Log Performance

#1

Hello! I am using Kong as an API gateway in front of some restful API’s. As I am doing some load testing, I have found that the Http Log plugin is fairly slow. For example - If I make 1000 requests through the Kong API to my backend and they are processed around 50 TPS (So approx 20 seconds total time) - the Kong Http Log plugin works - however it takes upwards of 15 minutes for the plugin to process and send the 1000 messages to my http server (Simple Node.JS/Express app). I have verified it is not an issue with the Node application - looking at the load balancer logs in front of my NodeJS application - we are getting less than 1 TPS from the Http Log Plugin. I spent time looking into the code of the Http Log Plugin - it looks like it is using a queuing mechanism to send the logs async which is great, however I need much higher throughput. I also looked into the TCP log plugin - it looks like it fires off the messages right away (no queuing). Should I move to the TCP log plugin?

Any help is greatly appreciated!

#2

Care to post the JSON settings you ran with for the plugin config? How many Nginx worker processes you run with? I use a variant of the HTTP Log plugin as well and like the functionality of a batch queue then send design but I can’t speak much to noticing a hindrance on throughput @ 50 TPS.

#3

I am using the default values for nginx.

Here is the JSON config:

{
“created_at”: 1557758803,
“config”: {
“flush_timeout”: 2,
“http_endpoint”: “http://host.docker.internal:6000/gateway/audit”,
“retry_count”: 10,
“timeout”: 100,
“queue_size”: 1,
“keepalive”: 60000,
“content_type”: “application/json”,
“method”: “POST”
},
“id”: “62f5fb8c-8f40-4de4-aeb5-6bd17c4bb873”,
“service”: null,
“name”: “http-log”,
“protocols”: [“http”, “https”],
“enabled”: true,
“run_on”: “first”,
“consumer”: null,
“route”: null,
“tags”: null
}

#4

bump. Looking for recommendations here!