Shipping Kong logs to S3

Kubernetes
1

I wanted to get my Kong logs into S3.

The approach I took is:

  • Enable the File-log Kong plugin
  • Add a FluentD sidecar container to the Kong deployment. The FluentD container tails the log file and ships the logs to S3
  • Add a LogRotate sidecar container to rotate the log file, to avoid DoSing Kong in the long run

I’m mainly asking for validation of the above approach. Was this a good idea? Is there a more efficient way of getting this done?

Note that the Kong logs are the only logs in my cluster that need to be shipped to S3, hence the sidecar approach instead of a daemonset.

2

This seems workable. I suppose my only open question would be could it be simplified by syslogging to a sidecar that ships the logs to S3? Effectively replacing file -> fluent -> logrotate with one plugin and a sidecar?

I’m not actually sure just talking out loud