Stdout/stderr being redirected to /tmp

I’m running Kong 0.12.1 in Kubernetes and I noticed the Docker overlay was filling up on one of the k8s nodes. I had set the KONG_PROXY_*_LOG environment variables to redirect to /dev/stdout and /dev/stderr, however our node’s disk usage was climbing wherever Kong was deployed.

After some sleuthing, I found Kong was redirecting stdout and stderr to /tmp/lua_*:

root@b2e3372dddc0 /]# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root        42  0.0  0.0  11644  2672 ?        S    19:14   0:00 sh -c /usr/local/openresty/nginx/sbin/nginx -p 
/usr/local/kong -c nginx.conf > /tmp/lua_soLvsa 2> /tmp/lua_ujogNz
root         1  0.0  0.2  45292  8640 ?        Ss   19:14   0:00 perl /usr/local/openresty/bin/resty /usr/local/bin/kong start --run-migrations --vv
root         7  0.0  0.2  51824 10712 ?        S    19:14   0:00 /usr/local/openresty/bin/../nginx/sbin/nginx -p /tmp/tQ2teKlN0L/ -c conf/nginx.conf
root        43  0.0  0.3 220836 12860 ?        S    19:14   0:00 nginx: master process 
/usr/local/openresty/nginx/sbin/nginx -p /usr/local/kong -c nginx.conf
nobody      50  0.0  0.7 240760 29872 ?        S    19:14   0:00 nginx: worker process
nobody      51  0.0  0.7 240632 29632 ?        S    19:14   0:00 nginx: worker process
nobody      52  0.0  0.7 240632 29800 ?        S    19:14   0:00 nginx: worker process
nobody      53  0.0  0.7 240632 29564 ?        S    19:14   0:00 nginx: worker process

Specifically:

sh -c /usr/local/openresty/nginx/sbin/nginx -p 
/usr/local/kong -c nginx.conf > /tmp/lua_soLvsa 2> /tmp/lua_ujogNz

Is it possible to disable those redirects? I couldn’t find them in the CMD in the Dockerfile or in the docker-entrypoint.sh.

Thanks!

1 Like

I am seeing the exact same issue. Could someone please advise?

@billkoch thx for the details.

See https://github.com/Kong/kong/pull/3297 for an initial attempt to fix it. You might want to give that a try.

@billkoch @higalex While trying to write a test to check the fix, we had a hard time in reproducing the behaviour. Which surprised us.

After some fiddling we figured out that you have created your own Docker files and use the kong start command to start Kong. Instead of using the provided docker files which prepare the Kong prefix, and then directly start the underlying Nginx.

So you can fix your issue by modifying the docker files, or using the provided ones.

Your problem only occurs when all of the following applies:

  1. redirecting Nginx output to stdout/stderr,
  2. while it is NOT running as a daemon (eg in foreground),
  3. and started through the kong start script

Fix merged! Thank you for the report @billkoch and @Tieske for the patch. We will release it in the next minor version.