Idea for logging

Are any of the tcp/udp/syslog plugins currently able to log to /dev/stdout or /dev/stderror? As I am running in a docker container I would love to be able to direct the robust logging out like that. Theoretically I can do Kong to syslog and then syslog configured to /dev/stdout and /dev/stderror, but is it possible to go directly with any of the plugins? Or would modifying the syslog plugin as a custom plugin be the easiest route? Seems better to not have to make an extra jump from kong -> syslog -> docker /dev/stdout and be able to do kong -> /dev/stdout especially since one change in Kong already enables:

#proxy_access_log = /dev/stdout
#proxy_error_log = /dev/stderror
#admin_access_log = /dev/stdout
#admin_error_log = /dev/stderror

Thoughts on the idea and best way to implement or concerns? I have an EFK stack to handle those docker logs if kong can get em out there :slight_smile: .

@jeremyjpj0916 Have a look at https://github.com/edenlabllc/kong-plugin-stdout-log

1 Like

@samgaw I glanced at it, looks like it uses the same base as kongs other log plugins - ->local basic_serializer = require “kong.plugins.log-serializers.basic” <- ? And did you make this stdout plugin? So it basically is accomplishing the same logging that the syslog plugin does except it directs to stdout right? Seems it can be enabled per API or globally, would you say that is a correct assumption? If so then its exactly what I need :slight_smile: !

@jeremyjpj0916 I didn’t write it but yes, it’s per API and redirects log output to stdout. You should then be able to use whatever Docker/Kubernetes toolchain you have in place to capture & forward the logs to Elasticsearch.

@samgaw Thanks for the reply, I will test it out today, I hope it can be enabled globally so I don’t have to enable the plugin “per api”(where global means every API trans info gets logged to stdout) but ofc I would hope the logging results are on a “per api transaction” basis just like all the other internal kong plugins are :slight_smile: .

Well I thought installing plugins would be easy, seems to not be the case… in my Docker file I have:

RUN apk add --no-cache --virtual .build-deps wget tar ca-certificates \
	&& apk add --no-cache libgcc openssl pcre perl tzdata luarocks \
	&& wget -O kong.tar.gz "https://bintray.com/kong/kong-community-edition-alpine-tar/download_file?file_path=kong-community-edition-$KONG_VERSION.apk.tar.gz" \
	&& echo "$KONG_SHA256 *kong.tar.gz" | sha256sum -c - \
	&& tar -xzf kong.tar.gz -C /tmp \
	&& rm -f kong.tar.gz \
	&& cp -R /tmp/usr / \
	&& rm -rf /tmp/usr \
	&& cp -R /tmp/etc / \
	&& rm -rf /tmp/etc \
	&& apk del .build-deps
	
RUN mkdir /usr/local/kong
RUN chmod -R 777 /usr/local/kong

COPY docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod -R 777 /docker-entrypoint.sh

#Rewrite nginx conf to only enable TLS1.2
RUN sed -i '/ssl_protocols TLSv1.1 TLSv1.2;/c\    ssl_protocols TLSv1.2;' /usr/local/share/lua/5.1/kong/templates/nginx_kong.lua

#Install custom plugins with luarocks
RUN luarocks install kong-plugin-stdout-log

I see in the build output it says:

Removing intermediate container ab4f85d0fa15
Step 12 : RUN luarocks install kong-plugin-stdout-log
 ---> Running in 8390bff3d0d3
Installing https://luarocks.org/kong-plugin-stdout-log-0.0.1-1.src.rock
kong-plugin-stdout-log 0.0.1-1 is now installed in /usr/local (license: MIT)

I also included the plugin in my custom environment variables:
KONG_CUSTOM_PLUGINS : kong-plugin-stdout-log

But the error I get is

2017/12/31 08:09:12 [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:154: kong-plugin-stdout-log plugin is enabled but not installed;
module 'kong.plugins.kong-plugin-stdout-log.handler' not found:No LuaRocks module found for kong.plugins.kong-plugin-stdout-log.handler
	no field package.preload['kong.plugins.kong-plugin-stdout-log.handler']
	no file './kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file './kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/site/lualib/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/openresty/site/lualib/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/lualib/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/openresty/lualib/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta2/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/luajit/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/openresty/luajit/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/.luarocks/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/.luarocks/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/site/lualib/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/openresty/lualib/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file './kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/lib/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/openresty/luajit/lib/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file '/.luarocks/lib/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/openresty/site/lualib/kong.so'
	no file '/usr/local/openresty/lualib/kong.so'
	no file './kong.so'
	no file '/usr/local/lib/lua/5.1/kong.so'
	no file '/usr/local/openresty/luajit/lib/lua/5.1/kong.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file '/.luarocks/lib/lua/5.1/kong.so'
stack traceback:
	[C]: in function 'assert'
	/usr/local/share/lua/5.1/kong/init.lua:154: in function 'init'
	init_by_lua:3: in main chunk
nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:154: kong-plugin-stdout-log plugin is enabled but not installed;
module 'kong.plugins.kong-plugin-stdout-log.handler' not found:No LuaRocks module found for kong.plugins.kong-plugin-stdout-log.handler
	no field package.preload['kong.plugins.kong-plugin-stdout-log.handler']
	no file './kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file './kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/site/lualib/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/openresty/site/lualib/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/lualib/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/openresty/lualib/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta2/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/luajit/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/usr/local/openresty/luajit/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/.luarocks/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.lua'
	no file '/.luarocks/share/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler/init.lua'
	no file '/usr/local/openresty/site/lualib/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/openresty/lualib/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file './kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/lib/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/openresty/luajit/lib/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file '/.luarocks/lib/lua/5.1/kong/plugins/kong-plugin-stdout-log/handler.so'
	no file '/usr/local/openresty/site/lualib/kong.so'
	no file '/usr/local/openresty/lualib/kong.so'
	no file './kong.so'
	no file '/usr/local/lib/lua/5.1/kong.so'
	no file '/usr/local/openresty/luajit/lib/lua/5.1/kong.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file '/.luarocks/lib/lua/5.1/kong.so'
stack traceback:
	[C]: in function 'assert'
	/usr/local/share/lua/5.1/kong/init.lua:154: in function 'init'
init_by_lua:3: in main chunk

Seems like by default Kong does not know where the plugin is maybe? Hard for me to really evaluate the files at this stage because the pod crashes when it errors out like this…

Might be getting somewhere now guys, I actually turned off the custom plugin parameter so i could see the structure of how it is installed and I see the folder like this:

plugin_file_locations

Maybe I need to list the plugins actual name as “stdout-log” and not “kong-plugin-stdout-log” in the KONG_CUSTOM_PLUGINS env variable.

EDIT, solved! that was the problem… Interesting to know the luarocks install name might differ from the name to pass the kong plugin parameter, might be becuse the pluginc creator just did not create them them with the same naming convention.