Monitor Kong and capture exceptions/errors with NewRelic

What is the suggest method to monitor kong with a monitoring tool such as newrelic?

We are currently trying to see if Kong will fit our needs in terms of gateway, and we are seeing the following errors when calling the API with malformored requests.

We would like to capture the below with Newrelic. Does Kong provide a plugin for gateway monitoring.

 | 2018/07/03 22:10:02 [error] 46#0: *35347 lua coroutine: runtime error: /usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc:0: attempt to index a string value
 | stack traceback:
 | coroutine 0:
 | 	/usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc: in function ''
 | 	/usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc: in function </usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc:0>
 | 	/usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc: in function 'normalize_nested_params'
 | 	/usr/local/openresty/site/lualib/kong/api/init.ljbc: in function </usr/local/openresty/site/lualib/kong/api/init.ljbc:0>
 | coroutine 1:
 | 	[C]: in function 'resume'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:393: in function 'handler'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'resolve'
 | 	/usr/local/openresty/site/lualib/kong/api/init.ljbc: in function 'handler'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'handler'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:163: in function </usr/local/share/lua/5.1/lapis/application.lua:159>
 | 	[C]: in function 'xpcall'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:159: in function 'dispatch'
 | 	/usr/local/share/lua/5.1/lapis/nginx.lua:215: in function 'serve_admin_api'
 | 	content_by_lua(nginx-kong.conf:294):2: in function <content_by_lua(nginx-kong.conf:294):1>, client: 192.168.160.1, server: kong_admin, request: "POST /routes/ HTTP/1.1", host: "localhost:8001"
 | 2018/07/03 22:10:02 [error] 46#0: *35347 [lua] init.ljbc:0: handle_error(): /usr/local/share/lua/5.1/lapis/application.lua:397: /usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc:0: attempt to index a string value
 | stack traceback:
 | 	/usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc: in function ''
 | 	/usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc: in function </usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc:0>
 | 	/usr/local/openresty/site/lualib/kong/api/api_helpers.ljbc: in function 'normalize_nested_params'
 | 	/usr/local/openresty/site/lualib/kong/api/init.ljbc: in function </usr/local/openresty/site/lualib/kong/api/init.ljbc:0>
 |
 | stack traceback:
 | 	[C]: in function 'error'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:397: in function 'handler'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'resolve'
 | 	/usr/local/openresty/site/lualib/kong/api/init.ljbc: in function 'handler'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'handler'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:163: in function </usr/local/share/lua/5.1/lapis/application.lua:159>
 | 	[C]: in function 'xpcall'
 | 	/usr/local/share/lua/5.1/lapis/application.lua:159: in function 'dispatch'
 | 	/usr/local/share/lua/5.1/lapis/nginx.lua:215: in function 'serve_admin_api'
 | 	content_by_lua(nginx-kong.conf:294):2: in function <content_by_lua(nginx-kong.conf:294):1>, client: 192.168.160.1, server: kong_admin, request: "POST /routes/ HTTP/1.1", host: "localhost:8001"

I don’t know of any plugins or integrations that store the lua stacktrace errors in an external location(besides to stdout or the nginx logs). We are personally running dynatrace + splunk + (netdata + prometheus + grafana) for some resiliency in logging.

Dynatrace - Monitors overall server health + api tx logging + database health(for cassandra)
Splunk - External view into API transaction info and an excellent tool to perform manipulate and evaluate your gateway transaction data.
Netdata+Prometheus+Grafana - Probably the choice combo for 100% opensource groups who don’t have budget for the above products. Netdata provides nice server monitoring and alerting + it acts as a statsD data sink for Kongs statsD plugin. Prometheus then scrapes Netdata data and then Grafana visualizes the Prometheus data into elegant dashboards. Grafana has lots of neat alerting baked into it as well with emails and visual graph alerts at time of issues.

I am kinda curious what you are running thats creating that lua stacktrace error though and what kong version+settings you rock.

Hey @jeremyjpj0916 thanks for the detailed reply.

I will look into the stack you’ve recommended.

I was able to hit that stack trace by sending json string with application/x-www-form-urlencoded mime type to the admin API. The API just returned a basic 500 error, but the stacktrace was printed on the console.

@billli did you find a solution? I’m curious if you tried https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/nginx-monitoring-integration ? Thanks in advance for anything you can share!