Kong Ingress Controller - Datadog Plugin

Kubernetes
1

Hi,

I’m using the kong ingress controller and I’m trying to configure the Datadog plugin. Datadog is deployed to Kubernetes as well via a DaemonSet. The way other services talk to the Datadog agent is via an environment variable set via the Kubernetes downward API. I added the following to my kong deployment:

env:
  - name: DD_AGENT_SERVICE_HOST
    valueFrom:
    fieldRef:
      apiVersion: v1
      fieldPath: status.hostIP

This environment works and I’m able to access $DD_AGENT_SERVICE_HOST from each kong pod.

I’m now trying to configure the Datadog plugin with the following:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  labels:
    global: "true"
  name: datadog
plugin: datadog
config:
  host: $DD_AGENT_SERVICE_HOST

After applying the above, the kong-ingress-controller pod logs the following:
ingress-controller creating a global Kong plugin [ nil datadog nil nil nil nil map[host:$DD_AGENT_SERVICE_HOST] ]: 400 Bad Request {"message":"schema violation (config.host: invalid value: $DD_AGENT_SERVICE_HOST)","name":"schema violation","fields":{"config":{"host":"invalid value: $DD_AGENT_SERVICE_HOST"}},"code":2}

Is it possible to configure Kong plugins with environment variables? If not, any recommendations or workarounds? I tried setting the host value to a specific node’s IP in my cluster and the plugin worked, but that won’t work long-term because node IPs are ephemeral.

Thanks!

2

Indeed, this is a sane and valid deployment model but I’m afraid I’m not aware of any way to do solve with plugin configs.

I’m not very familiar with k8s’s hostNetwork interface, may be you could setup up DNS in such a way that it resolves within the hostNetwork and hence to your daemon set?

3

Hello, Did you find a workaround to make the datadog collect info from kong? I haven’t been able to make the plugin work

4

The Datadog plugin doesn’t allow for IP substitution in that fashion yet–the value must be an actual literal IP string.

Please see this previous answer, but to sum up briefly there are some other limitations that prevent us from implementing this properly yet. Those will hopefully be clear in the fairly near future–ideally I’d like a complete solution for this sometime later this year, but I can’t commit to a specific timeframe yet myself.

5

My temporary resolution was digging through semvers of the DD Helm Chart and I found a version that still supports the latest DD agent and a K8S Deployment object. I ended up deploying a standalone DD statsd Deployment bound to a service in order to give a fixed host to Kong. This does add overhead as it has to traverse nodes as compared to the DaemonSet but it was the only solution I could come up with. In the future I guess I will have to fork that version of the chart in order to maintain metrics.