# Kong Gateway plugin for routing OpenAI-compatible requests to local Ollama nodes

**URL:** <https://discuss.konghq.com/t/kong-gateway-plugin-for-routing-openai-compatible-requests-to-local-ollama-nodes/14015>\
**Category:** General\
**Tags:** plugins, kong-gateway, ai-gateway\
**Created:** [May 22, 2026, 6:29pm UTC](https://discuss.konghq.com/t/kong-gateway-plugin-for-routing-openai-compatible-requests-to-local-ollama-nodes/14015 "2026-05-22T18:29:44Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![grulka](https://avatars.discourse-cdn.com/v4/letter/g/f14d63/32.png) [@grulka](https://discuss.konghq.com/u/grulka)\
**Post date:** [May 22, 2026, 6:29pm UTC](https://discuss.konghq.com/t/kong-gateway-plugin-for-routing-openai-compatible-requests-to-local-ollama-nodes/14015/1 "2026-05-22T18:29:44Z")

</div>

Hi Kong community,

We’ve published a Kong Gateway plugin for routing OpenAI-compatible chat completion traffic to one or more `ollama-agent-router` runtime nodes.

The plugin is available on LuaRocks: [kong-plugin-ollama-agent-router](https://luarocks.org/modules/grulka/kong-plugin-ollama-agent-router)

You can check the setup flow, configuration examples, and security details in the README here: [kong-ollama-agent-router](https://github.com/ExeconOne/kong-ollama-agent-router)

What it provides:

- routes OpenAI-compatible `/v1/chat/completions` traffic through Kong to one or more `ollama-agent-router` nodes

- keeps routing decisions, model selection, runtime state, node capabilities, and async job handling behind the Kong Gateway layer

- supports static node-router discovery with weighted nodes

- can prefer already loaded models, respect node weights, and fail over on execution errors

- fetches model specs, runtime capabilities, GPU/VRAM state, queues, and job policies from each node-router instead of duplicating them in Kong config

- supports secured runtime-agent communication using bearer tokens, custom headers, per-node credentials, TLS verification, and optional client certificates

- preserves an OpenAI-compatible gateway surface for clients while delegating local machine state and execution details to the runtime nodes

It’s free to use and MIT licenced. Enjoy and all the feedback is welcomed