# Configurig Mistral with Kong's AI Semantic Plugin **URL:** \ **Category:** Questions\ **Tags:** kong-gateway, ai-gateway\ **Created:** [February 25, 2025, 3:27pm UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384 "2025-02-25T15:27:15Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![jesperancinha](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/jesperancinha/32/4340_2.png) [@jesperancinha](https://discuss.konghq.com/u/jesperancinha)\ **Post date:** [February 25, 2025, 3:27pm UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/1 "2025-02-25T15:27:15Z")
I’ve been having a ton of issues trying to get the AI Semantic plugin to work with Mistral. After setting up the plugin using Kong Konnect I was finally able to enable the plugin to work with the routes having an hybrid gateway manger and a node working on my machine in the same network as the REDIS database that I am using for the same effect. Unfortunatelly, everytime I activate the plugin, all requests get rejected and I get a 400 Bad Requests. I tryed to change parameters and made a ton of experiments, but the only thing I was finally able to do was to simply use the AI proxy plugin to communicate with Mistral and that works like a breeze. Not the AI semantic prompt guard plugin however. So my configuration for Mistral looks like this in terms of the AI proxy: ```auto curl -X POST \ https://eu.api.konghq.com/v2/control-planes/{control_plane_id}/core-entities/services/{service_id}/plugins \ --header "accept: application/json" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer KONG_API_KEY" \ --data '{ "name": "ai-proxy", "config": { "model": { "name": "mistral-medium", "provider": "mistral", "options": { "mistral_format": "openai", "upstream_url": "https://api.mistral.ai/v1/chat/completions" } }, "auth": { "header_name": "Authorization", "header_value": "Bearer MISTRAL_API_KEY" }, "route_type": "llm/v1/chat" } }' ``` and it looks like this for the AI Semantic Prompt Guard plugin: ```auto curl -X POST \ https://eu.api.konghq.com/v2/control-planes/{control_plane_id}/core-entities/routes/{route_id}/plugins \ --header "accept: application/json" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer KONG_API_KEY" \ --data '{ "name": "ai-semantic-prompt-guard", "config": { "rules": { "match_all_conversation_history": true, "allow_prompts": ["Questions about StarWars"], "deny_prompts": ["Questions about StarTrek"] }, "embeddings": { "auth": { "header_name": "Authorization", "header_value": "Bearer MISTRAL_API_KEY" }, "model": { "provider": "mistral", "name": "mistral-embed", "options": { "upstream_url": "https://api.mistral.ai/v1/embeddings" } } }, "vectordb": { "dimensions": 1024, "distance_metric": "cosine", "strategy": "redis", "threshold": 0.1, "redis": { "host": "redis", "port": 6379 } } } } ' ``` I configued the service and the route manually in Kong Konnect after createing a hybrid mode gateway manager and having two containers running locally. One for Kong and the other for Redis. However nothing works yet and the plugin only blocks the requests. The full examples can be found on my repo at [GitHub - jesperancinha/kong-test-drives: Kont test project](https://github.com/jesperancinha/kong-test-drives) on folder: [kong-test-drives/kong-ai at main · jesperancinha/kong-test-drives · GitHub](https://github.com/jesperancinha/kong-test-drives/tree/main/kong-ai). Can anyone help me with this and could I be missing at this point? Please let me know! Cheers! PS: I dot not want to use OpenAI for this example. I specifically want to use Mistral. Thank you! ---
**Author:** ![Wangchong\_Zhou](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/wangchong_zhou/32/899_2.png) [@Wangchong\_Zhou](https://discuss.konghq.com/u/Wangchong_Zhou)\ **Post date:** [February 26, 2025, 6:56am UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/2 "2025-02-26T06:56:28Z")
Hi @jesperancinha thanks for reaching out. When you mention Kong returns 400, what does the error message say? ---
**Author:** ![jesperancinha](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/jesperancinha/32/4340_2.png) [@jesperancinha](https://discuss.konghq.com/u/jesperancinha)\ **Post date:** [February 26, 2025, 8:22am UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/3 "2025-02-26T08:22:13Z")
> [@Wangchong\_Zhou](#): > > Hi Hello there, It always says 400 Bad Request and a reference number and that’s it. Nothing else and there are also no logs being recorded of any attempt to connect to Redis and so I’m not even sure of Kong was able to establish a connection with Redis. Both containers have no logs regarding that. ---
**Author:** ![JackGPT](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JackGPT](https://discuss.konghq.com/u/JackGPT)\ **Post date:** [March 6, 2025, 4:36pm UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/4 "2025-03-06T16:36:53Z")
Hey @jesperancinha It might be the same problem I had: you need to run a redis image with FT indexes installed. That isn’t in Docker Hub “redis” image, you need e.g: `docker run -it --rm --name redis -p 6379:6379 redis/redis-stack-server` You able to check again with this image? ---
**Author:** ![JackGPT](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JackGPT](https://discuss.konghq.com/u/JackGPT)\ **Post date:** [March 6, 2025, 4:48pm UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/5 "2025-03-06T16:48:04Z")
It is also the wrong model for Mistral embeddings I think. `mistral-medium` is a chat model. This is my configuration (Mistral only support maximum 1024 vectors): ```yaml - name: ai-semantic-cache config: embeddings: auth: header_name: "Authorization" header_value: "{vault://env/MISTRAL_AUTH_HEADER}" model: provider: mistral name: mistral-embed vectordb: strategy: redis distance_metric: euclidean dimensions: 1024 threshold: 0.2 redis: host: redis port: 6379 ``` ---
**Author:** ![jesperancinha](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/jesperancinha/32/4340_2.png) [@jesperancinha](https://discuss.konghq.com/u/jesperancinha)\ **Post date:** [March 7, 2025, 10:05am UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/6 "2025-03-07T10:05:46Z")
Hello there @JackGPT , Thanks for reaching out. Thanks to your input I was able to connect to Redis, and I can finally see connection logs that confirm that the vector database is being created. However, there seems to be a problem still because I still get a “bad request”. This is my current configuration. The AI proxy-plugin looks like this: ```auto curl -X POST \ "https://eu.api.konghq.com/v2/control-planes/$CONTROL_PLANE_ID/core-entities/services/$ROUTE_ID/plugins" \ --header "accept: application/json" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer $KONG_API_KEY" \ --data "{ 'name': 'ai-proxy', 'config': { 'model': { 'name': 'mistral-medium', 'provider': 'mistral', 'options': { 'mistral_format': 'openai', 'upstream_url': 'https://api.mistral.ai/v1/chat/completions' } }, 'auth': { 'header_name': 'Authorization', 'header_value': 'Bearer $MISTRAL_API_KEY' }, 'route_type': 'llm/v1/chat' } }" ``` Here I have a question. Do I also need the “ai-proxy” to work with “mistral-embed” I couldn’t get it to work like that. I was able to configure that though in the “ai-semantic-prompt-guard” though: ```auto curl -X POST \ "https://eu.api.konghq.com/v2/control-planes/$CONTROL_PLANE_ID/core-entities/services/$ROUTE_ID/plugins" \ --header "accept: application/json" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer $KONG_API_KEY" \ --data " { 'name': 'ai-semantic-prompt-guard', 'config': { 'embeddings': { 'auth': { 'header_name': 'Authorization', 'header_value': $MISTRAL_API_KEY }, 'model': { 'provider': 'mistral', 'name': 'mistral-embed', 'options': { 'upstream_url': 'https://api.mistral.ai/v1/embeddings' } } }, 'rules': { 'match_all_conversation_history': true, 'allow_prompts': [ 'Questions about the sun' ] }, 'vectordb': { 'strategy': 'redis', 'distance_metric': 'euclidean', 'dimensions': 1024, 'threshold': 0.2, 'redis': { 'host': '$REDIS_HOST', 'port': 6379 } } } } " ``` But still, in both cases, I’m still getting a “bad request” the moment I activate the semantic plugin. Here is an example of an error I get: ```auto curl -X POST http://localhost:8000/mistral \ > -H "Content-Type: application/json" \ > -d '{ quote> "messages": [ quote> { quote> "role": "system", quote> "content": "tell me the color of the sun" quote> } quote> ] quote> }' {"error":{"message":"bad request"}}% ``` When I deactivate it, then I get a correct response from Mistral. Let me know if you see anything in this config that is wrong. It may be that I need to also use embed in the “ai-proxy” but I’m not entirely sure. ---
**Author:** ![JackGPT](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JackGPT](https://discuss.konghq.com/u/JackGPT)\ **Post date:** [March 14, 2025, 5:32pm UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/7 "2025-03-14T17:32:48Z")
I think your embeddings auth might be wrong. Maybe? `'header_value': $MISTRAL_API_KEY` Mistral accepts API key like `Bearer $MISTRAL_API_KEY` ---
**Author:** ![jesperancinha](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/jesperancinha/32/4340_2.png) [@jesperancinha](https://discuss.konghq.com/u/jesperancinha)\ **Post date:** [March 15, 2025, 9:51am UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/8 "2025-03-15T09:51:13Z")
I just checked it. It didn’t work. The ai-semantic-cache plugin does work with Mistral very well actually and kong’s webpage provides enough information to do that. I got it to work with the ai-proxy and the ai-semantic-cache plugin. However, with the same configurartion, it just doesn’t seem to work with the ai-semantic-prompt plugin yet. Would this have something to do with possible limitations of Mistal’s API Key? It is quite impossible at this point to assess anything because I can’t get any good log information anywhere about why I simply get a Bad Request. It is, however good to know that the ai-semantica-cache plugin does work and very werll 🙂 Let me know if you have any idea. I’ve updated the repo, this is my current config: ````auto curl -X POST \ "https://eu.api.konghq.com/v2/control-planes/$CONTROL_PLANE_ID/core-entities/routes/$ROUTE_ID/plugins" \ --header "accept: application/json" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer $KONG_API_KEY" \ --data ' { "name": "ai-semantic-prompt-guard", "config": { "embeddings": { "auth": { "header_name": "Authorization", "header_value": "Bearer '$MISTRAL_API_KEY'" }, "model": { "provider": "mistral", "name": "mistral-embed", "options": { "upstream_url": "https://api.mistral.ai/v1/embeddings" } } }, "rules": { "match_all_conversation_history": true, "allow_prompts": [ "Questions about the sun" ] }, "vectordb": { "strategy": "redis", "distance_metric": "euclidean", "dimensions": 1024, "threshold": 0.2, "redis": { "host": "'$REDIS_HOST'", "port": 6379 } } } } '``` ```` ---
**Author:** ![JackGPT](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@JackGPT](https://discuss.konghq.com/u/JackGPT)\ **Post date:** [March 17, 2025, 11:39am UTC](https://discuss.konghq.com/t/configurig-mistral-with-kongs-ai-semantic-plugin/13384/9 "2025-03-17T11:39:28Z")
I’m not sure on this one, it works over here, I really think your header value is still wrong; it should be very specific: ```json { "name": "ai-semantic-prompt-guard", "config": { "embeddings": { "auth": { "header_name": "Authorization", "header_value": "Bearer '$MISTRAL_API_KEY'" }, ``` Remove the single quotes around the key: ```json { "name": "ai-semantic-prompt-guard", "config": { "embeddings": { "auth": { "header_name": "Authorization", "header_value": "Bearer $MISTRAL_API_KEY" }, ``` Same as ai-proxy exactly. Maybe we could just clear this up on the Community Slack / Zoom, I’ll find you.