Kong ingress controller always logs "schema violation (headers: length must be at least 1)" and runs into an infinite loop

We are using Kong ingress controller 0.9.1 and have in trouble with the following issue:

E0325 09:46:18.072038       1 controller.go:128] unexpected failure updating Kong configuration:

6 errors occurred:
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
W0325 09:46:18.072083 1 queue.go:112] requeuing technical-tool/ingress-sfds-technical-tool-9-1612497380084, err 6 errors occurred:
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}
while processing event: {Create} failed: 400 Bad Request {“message”:“schema violation (headers: length must be at least 1)”,“name”:“schema violation”,“fields”:{“headers”:“length must be at least 1”},“code”:2}

I reviewed the KIC code, it shows that kong client sends a PUT request to create a route object.

“PUT /routes/0c94f21e-a052-4662-a357-312261054a99 HTTP/1.1” 400 151 “-” “Go-http-client/1.1”

We have too many routes in our system, so that we can’t find the one which configuration causes this issue. And the log also does not give us any clue.

By review the Kong code, we found that Kong does not have the mini length check for headers in route object.

{

“headers”: {
“keys”: {
“type”: “string”,
“match_none”: [
{
“err”: “cannot contain ‘host’ header, which must be specified in the ‘hosts’ attribute”,
“pattern”: “^[Hh][Oo][Ss][Tt]$”
}
],
“required”: true
},
“values”: {
“elements”: {
“type”: “string”
},
“type”: “array”,
“required”: true
},
“type”: “map”
}
},

And seems the error caused in an infinite loop and it’s very similar to Any configuration error breaks synchronization loop · Issue #571 · Kong/kubernetes-ingress-controller · GitHub

I am new to Kong, I would really appreciate that if anyone can tell me a clue.

kong: 2.1.3
kic: 0.9.1

Thanks.

found the root cause.


© 2019 Kong Inc.    Terms  •  Privacy  •  FAQ