Hi, I have been trying to get Kong working alongside Istio for routing to a gRPC service (Hello World). For context, I am using the official Kong helm chart at 1.3.0 and Istio 1.4.3. Based on the Kong/Istio blogpost, it seems like I would simply need to enable sidecar injection for the Kong proxy deployment. Kong works and routes correctly to the gRPC service without the sidecar, however, once it is injected, I see the error “rpc error: code = Internal desc = server closed the stream without sending trailers”.
Some other details of my setup:
- I have set the annotation ’ traffic.sidecar.istio.io/includeInboundPorts: “” ’ on the Kong pods to avoid redirection of incoming traffic
- I have set ’ ingress.kubernetes.io/service-upstream: “true” ’
- I have set the port name of the gRPC service to “grpc” for Istio protocol selection
- There are no errors on the gRPC server-side
- I see a log entry ’ [02/Mar/2020:23:12:32 +0000] “POST /helloworld.Greeter/SayHello HTTP/2.0” 200 18 “-” “grpc-go/1.26.0” ’ which seems to imply the request/response was ok
- Looking at debug level logs in the kong envoy sidecar, I see this entry: ’ [Envoy (Epoch 0)] [2020-03-02 05:42:37.055][debug][pool] [external/envoy/source/common/http/http1/conn_pool.cc:206] [C102] response complete ’ which suggests an http1 connection was used. I do not see this log on the server-side envoy sidecar nor when I call the server from another pod within the mesh
- Enabling PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_INBOUND in Istio Pilot and renaming the service port name to something random (so that it default to TCP) appears to get rid of this error, but Istio routing rules would no longer apply in this case.
It seems like the issue is that the http2 connection is being downgraded to http1 in the Kong envoy sidecar. Is there some reason for this or some configuration I am missing? Happy to provide more info if needed. I have poked around quite a bit with no success.