./jq plugin on github

Questions
1

Just on the jq plugin front, thoughts on storing the config as JSON?

{“doc1”:”https://kong.my.domain/json/doc1,{method=GET}”,
“doc2”:”https://kong.my.domain/json/doc2”,
…
“jq”:”{ some operation on docs}”
}

Or will doing it all in a lua pre/post-function be all that’s needed? So no need for jq as a plugin?

I’m finding pre/post functions a little problematic. Some issue with tab chars? Sometimes the script parses sometimes it doesn’t? Sometimes it parses, then fails when caching?

2

Maybe incrementally transform each request result?
{req1:{url1,jq1},
req2:{url2,jq2},
jq_expression
}
Need some method(s) to pass/filter upstream headers/body

3

The incremental bits of jq could be used to make the intermediary objects easier to work with.

eg

url1 returns
{
“result”: [
“Aachen Cropper”,
“Aachen Lacquer Shield Owl pigeon”,
“Aargau Peak-crested”
]
}

url2 returns
{
“result”: [
“Pacific gull”,
“Belcher’s gull”,
“Olrog’s gull”
]
}

jq1: ‘{pigeons: .result}’
{
“pigeons”: [
“Aachen Cropper”,
“Aachen Lacquer Shield Owl pigeon”,
“Aargau Peak-crested”
]
}

jq2: ‘{gulls: .result}’
{
“gulls”: [
“Aachen Cropper”,
“Aachen Lacquer Shield Owl pigeon”,
“Aargau Peak-crested”
]
}

But what then?

Do we just stitch them together before running the final jq?
{
“pigeons”: [
“Aachen Cropper”,
“Aachen Lacquer Shield Owl pigeon”,
“Aargau Peak-crested”
],
“gulls”: [
“Aachen Cropper”,
“Aachen Lacquer Shield Owl pigeon”,
“Aargau Peak-crested”
]
}

jq ‘{birds: .}’
“birds”: {
“pigeons”: [
“Aachen Cropper”,
“Aachen Lacquer Shield Owl pigeon”,
“Aargau Peak-crested”
],
“gulls”: [
“Pacific gull”,
“Belcher’s gull”,
“Olrog’s gull”
]
}
}

Or should we impose some kind of json structure automatically?
{
“resultf” [
“result1”: […],
“result2”: […]
]
}

There must be a better solution?

4

I’ve heard from a colleague that it may not be a good idea to implement multiple subrequests on the api gateway, as the subsequent requests could be dependent on the prior request. Whether or not they run, and building custom parameters etc. Unless we can describe this logic in our json configuration somehow. Maybe have a jq request body builder?

5

Maybe jq could be extended to make http requests. Or a thin query language built over the top. This QL could select documents from multiple http sources and join them.

And we could make a plugin to execute this QL

6

So far it’s been useful to filter down the payload to make responses smaller.

And to wrap/decorate the content to simplify use of confluent’s Kafka rest api

I’ve used it with sub requests to do json joins.

Haven’t had a need to rename keys. But some people may require this.

7

WRT smaller payloads, we can redact sensitive data on kong