LIST(GET) Route/Service querying vs API resources

Questions
1

With API resources GET was throttled to 100 with the “size” by default. Documentation with Routes/Services do not mention such a size arg. Is it still there in the background, documentation just lacking it? Personally in the past I have leaned towards simply ?size=2147483647 because the # of api resources in a single gateway should generally stay less than 10k or so imo, and if the Admin API can’t return that in a few seconds max then there are bigger problems lol. I also see offset discussed in Routes/Services but without an arg determining what the offset represents in terms of pagination then what is life :confused: ?

2

You can pass size and offset to routes/services.

For size the default is 100 and max is 1000.

And to get 10000 routes, you need to call 10 times (you will get next after each call).

1 Like
3

Offset is opaque to client. We support both PostgreSQL and Cassandra, and for Postgres everything is easy, but for Cassandra everything is more complex, such as offsets.

1 Like
4

@bungle thanks for the insight, so documentation is just missing it. If I get a chance to fix it up later today I will make a PR to the docs.

Lastly a few more questions(routes and services are same in pagination behavior so lets look at routes to cover both):

"total": 10,

is this the total in the routes table or just the total returned during this data list instance (max total(which would match max size) could be 1000 potentially in that case. I imagine its the size of the data list instance).

So how will we know if we handle a 10k query in 1000 size increments that we have reached the end of the routes table? Will the -

"next": "http://localhost:8001/services/foo/routes?offset=6378122c-a0a1-438d-a5c6-efabae9fb969"

Be blank/null/not passed if we have reached the end? Will the routes offset uuid be the same from the prior query if we are at the end of all the table entries? What is the behavior in this design?

Also curious why next is a full URL rather than just the next uuid to query on for offset, but I guess visually in the doc’s like that it drives home the point that its a driver for pagination on the /routes endpoint :slight_smile: .

EDIT- I looked at this today - https://github.com/Kong/kong/issues/3372

So total is not a thing, offset does come back as just a uuid and I can probably ignore the “next” tag, I guess it depends on implementation taste. But my last question still remains, at the end of the paginated routes/services how does the last call respond? will offset in the body be “”(empty string)/null/not passed if you are at the end of it? Or will offset just be the same UUID over and over at the end of the list?

Thanks,
Jeremy

5

So I investigated this a bit more just through testing, findings are this.

If there are MORE elements to iterate on, then next and offset will populate in the response.

If there are no more elements to iterate on, then next will be null and offset will not even be present in the json response.

Hopefully that is the intended behavior, hope this can clarify that for others too. Will make a PR to kong docs later tonight.

EDIT - Don’t forget to URL encode the offset parameter guys!

6

@jeremyjpj0916 just wanted to thank you. You have been giving us very valuable input, and contributed a lot to community. Yes, we want to make these details right. About the pagination, I think we want to use Web Linking (RFC 8288: https://tools.ietf.org/html/rfc8288), and collection endpoints would then give you JSON array. That way we can get rid of our proprietary enveloping of the response. It would make a nicer interface.

1 Like