# Array of strings in plugin configuration: order not respected

**URL:** <https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466>\
**Category:** Questions\
**Created:** [July 13, 2018, 4:04pm UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466 "2018-07-13T16:04:36Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![pamiel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/pamiel/32/153_2.png) [@pamiel](https://discuss.konghq.com/u/pamiel)\
**Post date:** [July 13, 2018, 4:04pm UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/1 "2018-07-13T16:04:36Z")

</div>

Dear Kong-ers

I’m currently developing a plugin where a configuration parameters (let’s call it `param`) is an array of strings.  
I have different ways to set the configuration parameters for this plugin:

- Using a single string containing the list of comma separated values. Within a curl command, it would be something like `--data “config.param=value1,value2,value3,value4”`
- Or using the `[]` notation for defining item per item, as what I have seen in an example in the request-transformer plugin documentation (that is heavily using arrays of strings): `--data “config.param[1]=value1” --data “config.param[2]=value2” --data “config.param[3]=value3 --data “config.param[4]=value4”`

The issue is that:

- When using the first method, the order to the strings within the array is kept: the order of the items presents, at the end, in the `config.param` Lua table accessible in the `handler` function of the plugin is the same as in the curl command sent to configure the plugin.
- But when using the second method, the order is NOT respected: the items in the `config.param` Lua table are scrambled

and of course, the order to the items is (very) important for me ☹

I dug into Kong’s code up to the Lapis routes attachment (within `kong/api/init.lua`), dumping the content of the `config` object at the earlier step as possible, to see that the order was already wrong at that level:

```auto
dump(): {
  api_name_or_id = "myApi",
  name = "myPlugin",
  ["config.param"] = {
    ["4"] = "value4",
    ["3"] = "value3"
    ["1"] = "value1",
    ["2"] = "value2",
  }
}

```

Then, obviously, after transformation into the `kong/apis/init.lua/parse_params` function, the order is still wrong:

```auto
dump(): {
  api_name_or_id = "myApi",
  config = {
    param = {
      "value4",
      "value3",
      "value1",
      "value2"
    }
  },
  name = "myPlugin"
}

```

and this is what I get in the `handler`.

So where does the scrambling happens ? Well, it looks it is inside the Lapis framework… more precisely here: [https://github.com/leafo/lapis/blob/master/lapis/application.lua#L449](https://github.com/leafo/lapis/blob/master/lapis/application.lua#L449)  
Tracing the json string before this line, the order within the string is still correct; tracing the Lua table after this call, the order is incorrect ! ☹  
String before calling `cjson` call:

```auto
dump(): "{\"name\": \"myPlugin\", \"config.param[1]\": \"value1\", \"config.param[2]\": \"value2\", \"config.param[3]\": \"value3\", \"config.param[4]\": \"value4\"}"

```

I know that my issue is then not directly inside Kong, but I would get the advice from Kong and Lua experts:

- Is the usage of “[]” in the curl/httpie commands for plugin configuration a recognized way for inputting arrays of string, or is it just a “trick” to be forbidden? Is there alternate ways to input arrays of strings, other than the two I presented above, and that I can test?
- Is it normal that `cjson` does not respect the order of the original string using the [] notation ? Is there any way to enfore the order within `cjson`?

Thanks for your valuable help.

---

<div class="post-metadata">

**Author:** ![jeremyjpj0916](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/jeremyjpj0916/32/1388_2.png) [@jeremyjpj0916](https://discuss.konghq.com/u/jeremyjpj0916)\
**Post date:** [July 17, 2018, 2:49am UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/2 "2018-07-17T02:49:27Z")

</div>

My understanding was Lua doesn’t have a guaranteed order on table entries. One way you can do it is just string with comma separated and then string split the comma when needed. That will maintain your order(not the most elegant solution but hey works).

---

<div class="post-metadata">

**Author:** ![Tieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/tieske/32/54_2.png) [@Tieske](https://discuss.konghq.com/u/Tieske)\
**Post date:** [July 26, 2018, 2:46pm UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/3 "2018-07-26T14:46:20Z")

</div>

what seems to go wrong here is that the indices get decoded as strings. As @jeremyjpj0916 pointed out there is no order in Lua, but that only applies to the hash-part of a table. In this case the numeric indices are strings, which hence end up in the hash part, without explicit order.

your first code block shows the indices as `["4"]` indicating the string values. The second block it has been decoded to a list, with numeric indices, which you can tell by the absence of the indices.

I fail to see why this could not be handled properly?

---

<div class="post-metadata">

**Author:** ![Tieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/tieske/32/54_2.png) [@Tieske](https://discuss.konghq.com/u/Tieske)\
**Post date:** [July 27, 2018, 12:46pm UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/4 "2018-07-27T12:46:16Z")

</div>

@pamiel I cannot reproduce this.

The code doing the transformation is here: [https://github.com/Kong/kong/blob/master/kong/tools/utils.lua#L325-L377](https://github.com/Kong/kong/blob/master/kong/tools/utils.lua#L325-L377)

It is invoked when you do a request like:

```auto
curl -X POST http://localhost:8001/services/example-service/plugins \
  --data "name=request-transformer" \
  --data "config.add.headers[1]=h1:v1" \
  --data "config.add.headers[2]=h2:v1" \
  --data "config.add.headers[3]=h3:v2" \
  --data "config.add.headers[4]=h4:v3"

```

when adding some debug statements:

```lua
  local function decode_array(t)
print("1", require("pl.pretty").write(t))

    local keys = {}
    local len = 0
    for k in pairs(t) do
      len = len + 1
      local number = tonumber(k)
      if not number then
        return nil
      end
      keys[len] = number
    end
print("2", require("pl.pretty").write(keys))

    table.sort(keys)
    local new_t = {}

    for i=1,len do
      if keys[i] ~= i then
        return nil
      end
      new_t[i] = t[tostring(i)]
    end
print("3", require("pl.pretty").write(new_t))

    return new_t
  end

```

I get this in the logs, when doing the above curl command:

```auto
2018/07/27 12:30:51 [notice] 19244#0: *14 [lua] utils.lua:326: decode_array(): 1{
  ["2"] = "h2:v1",
  ["4"] = "h4:v3",
  ["1"] = "h1:v1",
  ["3"] = "h3:v2"
}, client: 127.0.0.1, server: kong_admin, request: "POST /services/example-service/plugins HTTP/1.1", host: "localhost:8001"
2018/07/27 12:30:51 [notice] 19244#0: *14 [lua] utils.lua:338: decode_array(): 2{
  2,
  4,
  1,
  3
}, client: 127.0.0.1, server: kong_admin, request: "POST /services/example-service/plugins HTTP/1.1", host: "localhost:8001"
2018/07/27 12:30:51 [notice] 19244#0: *14 [lua] utils.lua:349: decode_array(): 3{
  "h1:v1",
  "h2:v1",
  "h3:v2",
  "h4:v3"
}, client: 127.0.0.1, server: kong_admin, request: "POST /services/example-service/plugins HTTP/1.1", host: "localhost:8001"

```

Note: the array conversion bails out if the array is malformed (an entry missing for example, or also having a non-numeric key in the table)

---

<div class="post-metadata">

**Author:** ![pamiel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/pamiel/32/153_2.png) [@pamiel](https://discuss.konghq.com/u/pamiel)\
**Post date:** [August 21, 2018, 1:00pm UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/5 "2018-08-21T13:00:10Z")

</div>

Hello @Tieske,

Sorry for the late answer, I was 15000km far from my computer… fighting against earthquakes…

Now I’m back (and safe!), and I’m trying to put the same logs as yours…  
…and I’m realizing that this piece of code has slightly changed in version 0.13.x and 0.14.x… and the tests I performed where on 0.12.3 ! ☹

So by making some copy-paste into my 0.12.3 instance (adding the `decode_array` function and the `decode_args` functions) + modifying the `kong/api/init.lua/parse_params` function in order to make it similar to [https://github.com/Kong/kong/blob/92c87dfa5d0ace7381d877b3ce2d3b737cd6b519/kong/api/init.lua#L38](https://github.com/Kong/kong/blob/92c87dfa5d0ace7381d877b3ce2d3b737cd6b519/kong/api/init.lua#L38)):

```auto
elseif find(content_type, "application/x-www-form-urlencode", 1, true) then
   self.params = decode_args(self.params)

```

then I made it work: the parameters are now well sorted… but only in case the parameters are provided with a content type set to “application/x-www-form-urlencode”, i.e. in the form:

```auto
curl -X POST http://localhost:8001/apis/example/plugins --data "name=myPlugin" --data "config.param[1]=value1" --data "config.param[2]=value2" --data "config.param[3]=value3" --data "config.param[4]=value4"

```

When doing the same test, setting the parameters as a JSON file, with a content-type set to “application/json”, like the following:

```auto
curl -v -X POST http://localhost:8001/apis/example/plugins --header "Content-Type: application/json" --data '{"name":"myPlugin", "config.param[1]":"value1", "config.param[2]":"value2", "config.param[3]":"value3", "config.param[4]":"value4"}'

```

then the parameters are still scrambled, which makes sense as the call to `decode_args` (which calls `decode_array`) is only performed in case the content type is “application/x-www-form-urlencode”.

I tried to complement `parse_params` so that it also manages the “application/json” content-type the same way:

```auto
elseif find(content_type, "application/x-www-form-urlencode", 1, true) then
    self.params = decode_args(self.params)
elseif find(content_type, "application/json", 1, true) then
    self.params = decode_args(self.params)

```

and then it works for JSON input as well… but to be frank, I don’t know what could be the side effects of adding these lines for other use cases!

As a conclusion, I would say that:

- Version 0.12.3 is buggy on this point as the parameters are not sorted… and I think I have no easy way to solve the issue… else than upgrading to 0.14 😛
- Despite the changes made in versions 0.13.x and 0.14.x, I suspect that the same bug is still present, but only if the parameters are provided in JSON format (I only _suspect_ because I only made some tests on my patched 0.12.3 which behavior on this part _should_ look like the 0.14…).

I may make a PR on the 0.14 to manage the JSON content type by adding the few lines mentioned above, but I would like to get your ‘go’ first, regarding the potential side effects of this update.

---

<div class="post-metadata">

**Author:** ![Tieske](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/tieske/32/54_2.png) [@Tieske](https://discuss.konghq.com/u/Tieske)\
**Post date:** [August 28, 2018, 6:53am UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/6 "2018-08-28T06:53:05Z")

</div>

@pamiel the example you gave with `json` doesn’t work because it is not proper `json`. The dots in the parameter names are separators, and hence should be nested structures. Try formatting it as proper `json`.

try this:

`{"name": "myPlugin", "config": { "param": ["value1", "value2", "value3", "value4"]}}`

---

<div class="post-metadata">

**Author:** ![pamiel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.konghq.com/pamiel/32/153_2.png) [@pamiel](https://discuss.konghq.com/u/pamiel)\
**Post date:** [August 28, 2018, 2:57pm UTC](https://discuss.konghq.com/t/array-of-strings-in-plugin-configuration-order-not-respected/1466/7 "2018-08-28T14:57:27Z")

</div>

Oops 😯 , sorry, you are fully right ! My JSON is obvisouly not correct…  
So, running again the tests:

- On v0.12.x, arrays provided in JSON format are well processed (i.e. sorted) while arrays provided in the form of a “application/x-www-form-urlencode” are not…
- That’s, I suppose, the reason why some code has changed on this part in version 0.13.0 (adding the call to `utils.decode_args`) so that, starting from 0.13.0, arrays are now well sorted in all cases (when input as a JSON or as a “application/x-www-form-urlencode” content-type)

Thanks a lot for your help, it is now clear in my mind what I need to patch on 0.12… or (preferably) that I’m safe on this point to migrate on 0.14 !

Indeed, what generated some trouble in my mind is the fact that I’m using `httpie` for easily providing the JSON structure. When providing a string such as `"a.b.c=foo"` in the `httpie` command line, it is transformed into a `"a.b.c": "foo"` element into the JSON, and not what I expected to be: `{"a": {"b": {"c": "foo"}}}`.  
I did not realized this because this “erroneous” JSON is indeed processed correctly by Kong (or let’s say the underlying `cjson` parsing layer), as if it was the expected JSON structure `{"a": {"b": {"c": "foo"}}}`!  
So far, I did not encountered any issue as I was only providing elements containing a string, a boolean, a numerical, and even array values… but it is not working for a _sorted_ array!  
For sure, I cannot blame anyone as the original JSON is not correct (I may ask the question if such a JSON is really supposed to be parsed like that when containing element names using dots…)… and as a good JSON produces a good result!  
Thanks again !