How should I use "curl terminal commands" and browser to test Bot Detection plugin with blacklist bot-detection enabled?

Announcements
1

create a service
[root@contoso ~]# curl -i -X POST
–url http://localhost:8001/services/
–data ‘name=book’
–data ‘url=Your request has been blocked. This could be due to several reasons.’
HTTP/1.1 201 Created
Date: Tue, 15 May 2018 16:41:42 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.13.1

{
“host”: “contoso.com”,
“created_at”: 1526373702,
“connect_timeout”: 60000,
“id”: “bd9f9240-3e9e-47d9-b6f6-858ea740ba7b”,
“protocol”: “http”,
“name”: “book”,
“read_timeout”: 60000,
“port”: 80,
“path”: “/v1/books”,
“updated_at”: 1526373702,
“retries”: 5,
“write_timeout”: 60000
}

create a route for the book service
URL Format http://localhost:8001/services/{name of servie}/routes
[root@contoso ~]# curl -i -X POST
–url http://localhost:8001/services/book/routes
–data ‘paths=/v1/books’
HTTP/1.1 201 Created
Date: Tue, 15 May 2018 16:41:55 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.13.1

{
“created_at”: 1526373715,
“strip_path”: true,
“hosts”: null,
“preserve_host”: false,
“regex_priority”: 0,
“updated_at”: 1526373715,
“paths”: [
“/v1/books”
],
“service”: {
“id”: “bd9f9240-3e9e-47d9-b6f6-858ea740ba7b”
},
“methods”: null,
“protocols”: [
“http”,
“https”
],
“id”: “b4fe9cc7-0c90-4982-8d21-f8aad3c09760”
}

URL Format http://localhost:8001/services/{name of servie}/plugins
[root@contoso ~]# curl -i -X POST
–url http://localhost:8001/services/book/plugins
–data “name=cors”
–data “config.origins=http://contoso.com”
–data “config.methods=GET, POST”
–data “config.headers=Accept, Accept-Version, Content-Length, Content-MD5, Content-Type, Date, X-Auth-Token”
–data “config.exposed_headers=X-Auth-Token”
–data “config.credentials=true”
–data “config.max_age=3600”
HTTP/1.1 201 Created
Date: Tue, 15 May 2018 17:01:20 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.13.1

{
“created_at”: 1526403680000,
“config”: {
“methods”: [
“GET”,
“POST”
],
“exposed_headers”: [
“X-Auth-Token”
],
“max_age”: 3600,
“headers”: [
“Accept”,
“Accept-Version”,
“Content-Length”,
“Content-MD5”,
“Content-Type”,
“Date”,
“X-Auth-Token”
],
“credentials”: true,
“origins”: [
“http://contoso.com”
],
“preflight_continue”: false
},
“id”: “c7d7e10c-0d5e-49aa-bbdb-46640589db0f”,
“enabled”: true,
“service_id”: “bd9f9240-3e9e-47d9-b6f6-858ea740ba7b”,
“name”: “cors”
}

URL Format http://localhost:8001/routes/{route_id}/plugins
[root@contoso ~]# curl -i -X POST
–url http://localhost:8001/routes/b4fe9cc7-0c90-4982-8d21-f8aad3c09760/plugins
–data “name=cors”
–data “config.origins=http://contoso.com”
–data “config.methods=GET, POST”
–data “config.headers=Accept, Accept-Version, Content-Length, Content-MD5, Content-Type, Date, X-Auth-Token”
–data “config.exposed_headers=X-Auth-Token”
–data “config.credentials=true”
–data “config.max_age=3600”
HTTP/1.1 201 Created
Date: Tue, 15 May 2018 17:01:53 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.13.1

{
“created_at”: 1526403714000,
“config”: {
“methods”: [
“GET”,
“POST”
],
“exposed_headers”: [
“X-Auth-Token”
],
“max_age”: 3600,
“headers”: [
“Accept”,
“Accept-Version”,
“Content-Length”,
“Content-MD5”,
“Content-Type”,
“Date”,
“X-Auth-Token”
],
“credentials”: true,
“origins”: [
“http://contoso.com”
],
“preflight_continue”: false
},
“id”: “84691e27-389d-4a4d-9785-08b4ac6dbb00”,
“enabled”: true,
“route_id”: “b4fe9cc7-0c90-4982-8d21-f8aad3c09760”,
“name”: “cors”
}

URL Format http://localhost:8001/services/{name of servie}/plugins
[root@contoso ~]# curl -i -X POST
–url http://localhost:8001/services/book/plugins
–data “name=bot-detection”

HTTP/1.1 201 Created
Date: Tue, 15 May 2018 16:48:17 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.13.1

{
“created_at”: 1526402896000,
“config”: { },
“id”: “888e081e-68a5-48b2-8aa1-06e870af67c7”,
“enabled”: true,
“service_id”: “bd9f9240-3e9e-47d9-b6f6-858ea740ba7b”,
“name”: “bot-detection”
}

URL Format http://localhost:8001/routes/{route_id of the book service}/plugins

[root@contoso ~]# curl -i -X POST
–url http://localhost:8001/routes/b4fe9cc7-0c90-4982-8d21-f8aad3c09760/plugins
–data “name=bot-detection”
HTTP/1.1 201 Created
Date: Tue, 15 May 2018 16:49:29 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.13.1

{
“created_at”: 1526402969000,
“config”: { },
“id”: “464b757f-882a-42e5-bc60-d8b2c551d96b”,
“enabled”: true,
“route_id”: “b4fe9cc7-0c90-4982-8d21-f8aad3c09760”,
“name”: “bot-detection”
}

How can I use the curl terminal command to see the effect of the client browser operating system IP being added to the blacklist that can’t be accessed to the resource interface address of http://contoso.org:8000/v1/books?

2

hi - the bot-detection plugin works by parsing one of the request headers of each request, called User-Agent, with a known list of values used on that header by known bots. You can see the list here:

curl has its own User-Agent header, but you can “spoof” it with the -A option. For example, the following request would have a User-Agent similar to GoogleBot. If everything is properly configured, it should be caught and rejected by the bot-detection plugin.

curl -A "Googlebot" http://contoso.org:8000/v1/books

You can use any string that matches the rules I linked earlier for the User-Agent.