ModSec v3+OWASP CRS3 On Kong Discussion/Guide

Just started dabbling in running Kong with ModSecurity v3 with the Nginx connector and the OWASP CRS3. Figured I would throw together some snippets of code that may help folks out that want to play around with WAF embedded in their own Kong deployments.

First, the code. Note I am building via a docker file with base image Alpine, running Kong 1.4.3 right now.

Some dependencies you may need(not all relate to the WAF but 3-4 of these do).

RUN apk add --no-cache --virtual build-deps wget tar  ca-certificates \
        && apk add --update --no-cache readline-dev outils-md5 linux-headers bsd-compat-headers m4 \
	yaml-dev build-deps bash patch bash-completion automake bash-doc make gcc g++ libgcc zlib-dev curl curl-dev autoconf libtool perl pcre libxml2 libxml2-dev libmaxminddb-dev yajl yajl-dev pcre-dev unzip tzdata wrk luarocks git

Then the building of libs and some basic settings/files:

######------ ModSec NGINX WAF ------######

RUN git clone --depth 1 -b v3/master --single-branch \
    && chmod -R 777 /ModSecurity \
    && cd ModSecurity \
    && git submodule init \
    && git submodule update \
    && ./ \
    && ./configure \
    && make \
    && make install \
    && mv /ModSecurity/modsecurity.conf-recommended /ModSecurity/modsecurity.conf
    # ^^^ Rename the conf file to be proper ^^^

#Return to ROOT DIR
RUN cd /

#NGINX ModSec v3 connector dynamic module
RUN git clone --depth 1 \
    && chmod -R 777 /ModSecurity-nginx
RUN wget \
    && tar -xzvf v3.2.0.tar.gz \
    && mv owasp-modsecurity-crs-3.2.0 /usr/local \
    && mv /usr/local/owasp-modsecurity-crs-3.2.0/crs-setup.conf.example /usr/local/owasp-modsecurity-crs-3.2.0/crs-setup.conf \
    && echo 'SecAction "id:900000, phase:1, nolog, pass, t:none, setvar:tx.paranoia_level=2"' >> /usr/local/owasp-modsecurity-crs-3.2.0/crs-setup.conf \
    && chmod -R 777 /usr/local/owasp-modsecurity-crs-3.2.0
# Create combined config file
RUN touch /usr/local/modsec_includes.conf \
    && echo "include /ModSecurity/modsecurity.conf" >> /usr/local/modsec_includes.conf \
    && echo "include /usr/local/owasp-modsecurity-crs-3.2.0/crs-setup.conf" >> /usr/local/modsec_includes.conf \
    && echo "include /usr/local/owasp-modsecurity-crs-3.2.0/rules/*.conf" >> /usr/local/modsec_includes.conf \
    && chmod 777 /usr/local/modsec_includes.conf
# Make modifications to config
RUN sed -i "s/SecRuleEngine DetectionOnly/SecRuleEngine On/" /ModSecurity/modsecurity.conf \
    && sed -i "s/SecAuditLogType Serial/SecAuditLogType Concurrent/" /ModSecurity/modsecurity.conf \
    && sed -i "s|SecAuditLog /var/log/modsec_audit.log|SecAuditLog /usr/local/kong/logs/modsec_audit.log|" /ModSecurity/modsecurity.conf

######------ END OF ModSec NGINX WAF ------######

Then its worth noting, I am still using the kong openresty-build-tools deprecated lib for now, which allowed the adding of a 3rd party module with ease in its scripting, I am sure the newer repo offers that too :slight_smile: . Note the --add-module /ModSecurity-nginx line, as my new additional arg.

RUN git clone
RUN rm openresty-build-tools/kong-ngx-build
COPY kong-ngx-build openresty-build-tools/kong-ngx-build
RUN chmod -R 777 /openresty-build-tools
COPY patches/ngx/* /patches/
RUN cd openresty-build-tools \
    && mkdir work \
    && ./kong-ngx-build -p $KONG_PATH \
        --openresty $RESTY_VERSION \
        --openssl $RESTY_OPENSSL_VERSION \
        --luarocks $RESTY_LUAROCKS_VERSION \
	--add-module /ModSecurity-nginx \
        #--pcre $PCRE_VERSION \

Lastly in the nginx.conf produced by Kong you will want to ensure to include these within the server or location block of the conf so ModSec will run on your kong server inbound requests:

  server {
      server_name kong;
      modsecurity on;
      modsecurity_rules_file /usr/local/modsec_includes.conf;

Which enables ModSec and then a link to your file that contains rules/config references.

Then we thought up a number of threat vectors we wanted to validate security on. Some of the first that came to mind we check-listed against:

• Cross site scripting TESTED & VALIDATED IN P2 OWASP
• XML/JSON bomb (infinitely recursive data) TESTED & VALIDATED IN P2 OWASP
• Rate limiting (Kong does this)

So some examples of our tests to ensure ModSec was catching things was:

XML Bomb Payload:

<?xml version="1.0"?>
<!DOCTYPE lolz [
 <!ENTITY lol "lol">
 <!ELEMENT lolz (#PCDATA)>
 <!ENTITY lol1 "&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;">
 <!ENTITY lol2 "&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;">
 <!ENTITY lol3 "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;">
 <!ENTITY lol4 "&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;">
 <!ENTITY lol5 "&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;">
 <!ENTITY lol6 "&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;">
 <!ENTITY lol7 "&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;">
 <!ENTITY lol8 "&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;">
 <!ENTITY lol9 "&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;">



curl --request POST \
  --url \
  --header 'content-type: application/x-www-form-urlencoded' \
  --cookie dtCookie=36%24946B53A8A942C3A95C28C4CA0FD72AD2 \
  --data 'item1=a'\'';DROP TABLE users; SELECT * FROM userinfo WHERE '\''t'\'' '

And well JUMBO payloads you get the point here, didn’t setup a remote XML doc that was massive though:

And a simple example of caught attack:

curl http://localhost:8000?exec=/bin/bash

And what shows up in your logging too:


Final thoughts after this initial play around come to my next steps, which involve how to harden and minimize the rule-set for checks specific to the API Gateway space and relevant attack vectors, as well as productionalizing this WAF and minimizing the FP(false positives) associated with legitimate API transactions.

I began to ask a few q’s in Slack channels around modsec and owasp to gain more insight, these were what some folks had to say:

How to minimize rules quickly:

coderpatros 1 day ago
The rules have different tags.
So if you don’t use php you can SecRuleRemoveByTag “language-php” to remove a bunch of rules
you don’t need. Someone here might be able to link to a definitive list. But doing a find all
for “platform-” “language-” and “attack-” in the core rule set should bring them all back.

On the discussion of minimizing the rule-set for API Gateway use case:

dune73 10:25 AM ( - Author of the #ModSecurity Handbook 2ed. OWASP @CoreRuleSet project co-lead and program chair @SwissCyberStorm.)
Jeremy, what you are probably looking for is a cut down version of rules that cause even less
FPs than PL1. We are thinking about such a thing, maybe developing it together with some CDNs
and internet hosters. It might be a rule exclusion package, or (new idea) a paranoia level 0.
For the time being, I would run PL1, don’t edit the rule set yourself, but disable the parts
of the ruleset you do not like / need with SecRuleRemoveById 9xxxxx-9yyyyy.

FYI: I’m here at OWASP AppSecCali in Santa Monica. Yesterday I had a conversation with the new WAF teamlead from XYZ BIG CO and somebody from ABC BIG CO who works very closely with the WAF team. XYZ BIG CO runs now close to 100 Tbit/s over CRS. Not everything is blocking of course, and users need to pay to get access to the reports. But they run everything through CRS3. They are open to run experimental rules for us (to gauge the FP rate!) and they are also willing to share FPs with us. It sounds too good to be true. ABC BIG CO is more conservative, but they still use 10% of CRS3 on their traffic. Both companies agree that they combined control > 2/3 of the global internet traffic.
Cloudflare is just strong with the number of domains due to their free plan. But if this is about streaming the Olympics, it’s those guys. I’m a wee bit impressed.

My other thoughts would be how cool it would be someday to have OS Kong options to embed/enable the ModSec v3 with OWASP CRS3 or some trimmed down Kong approved rulset to run on Kong instances. This could all be done with Kong branded environment variables as is tradition and let Kong engineers do the heavy lifting with the build process in the background! Best OS API Gateway meets most adopted OS WAF+RuleSet in harmony :laughing: ?


Thanks for reading!


Will note, we initially saw some really major performance hits with the OOTB configurations. Even after tuning you can see latency results here from some performance tests (also in that thread you can see many of my current tunings which may be of use to you):

My current opinion is that the latency/throughput hits are a downer, but not high enough necessarily to shake my interest in throwing some weight and continuing to leverage the application. I do hope subsequent releases may improve upon performance at the WAF layer or something more lightweight can be tuned further to support the API Gateway use case to block malicious traffic from REST/SOAP/Websocket/XYZ future data services.

This will likely be my last post on findings. Wrapping up, we wanted a way to dynamically set properties of our WAF upon deploy time. Lucky for us, Kong has nice built in parsing of the nginx conf and a logic in place for ENV variables being able to override those conf static fields upon Kong’s deployment in a given environment.

In our case we took the Kong defaults lua file:

Added in a few extra fields for some defaults (Otherwise the environment variables won’t take effect):

waf_mode = On
waf_request_file_size_limit = 25000000
waf_request_no_file_size_limit = 20000000
waf_pcre_match_limit = 10000
waf_pcre_match_limit_recursion = 10000
waf_debug_level = 0
waf_audit = Off
waf_paranoia_level = 1

Then what you can do is leverage environment variables via these fields:

KONG_WAF_MODE = On || Off || DetectionOnly
KONG_WAF_REQUEST_FILE_SIZE_LIMIT = 25000000 , for 25MB files
KONG_WAF_REQUEST_NO_FILE_SIZE_LIMIT = 20000000 , for 20 MB payloads(no file in payload)
KONG_WAF_DEBUG_LEVEL = 0 for off, 9 for on basically
KONG_WAF_AUDIT = RelevantOnly || On || Off
KONG_WAF_PARANOIA_LEVEL can be 1 || 2 || 3 || 4

And last but not least, you will need to add to the custom Kong template these references in the Mod Security module directive:

  server {
      server_name kong;
      modsecurity on;
      modsecurity_rules_file /usr/local/modsec_includes.conf;
      modsecurity_rules '
        SecRuleEngine ${{WAF_MODE}}
        SecRequestBodyLimit ${{WAF_REQUEST_FILE_SIZE_LIMIT}}
        SecRequestBodyNoFilesLimit ${{WAF_REQUEST_NO_FILE_SIZE_LIMIT}}
        SecPcreMatchLimit ${{WAF_PCRE_MATCH_LIMIT}}
        SecPcreMatchLimitRecursion ${{WAF_PCRE_MATCH_LIMIT_RECURSION}}
        SecDebugLogLevel ${{WAF_DEBUG_LEVEL}}
        SecAuditEngine ${{WAF_AUDIT}}
        SecAction "id:900000,phase:1,nolog,pass,t:none,setvar:tx.paranoia_level=${{WAF_PARANOIA_LEVEL}}"

This enables us to quickly enable/disable/debug/tune the WAF without having to go through and complete a new build every deployment.

Hope this thread helps serve as a starter for those who want an Open Source WAF solution running on Kong!

Hi jeremyjpj0916, I use official docker image of Kong EE

I’m trying to include ModSec v3 via Dokerfile but it I have this error message

nginx: [emerg] unknown directive “modsecurity” in nginx-kong.conf:66

Do you override some existing docker image of Kong or build Kong from scratch? What command do you use in your Docketfile to start Kong with ModSec?

@mkopeyka Yes I build Nginx/OpenResty from source to accomplish this. Specifically these lines are important, as its where adding ModSec v3 as an nginx module is required:

I am not sure how Kong Enterprise approaches customers that want to build nginx/openresty from source and then apply Kong EE to that, I would touch base with your Kong support team and can hopefully get a technical engineer to reference this forum post and may be able to help guide an implementation approach for EE clients.

I use this cmd to start kong

/usr/local/openresty/nginx/sbin/nginx -p /usr/local/kong -c /usr/local/kong/nginx.conf

If I use “–force” option kong doesn’t start because of these errors:

Error loading shared library No such file or directory (needed by /usr/local/openresty/nginx/sbin/nginx)
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v26Tracer6GlobalEv: symbol not found
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v226propagation_error_categoryEv: symbol not found
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v26Tracer10InitGlobalESt10shared_ptrIS1_E: symbol not found
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v229DynamicallyLoadTracingLibraryEPKcRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE: symbol not found
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v227dynamic_load_error_categoryEv: symbol not found
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v229tracer_factory_error_categoryEv: symbol not found
Error relocating /usr/local/openresty/nginx/sbin/nginx: _ZN11opentracing2v23ext17sampling_priorityE: symbol not found

If I don’t use “–force” option kong doesn’t start because of this error:

nginx: [emerg] unknown directive “modsecurity” in /usr/local/kong/nginx-kong.conf:66

Sorry if I was a little late to responding here, hopefully you figured it out, or an alternative approach to your WAF needs, but this error indicates the module was never properly installed for NGINX to recognize it. Such as the installation step from source that does this:

RUN cd openresty-build-tools \
    && mkdir work \
    && ./kong-ngx-build -p $KONG_PATH \
        --openresty $RESTY_VERSION \
        --openssl $RESTY_OPENSSL_VERSION \
        --luarocks $RESTY_LUAROCKS_VERSION \
	--add-module /ModSecurity-nginx \
        #--pcre $PCRE_VERSION \

Which points at my /ModSecurity-nginx local dir containing my installed ModSec v3 stuff. See my original docker file in the post sample to see how the file structure is setup with what repo’s are leveraged.

Hello there ,
I am using Kong as ingress-controller, i need to install Modescurity on it. however i have installed the Modsec v3+OWASP CRS3 on kong in the same way @jeremyjpj0916 has defined earlier in this thread.

I successfully installed OWASP inside kong Docker Image, in which am installing a custom plugin also.

Now am not sure how can i further implement it at Kubernetes deployment on kong.

Can someone please guide me on this.

Thanks in Advance.