Web application firewall with ModSecurity
Context
As soon as you publish an endpoint to internet, it will be scanned by bots and/or malicious people. Web Application Firewall is a good solution to protect your applications and endpoints from those scanning.
K8saas provide a WAF by default based on modsecurity:
And it is confired with the OWASP Core Rule set v3.3.2 (last stable version):
This default configuration covers:
- SQL Injection (SQLi)
- Cross Site Scripting (XSS)
- Local File Inclusion (LFI)
- Remote File Inclusion (RFI)
- PHP Code Injection
- Java Code Injection
- HTTPoxy
- Shellshock
- Unix/Windows Shell Injection
- Session Fixation
- Scripting/Scanner/Bot Detection
- Metadata/Error Leakages
Use case
- Protect your HTTPS application from internet
- Pass Trustnest gotodemo Gate. Look at BSS enabler documentation to go further
Concept
To increase the security of an application or more globally of an infrastructure, you may have to perform a Threat Model like:
In green, you could see a boundaries that segregate areas. Inside an area, a system trusts all sub-systems. To protect these trust boundaries, it's very important to use perimeter protections mechanisms.
To protect an application efficiently, you should combine several mechanisms, from the lowest level, to the highest level.
Let use the OSI model:
There is several types of firewalls:
- The common network firewall works on level 3 and 4 to allow or deny network communication. For example: I would like to allow TCP:443 from internet to my application
- WAF are an applicative firewall. It means that it works on the applicative layers: 5, 6 and 7. For example: I would like to block specific user agents or URI.
This documentation describes how to enable a WAF using a nginx ingress controller with k8saas.
Synthesis of CRS rules
| Rule Name | Description |
|---|---|
| REQUEST-01-COMMON-EXCEPTIONS | Some rules are quite prone to causing false positives in well established software, such as Apache callbacks or Google Analytics tracking cookie. This file offers rules that will allow the transactions to avoid triggering these false positives. |
| REQUEST-10-IP-REPUTATION | These rules deal with detecting traffic from IPs that have previously been involved with malicious activity, either on our local site or globally. |
| REQUEST-12-DOS-PROTECTION | The rules in this file will attempt to detect some level 7 DoS (Denial of Service) attacks against your server. |
| REQUEST-13-SCANNER-DETECTION | These rules are concentrated around detecting security tools and scanners. |
| REQUEST-20-PROTOCOL-ENFORCEMENT | The rules in this file center around detecting requests that either violate HTTP or represent a request that no modern browser would generate, for instance missing a user-agent. |
| REQUEST-21-PROTOCOL-ATTACK | The rules in this file focus on specific attacks against the HTTP protocol itself such as HTTP Request Smuggling and Response Splitting. |
| REQUEST-30-APPLICATION-ATTACK-LFI | These rules attempt to detect when a user is trying to include a file that would be local to the webserver that they should not have access to. Exploiting this type of attack can lead to the web application or server being compromised. |
| REQUEST-31-APPLICATION-ATTACK-RFI | These rules attempt to detect when a user is trying to include a remote resource into the web application that will be executed. Exploiting this type of attack can lead to the web application or server being compromised. |
| REQUEST-41-APPLICATION-ATTACK-SQLI | Within this configuration file we provide rules that protect against SQL injection attacks. SQLi attackers occur when an attacker passes crafted control characters to parameters to an area of the application that is expecting only data. The application will then pass the control characters to the database. This will end up changing the meaning of the expected SQL query. |
| REQUEST-43-APPLICATION-ATTACK-SESSION-FIXATION | These rules focus around providing protection against Session Fixation attacks. |
| REQUEST-49-BLOCKING-EVALUATION | These rules provide the anomaly based blocking for a given request. If you are in anomaly detection mode this file must not be deleted. |
| RESPONSE-50-DATA-LEAKAGES-IIS | These rules provide protection against data leakages that may occur because of Microsoft IIS |
| RESPONSE-50-DATA-LEAKAGES-JAVA | These rules provide protection against data leakages that may occur because of Java |
| RESPONSE-50-DATA-LEAKAGES-PHP | These rules provide protection against data leakages that may occur because of PHP |
| RESPONSE-50-DATA-LEAKAGES | These rules provide protection against data leakages that may occur genericly |
| RESPONSE-51-DATA-LEAKAGES-SQL | These rules provide protection against data leakages that may occur from backend SQL servers. Often these are indicative of SQL injection issues being present. |
| RESPONSE-59-BLOCKING-EVALUATION | These rules provide the anomaly based blocking for a given response. If you are in anomaly detection mode this file must not be deleted. |
| RESPONSE-80-CORRELATION | The rules in this configuration file facilitate the gathering of data about successful and unsuccessful attacks on the server. |
What to do ?
By default, the nginx ingress controller already has a WAF configured, but not enabled.
To enable it, you just have to add the following lines to the annotations section of your ingress:
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecAuditLog /dev/stdout
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
HOWTO
Test a minimal configuration with a WAF ?
A hello world with a WAF is available here
Special case: whitelist IP
note
we strongly recommend to use nginx configuration to whitelist an IP rather than using WAF
Use the following configuration to whitelist an ip here: 192.168.1.100 in your ingress :
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecAuditLog /dev/stdout
SecRule REMOTE_ADDR "@ipMatch 192.168.1.100" "phase:1,id:'981033',t:none,nolog,pass,ctl:ruleEngine=Off"
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
warning
Make the SecRule line is above Include
How to disable a rule by ID
Update the annotation of the ingress with the appropriate IDs:
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecAuditLog /dev/stdout
SecRuleRemoveById "941000-941999"
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
How to override specific rule
Update the annotation of the ingress with the rule. In that case allow http verb GET HEAD POST OPTIONS PUT PATCH DELETE
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecAuditLog /dev/stdout
SecAction "id:900200,phase:1,nolog,pass,t:none, setvar:tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE"
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
How to disable a rule by Tag
Update the annotation of the ingress with the appropriate tag:
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecAuditLog /dev/stdout
SecRuleRemoveByTag "platform-iis"
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
Update Modsecurity Paranoia level
The CRS project sees the 4 Paranoia Levels as follows:
- PL 1: Baseline Security with a minimal need to tune away false positives. This is CRS for everybody running an HTTP server on the internet. If you encounter a false positive on a PL 1 system, please report it via GitHub.
- PL 2: Rules that are adequate when real customer data is involved. Perhaps an off-the-shelf online shop. Expect false positives and learn how to tune them away.
- PL 3: Online banking level security with lots of false positives. From a project perspective, false positives are accepted here, so you need to be able to help yourself by writing rule exclusions.
- PL 4: Rules that are so strong (or paranoid) they are adequate to protect the crown jewels. Use at your own risk and be prepared to get a large number of false positives.
Update the annotation of the ingress with the appropriate tx.paranoia_level value:
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecRuleEngine On
SecAuditLog /dev/stdout
SecAction "id:900000,phase:1,nolog,pass,t:none,setvar:tx.paranoia_level=4"
Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf
HOWTO Monitor WAF ?
K8SAAS provides a built-in WAF Dashboard in our observability stack.
To access it:
- Access to Cluster Monitoring Portal
- Look at "k8saas / WAF" dashboard
- Save as another name (keeping the tags)
- Update the view to send alerts to the right Log analytics: link
Then you see:
Look at the WAF logs ?
First, access to your dedicated log analytics. The documentation of log analytics is here
Then, the logs are accessible using the following query:
waf_CL
| project TimeGenerated, waf_ModSecMsg_s, waf_server_s, waf_ruleId_s
| sort by TimeGenerated asc
If there are no logs, you can follow the previous HOWTO: "Test a minimal configuration with a WAF"
How to access Audit Logs
By default only error logs are available in Log Analytics. If you wish to access "Audit logs" also to get more information on these errors, you should add the following annotation to your ingress:
Update the annotation of the ingress with the appropriate SecAudit values:
annotations:
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
nginx.ingress.kubernetes.io/modsecurity-snippet: |
SecAuditLog /dev/stderr
SecRuleEngine On
SecAuditLogType Serial
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLogFormat JSON
Explanation of these annotations :
- SecAuditLog /dev/stderr: This directs the audit logs to the standard error output.
- SecRuleEngine On: This turns on the ModSecurity rule engine, enabling it to process rules.
- SecAuditLogType Serial: This sets the type of audit logging to 'Serial', meaning all audit data is logged in a single continuous line.
- SecAuditEngine RelevantOnly: This tells the ModSecurity audit engine to log only relevant transactions, i.e., transactions that triggered a rule.
- SecAuditLogParts ABIJDEFHZ: This specifies the parts of the transaction data to be logged. Each letter represents a different part (A=audit log header, B=request headers, etc.). Offical Documentation: https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-(v2.x)#user-content-SecAuditLogParts)
- SecAuditLogFormat JSON: This sets the format of the audit log to JSON.
Now you can :
still get the error log in the log analytics WAF_CL table :
waf_CL
| project TimeGenerated, waf_ModSecMsg_s, waf_server_s, waf_ruleId_s
| sort by TimeGenerated ascHere note down the ruleId.
also all the audit log in log analytics INGRESS_CL table:
ingress_CL
| where transaction_messages_s contains '"ruleid":"XXX"'changing XXX with your wanted ruleid
info
Careful ! The ruleid is the id for the rule itself, ther is no id for the mosecurity "event" itself. It is important to also take into account the time range of the query.
Next Steps
Making Rules
Here is the official documentation
Audit my WAF
To audit your WAF configuration, we recommend to use gotestwaf. This tool generates a table and a PDF to understand the level of protection that your WAF offers.
Prerequisites:
- you should have a k8saas cluster
- an ingress should be configured with your WAF
Here is the steps to first install and run a test over k8saas:
# pull image on k8saas
kubectl create deployment --image=wallarm/gotestwaf gotestwaf -n dev -- /bin/sh -ec "while :; do echo '.'; sleep 5 ; done"
# start the audit
export YOUR_URL=https://the_url_you_want_to_test.com
export GOTESTWAF_POD_NAME=$(kubectl get pods -A --no-headers -o custom-columns=":metadata.name"|grep gotestwaf)
kubectl -n dev -c gotestwaf exec -it $GOTESTWAF_POD_NAME -- /app/gotestwaf --url=$YOUR_URL
Once done you should see an output like
Negative Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| TEST SET | TEST CASE | PERCENTAGE, % | BLOCKED | BYPASSED | UNRESOLVED | SENT | FAILED |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| community | community-lfi | 100.00 | 8 | 0 | 0 | 8 | 0 |
| community | community-rce | 100.00 | 12 | 0 | 0 | 12 | 0 |
| community | community-sqli | 100.00 | 32 | 0 | 0 | 32 | 0 |
| community | community-xss | 100.00 | 524 | 0 | 0 | 524 | 0 |
| community | community-xxe | 100.00 | 2 | 0 | 0 | 2 | 0 |
| owasp | crlf | 100.00 | 8 | 0 | 0 | 8 | 0 |
| owasp | ldap-injection | 85.71 | 12 | 2 | 2 | 16 | 0 |
| owasp | mail-injection | 83.33 | 15 | 3 | 6 | 24 | 0 |
| owasp | nosql-injection | 85.71 | 18 | 3 | 9 | 30 | 0 |
| owasp | path-traversal | 87.06 | 74 | 11 | 25 | 110 | 0 |
| owasp | rce | 100.00 | 14 | 0 | 4 | 18 | 0 |
| owasp | rce-urlparam | 100.00 | 9 | 0 | 0 | 9 | 0 |
| owasp | shell-injection | 85.71 | 36 | 6 | 6 | 48 | 0 |
| owasp | sql-injection | 79.66 | 47 | 12 | 13 | 72 | 0 |
| owasp | ss-include | 80.77 | 21 | 5 | 14 | 40 | 0 |
| owasp | sst-injection | 80.95 | 34 | 8 | 22 | 64 | 0 |
| owasp | xml-injection | 100.00 | 13 | 0 | 0 | 13 | 0 |
| owasp | xss-scripting | 83.15 | 74 | 15 | 47 | 136 | 0 |
| owasp-api | graphql | 100.00 | 6 | 0 | 0 | 6 | 0 |
| owasp-api | graphql-post | 100.00 | 4 | 0 | 0 | 4 | 0 |
| owasp-api | grpc | 0.00 | 0 | 0 | 0 | 0 | 0 |
| owasp-api | rest | 100.00 | 2 | 0 | 0 | 2 | 0 |
| owasp-api | soap | 100.00 | 2 | 0 | 0 | 2 | 0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| DATE: | PROJECT NAME: | AVERAGE SCORE: | BLOCKED (RESOLVED): | BYPASSED (RESOLVED): | UNRESOLVED (SENT): | TOTAL SENT: | FAILED (TOTAL): |
| 2022-05-03 | GENERIC | 93.28% | 967/1032 (93.70%) | 65/1032 (6.30%) | 148/1180 (12.54%) | 1180 | 0/1180 (0.00%) |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
Positive Tests:
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| TEST SET | TEST CASE | PERCENTAGE, % | BLOCKED | BYPASSED | UNRESOLVED | SENT | FAILED |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| false-pos | texts | 0.00 | 51 | 0 | 0 | 51 | 0 |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
| DATE: | PROJECT NAME: | FALSE POSITIVE SCORE: | BLOCKED (RESOLVED): | BYPASSED (RESOLVED): | UNRESOLVED (SENT): | TOTAL SENT: | FAILED (TOTAL): |
| 2022-05-03 | GENERIC | 0.00% | 51/51 (100.00%) | 0/51 (0.00%) | 0/51 (0.00%) | 51 | 0/51 (0.00%) |
+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+
report is ready: reports/waf-evaluation-report-2022-May-03-16-03-43.pdf
Then to get the PDF report on your laptop
# Copy the PDF on local
export PDF_FILE_NAME=the_filename_generated_by_the_previous_command
kubectl cp dev/$GOTESTWAF_POD_NAME:/app/reports/$PDF_FILE_NAME /tmp/$PDF_FILE_NAME -c gotestwaf
Then you should have a report like: WAF report
note
Depending on the modsecurity paranoia level, you will have a different score. With CRS 3.3.2, Level 1 gives D- 61.6/100 example here and Level 4 gives C 73.5/100 - example provided below