How to Add UNIQUE_ID to Response Headers？

[meguoe]

I want to know how to add the UNIQUE_ID response header, as it is important for tracking and debugging requests.

[airween]

Hi @meguoe,

have you seen our issue templates?

Both (for v2 and v3) has this section:

Server (please complete the following information):

• ModSecurity version (and connector): [e.g. ModSecurity v3.0.8 with nginx-connector v1.0.3]
• WebServer: [e.g. nginx-1.18.0]
• OS (and distro): [e.g. Linux, archlinux]

Please fill this block at least. Without that we don't know if it's possible or not.

[meguoe]

Sorry, my information is as follows:
ModSecurity version (and connector): ModSecurity v3.0.13 with nginx-connector v1.0.3
WebServer: nginx-1.27.2
OS (and distro): alpine:3.20.3

[airween]

Hi @meguoe,

thanks. Unfortunately that's not possible what you want with Nginx + libmodsecurity3. Actually the connector is not able to read variables from the library and pass them to the web server.

[meguoe]

Is there any way to achieve this? I need to display an error page with a unique request ID to the user upon rejection, as this is essential for subsequent analysis and troubleshooting. Any insights on how to implement this with libmodsecurity3 would be greatly appreciated. Thank you!

[meguoe]

Also, can I ask if libmodsecurity2 can do it?

[meguoe]

Hi @airween , I already have a solution, using $request_id directly can meet my needs.

# nginx conf
error_page 456 /waf.html;
location = /waf.html {
    internal;
     add_header X-Req-ID $request_id always;
}

# crs-setup.conf
SecDefaultAction "phase:1,log,auditlog,deny,status:456"
SecDefaultAction "phase:2,log,auditlog,deny,status:456"

# response.headers["X-Req-ID"] 
{
    "transaction": {
        "client_ip": "172.17.0.1",
        "time_stamp": "Fri Nov  1 12:31:55 2024",
        "server_id": "8507a00beee563f9f4b408a42c573665339dd511",
        "client_port": 64498,
        "host_ip": "172.17.0.2",
        "host_port": 7073,
        "unique_id": "173043551594.211523",
        "request": {
            "method": "GET",
            "http_version": 1.1,
            "uri": "/?testparam=1+OR+1=1--",
            "headers": {
                "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
                "Sec-Fetch-Site": "none",
                "sec-ch-ua-platform": "\"macOS\"",
                "Upgrade-Insecure-Requests": "1",
                "sec-ch-ua-mobile": "?0",
                "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
                "Cache-Control": "max-age=0",
                "sec-ch-ua": "\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"",
                "Sec-Fetch-User": "?1",
                "Sec-Fetch-Mode": "navigate",
                "Connection": "keep-alive",
                "Host": "127.0.0.1",
                "Sec-Fetch-Dest": "document",
                "Accept-Encoding": "gzip, deflate, br, zstd",
                "Cookie": "language=en",
                "Accept-Language": "zh-CN,zh;q=0.9"
            }
        },
        "response": {
            "http_code": 456,
            "headers": {
                "Server": "nginx",
                "Date": "Fri, 01 Nov 2024 04:31:55 GMT",
                "Content-Length": "3195",
                "Content-Type": "text/html",
                "Last-Modified": "Thu, 31 Oct 2024 07:48:15 GMT",
                "Connection": "keep-alive",
                "ETag": "\"6723363f-c7b\"",
                "X-Req-ID": "5e316f6d6c739ebff42bb19b72fce950"
            }
        },
        "producer": {
            "modsecurity": "ModSecurity v3.0.13 (Linux)",
            "connector": "ModSecurity-nginx v1.0.3",
            "secrules_engine": "Enabled",
            "components": [
                "OWASP_CRS/4.7.0\""
            ]
        },
        "messages": [
            {
                "message": "SQL Injection Attack Detected via libinjection",
                "details": {
                    "match": "detected SQLi using libinjection.",
                    "reference": "v16,10",
                    "ruleId": "942100",
                    "file": "/etc/nginx/waf/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf",
                    "lineNumber": "46",
                    "data": "Matched Data: 1&1c found within ARGS:testparam: 1 OR 1=1--",
                    "severity": "2",
                    "ver": "OWASP_CRS/4.7.0",
                    "rev": "",
                    "tags": [
                        "application-multi",
                        "language-multi",
                        "platform-multi",
                        "attack-sqli",
                        "paranoia-level/1",
                        "OWASP_CRS",
                        "capec/1000/152/248/66",
                        "PCI/6.5.2"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            }
        ]
    }
}

[airween]

hi @meguoe,

thanks for sharing your solution!

[meguoe]

hi @airween , My solution seems to have some issues; during phase:1 blocked, response.headers is empty. Do you have a better solution? My ultimate goal is to add $request_id to the audit log.

[meguoe]

Another question, when phase:1 triggers interception, two identical audit logs are always generated. What is the reason? I think nginx triggered the rule repeatedly after internal redirection. How to deal with this?

# nginx conf
error_page 456 /waf.html;
location = /waf.html {
    internal;
     add_header X-Req-ID $request_id always;
}

# crs-setup.conf
SecDefaultAction "phase:1,log,auditlog,deny,status:456"
SecDefaultAction _"phase:2,log,auditlog,deny,status:456"_

{"transaction":{"client_ip":"172.17.0.1","time_stamp":"Tue Nov  5 16:10:57 2024","server_id":"01d2eebc4e00cfeddd2ca6b4868ed31577fd49ae","client_port":55836,"host_ip":"172.17.0.2","host_port":7073,"unique_id":"173079425770.829853","request":{"method":"GET","http_version":1.1,"uri":"/21212","headers":{"Host":"192.168.32.246","User-Agent":"Abonti","Accept":"*/*"}},"response":{"http_code":456,"headers":{}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":["OWASP_CRS/4.8.0\""]},"messages":[{"message":"Access denied for IP in blacklist","details":{"match":"Matched \"Operator `IpMatch' with parameter `172.17.0.1/24' against variable `REMOTE_ADDR' (Value: `172.17.0.1' )","reference":"v0,10","ruleId":"10000","file":"/etc/nginx/waf/rules/REQUEST-902-USER-CUSTOM-RULES .conf","lineNumber":"10","data":"Matched Data: REMOTE_ADDR:/21212 found in the blacklist","severity":"4","ver":"OWASP_CRS/4.8.0","rev":"","tags":["user-ip-blacklist"],"maturity":"0","accuracy":"0"}}]}}
{"transaction":{"client_ip":"172.17.0.1","time_stamp":"Tue Nov  5 16:10:57 2024","server_id":"01d2eebc4e00cfeddd2ca6b4868ed31577fd49ae","client_port":55836,"host_ip":"172.17.0.2","host_port":7073,"unique_id":"173079425719.508334","request":{"method":"GET","http_version":1.1,"uri":"/21212","headers":{"Host":"192.168.32.246","User-Agent":"Abonti","Accept":"*/*"}},"response":{"http_code":456,"headers":{}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":["OWASP_CRS/4.8.0\""]},"messages":[{"message":"Access denied for IP in blacklist","details":{"match":"Matched \"Operator `IpMatch' with parameter `172.17.0.1/24' against variable `REMOTE_ADDR' (Value: `172.17.0.1' )","reference":"v0,10","ruleId":"10000","file":"/etc/nginx/waf/rules/REQUEST-902-USER-CUSTOM-RULES .conf","lineNumber":"10","data":"Matched Data: REMOTE_ADDR:/21212 found in the blacklist","severity":"4","ver":"OWASP_CRS/4.8.0","rev":"","tags":["user-ip-blacklist"],"maturity":"0","accuracy":"0"}}]}}
{"req_id":"91f77fe9657a20e33e9cf4e4f1039248","ngproxy":"b1a12fc0","protocol":"http","cache":"-","method":"GET","status":"456","host":"192.168.32.246","url":"/21212","proxy":"172.17.0.2","agent":"Abonti","req_time":"0.006","res_size":"1805","req_size":"78","client":"172.17.0.1","server":"-","timestamp":"2024-11-05T16:10:57+08:00","proxy_req_size":"-","proxy_res_size":"-","proxy_con_time":"-"}

[airween]

How looks your rule like? Could you show us?

[meguoe]

How looks your rule like? Could you show us?

This is my test rule
# REQUEST-902-USER-CUSTOM-RULES .conf
SecRule REMOTE_ADDR "@ipMatch 172.17.0.1/24" \
    "id:902101,\
    phase:1,\
    block,\
    t:none,\
    capture,\
    severity:'WARNING',\
    ver:'OWASP_CRS/4.8.0',\
    tag:'user-ip-blacklist',\
    msg:'Access denied for IP in blacklist',\
    logdata:'Matched Data: %{MATCHED_VAR_NAME}:%{MATCHED_VAR} found in the blacklist'"

This is the CRS rule
# REQUEST-913-SCANNER-DETECTION.conf
SecRule REQUEST_HEADERS:User-Agent "@pmFromFile scanners-user-agents.data" \
    "id:913100,\
    phase:1,\
    block,\
    capture,\
    t:none,\
    msg:'Found User-Agent associated with security scanner',\
    logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-reputation-scanner',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/118/224/541/310',\
    tag:'PCI/6.5.10',\
    ver:'OWASP_CRS/4.8.0',\
    severity:'CRITICAL',\
    setvar:'tx.inbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

I also tested other strategies in phase:1, which always produced 2 identical audit logs, but this situation only occurred when using a custom error page, that is, during internal redirection.

[meguoe]

If possible, could you prioritize looking into how to add $request_id to the audit log?