Fire Protection

Large web applications usually consist of several components, including a front end, back-end application servers, and a database. The front end handles a large part of the processing of requests, and if properly configured and tuned, it can help to shorten access times, relieving the load on the underlying application server and providing protection against unauthorized access.

The Nginx web server [1] has quickly established itself in this ensemble. Nginx (pronounced "Engine X") can act as a reverse proxy, load balancer, and static server. The system is known for high performance, stability, and frugal resource requirements. Although Apache is still king of the hill, approximately 30 percent of the top 10,000 websites already benefit from Nginx [2] (Figure 1).

Although Nginx is finding success with all sizes of networks, it is particularly known for its ability to scale to extremely high volumes. According to the project website, "Unlike traditional servers, Nginx doesn't rely on threads to handle requests. Instead it uses a much more scalable event-driven (asynchronous) architecture. This architecture uses small, but more importantly, predictable amounts of memory under load."

Nginx can provide several services as a web front end, including load balancing and hot standby (see the box titled "Balancing and Standby"), reverse proxy, static web service, web/proxy cache (see the box titled "Caching"), SSL offload and SNI, and header cleanup. Figure 2 shows a sample web application configuration with Nginx as the front end.

01 upstream backend {
02 backend1.example.com server weight = 5;
03 server backend2.example.com max_fails fail_timeout = 10s = 5;
04 backend3.example.com server;
05 backend4.example.com server down;
06 backend5.example.com backup server;
07 }
08
09 upstream fallback {
10 fallback1.example.com server: 8081;
11 }
12
13
14 server {
15 %
16 proxy_intercept_errors on;
17 ...
18 Filming location
19 ...
20 error_page 502 @ fallback;
21 proxy_next_upstream HTTP_500 http_502 http_503 http_504 timeout error invalid_header;
22 proxy_pass http://backend;
23

Nginx also comes with some other useful features for busy admins. You can reload the configuration via a HUP signal and even replace the Nginx binary on the fly without any disconnects [3]. The new SPDY protocol is implemented in Nginx version 1.3.15 and will probably be integrated into the stable version 1.4, which is expected in May. (A SPDY patch is available for older versions of the 1.3 branch [4].)

You can download stable and development versions of Nginx at nginx.org; the current versions are 1.2.8 (stable) and 1.3.16 (development). The documentation in the Nginx wiki [5] is comprehensive, including configuration examples, best practice guides, and HowTos. The mailing list is available to newcomers and hard-core users alike. Even Igor Sysoev, the main developer of Nginx, responds from time to time to technically demanding questions.

Like Apache, Nginx comes with several powerful modules that expand and extend its collection of core services. One of the more interesting recent additions to the Nginx ecosystem is the Naxsi module [6], which converts Nginx into a Web Application Firewall (WAP). Naxsi, which was introduced in 2011, is still in development, but many networks are already using it productively. The Naxsi firewall offers promising features to protect web servers against script kiddies, scanners, and other automated tools that search around the clock for low-hanging fruit.

In this article, I describe how Naxsi works and how to implement it to protect your web application infrastructure.

Introducing Naxsi

Naxsi comes with its own core ruleset and is extensible with user-specific rulesets. The configuration takes place in the Nginx context. Thanks to scores for individual rules and customizable thresholds for block actions, the WAF can be adapted to different environments and web applications.

Naxsi can check different values, such as URLs, request parameters, cookies, headers, or the POST body, and it can be switched on or off at location level in the Nginx configuration. Automatic whitelist creation makes it easy to deploy the firewall upstream and rule out 100 percent of false positives. Other tools, such as NX-Utils and Doxi, facilitate administration, report generation, and ruleset updates.

Naxsi comes with NX-utils, which is very useful for generating whitelists and reports. First, the NX-utils collection includes intercept mode, which allows Naxsi to save requests blocked by the WAF for future reports and whitelists in a database, and report mode, which visualizes the stored events. NX-Utils is currently under construction and will provide improved report processing and filtering to analyze the WAF events more precisely in a later version.

Modes: Live vs. Learning

Naxsi can operate in two modes: Live and Learning (Figure 3). Like any WAF or IDS, Naxsi must be adapted for the application. Developers can take very different approaches when programming web applications. For instance, 2KB cookies with large chunks of disorganized data are not uncommon and push even experienced WAF admins to the brink of madness. For these cases, Learning mode allows you to test an application fully behind a protected test domain and generate appropriate whitelists from the queries and events, which you can then feed to an active WAF in Live operations.

In Learning mode, requests are registered but not blocked. Whitelists can be generated from the false positives to prevent them from occurring in Live operation.

Rules

The Naxsi rules are simple in design, flexible in terms of handling, and simpler in structure than Apache ModSecurity or Snort rules. The rules consist of a designator, a search pattern (st or rx ), a short text (msg), the match zone (mz ), the score (s), and the unique ID (id).

Strings and regular expressions are allowed as search patterns, although strings are preferable in general for performance reasons. Match zones indicate the areas of a request in which Naxsi searches for the specified pattern. Match zones are combinable and can be defined using the following values:

• URL : Checks on the search pattern in the URL (server path).
• ARGS: Searches for the pattern in the request arguments.
• FILE_EXT: Tests the file attachment for the search pattern.
• BODY: Checks the body of a POST request for the search pattern; can be further limited with $BODY_VAR:VarName.
• HEADERS : Finds the search pattern in the header of a request; can be further delimited: $HEADERS_VAR:User-Agent, $HEADERS_VAR:Cookie, $HEADERS_VAR:Content-Type, $HEADERS_VAR:Connection, $HEADERS_VAR:Accept-Encoding.

The score indicates the value for each event. Thus, you can create signatures that do not lead to the firewall blocking the connection when left to their own devices, but only in combination with other events. Scores and check rules can be configured and extended freely.

Listing 2 shows an example of a Naxsi control set. Rule 1 in line 1 checks whether the given URL is accessed. The search pattern is a string. The UWA score increases by 8 if the rule applies. Rule 2 tests whether the given regular expression occurs in the BODY of a POST request. Rule 3 checks the authorization header to see whether it contains the given string (Base64-encoded password admin) when the /manager URL is invoked. Rule 4 then tests whether the given string (RFI) occurs in the request parameters, BODY, or Cookie, whether it is a GET or POST request. Finally, Rule 5 checks whether the given string exists in the URL, BODY, ARGS, or Cookie.

01 MainRule "str:/manager/html/upload" "msg:DN SCAN Tomcat" "mz:URL" "s:$UWA:8" id:42000217 ;
02 MainRule "rx:type( *)=( *)[\"|']symbol[\"|']" "msg:DN APP_SERVER Possible RAILS Exploit using type=symbol" "mz:BODY" "s:$ATTACK:8" id:42000233 ;
03 MainRule "str:basic ywrtaw46ywrtaw4=" "msg:APP_SERVER Tomcat admin-admin credentials" "mz:$URL/manager|$HEADERS_VAR:Authorization" "s:$ATTACK:8" id:42000216 ;
04 MainRule "str:http://" "msg:http:// scheme" "mz:ARGS|BODY|$HEADERS_VAR:Cookie" "s:$RFI:8" id:1100 ;
05 MainRule "str:/*" "msg:mysql comment (/*)" "mz:BODY|URL|ARGS|$HEADERS_VAR:Cookie" "s:$SQL:8" id:1003 ;

Core Ruleset

Naxsi comes with its own core ruleset; it contains generic signatures for SQL Injection (SQLi), Remote File Inclusions (RFIs), directory traversal, cross-site scripting (XSS), and some evading tricks, and it provides reliable protection against the exploitation of potential vulnerabilities. For example, the default installation detected and blocked [9] all ModSecurity bypasses by Trustwave Security Challenge in our lab. The core ruleset, consisting of about 50 rules, should always be loaded, even if most false positives occur in this area.

For sites with high potential for interaction, Naxsi will initially generate a lot of false positives. Whitelists help prevent the appearance of false positives. Whitelists can be location-based, which means you can maintain separate configurations for different areas or web applications on one server. Whitelists are easy to store in a file and then add to the configuration via include.

Whitelist rules are similar in structure to detection rules. The designator BasicRule is mandatory; it is followed by the rule ID, for which the exception should apply, followed by the match zone:

BasicRule wl:1000 "mz:$URL:/dontpanic/index.php|$ARGS_VAR:topic";
BasicRule wl:1005 "mz:$URL:/lib/exe/fetch.php|$ARGS_VAR:media";

Naxsi can be switched on or off at the location level; even parallel operation of Learning mode and Live mode is possible for different locations. The configuration of Naxsi is done in three steps: First, the administrator defines the rulesets to load at the HTML/server level. In the location context, the firewall is turned on and off or switched to Learning mode. Finally, the check rules specify the aggregated score, at which the firewall blocks the request. The individual configuration directives can be outsourced to files, which gives you a modular configuration (see Listings 3-5).

01 #LearningMode; #Enables learning mode
02 SecRulesEnabled;
03 #SecRulesDisabled;
04 DeniedUrl "/RequestDenied";

01 CheckRule "$SQL >= 8" BLOCK;
02 CheckRule "$RFI >= 8" BLOCK;
03 CheckRule "$TRAVERSAL >= 4" BLOCK;
04 CheckRule "$EVADE >= 4" BLOCK;
05 CheckRule "$XSS >= 8" BLOCK;
06
07 # UnWantedAccess -> see app-server.rules
08 CheckRule "$UWA >= 8" BLOCK;
09
10 # Identified Attacks
11 CheckRule "$ATTACK >= 8" BLOCK;

01 server {
02     ...
03     include        /etc/nginx/doxi-rules/rules.conf;
04     ...
05     location /dev/ {
06         ...
07         include        /etc/nginx/doxi-rules/learning-mode.rules
08         ...
09     }
10
11     location /live/ {
12         ...
13         include        /etc/nginx/doxi-rules/active-mode.rules
14         include        /etc/nginx/doxi-rules/local_whitelist.rules
15         ...
16     }
17 }

Doxi Rules

You don't need to just stand back and watch while your own server infrastructure is scanned for vulnerabilities day and night. The Doxi rules [10] were developed to detect a number of scans for current or older vulnerabilities (such as the WordPress TimThump-RFI [11]) and provide good protection against script kiddies and automated attack tools.

To allow this to happen, 200 server-specific rules were converted from the Emerging Threats rulesets to Naxsi format; they are freely available via the source code repository. On the basis of Emerging Threats, the rulesets are web_server.rules, scanner.rules, app_server.rules, web_apps.rules, and malware.rules. In a similar way to Emerging Threats, signatures are created when new vulnerabilities appear in popular apps or servers and then announced on the Naxsi mailing list and Doxi blog.

The Doxi tools [12] have emerged to help administrators keep their Naxsi installations up-to-date. Rulesets can be updated automatically from a centralized admin computer. With a simple, five-line Fabric recipe, the administrator can then update several WAFs simultaneously with new rulesets. The Doxi tools are suitable for production use; later versions should provide similar functions to Oinkmaster for updating Snort sensors.

Because Naxsi writes the events it detects to the Nginx error log, a small filter is all it takes to connect the application firewall with Fail2ban and punish multiple attempts with a ban at the IP level (see the Fail2ban configuration in Listing 6).

01 # jail.conf
02 [naxsi]
03
04 enabled = true
05 port = http,https
06 filter = naxsi
07 logpath = /var/log/nginx/error.log
08 maxretry = 6
09 banaction = iptables-multiport-log
10 action = %(action_mwl)s
11
12 # filter.d/naxsi.conf
13
14 [INCLUDES]
15 before = common.conf
16
17 [Definition]
18 failregex = NAXSI_FMT: ip=Host
19 ignoreregex = learning=1

Limits and Prospects

Naxsi is still in development, but it is already stable enough for production use. Currently, developers are extending Naxsi to include plugin APIs that will provide the opportunity to add custom detectors and modules, such as CSRF (Cross-Site Request Forgery) protection, loadable blacklists, or DDoS protection. Some features are still lacking, such as the filtering of outgoing responses, time-based thresholds to prevent brute-force attacks, and filtering by method (GET/POST/CONNECT).

Live Examples

Figure 4 shows the report from a Naxsi firewall that has been exposed to several automated attacks on the web since March 15. As of March 12, the associated networks were blocked at the firewall; as of March 28, the same attacks continued from other IPs, which were then blocked on the firewall four days later. As of April 8, the provider of the IP ranges from which the attacks originated was informed; since then, the attack statistics just contain the normal background level. The attack provided valuable data and was harmless, in that it attempted to use SQL injection to compromise several instances of Dokuwiki. But Dokuwiki works without file-based databases, so: no SQL, no injection. Figure 5 breaks down the suspicious patterns by their attack types.

Listing 7 shows the cumulative report (Doxi result) for six firewalls over a period of four weeks. You can see scans of old vulnerabilities (42000122, 42000089), various exploit scanners (42000227, 42000145, 42000181), and attempts to find vulnerabilities via brute force, even if the corresponding software is not installed.

       ID |  Count
------------------------------------
 42000122 |     2506 | DN SCAN WP Timthumb - Access
 42000004 |     1209 | DN APP_SERVER CGI_file access
 42000089 |     1202 | DN WEB_APPS XMLRPC - Access detected (misc Wordpress/Magento-Vulns)
42000002 |     1182 | DN APP_SERVER PHP-file-access
 42000227 |      977 | DN SCAN Scanner ZmEu exploit scanner
42000059 |      740 | DN WEB_APPS Possible unwanted Upload / Access To mm-forms-community upload dir
42000003 |      337 | DN APP_SERVER ASP_file access
     1007 |      296 | mysql comment (--)
 42000082 |      292 | DN WEB_SERVER Tomcat - Manager - Access
 42000077 |      242 | DN WEB_SERVER LIBWWW_perl-UA detected
42000071 |      187 | DN WEB_APPS PHPMYADMIN setup.php - Access
     1011 |      152 | parenthesis, probable sql/xss
42000210 |      127 | DN APP_SERVER Tomcat Auth Brute Force attempt (admin)
42000020 |      121 | DN APP_SERVER ASPX_file access
 42000145 |      113 | DN SCAN Scanner morfeus
 42000181 |      112 | DN SCAN Scanner webster pro
42000244 |      112 | DN SCAN PHPMyAdmin - Scanner (2)
42000253 |      110 | DN WEB_SERVER possible INC - File - Access
     1003 |       99 | mysql comment (/*)
     1004 |       96 | mysql comment (*/)
42000169 |       86 | DN SCAN Scanner Nmap
42000243 |       80 | DN SCAN PHPMyAdmin - Scanner
     1006 |       75 | mysql keyword (&&)
     1302 |       72 | html open tag
42000216 |       74 | DN APP_SERVER Tomcat admin-admin login credentials
     1102 |       68 | ftp:// scheme
42000073 |       63 | DN SCAN Python-urllib UA, possible Scanner
     1205 |       55 | backslash
     1312 |       52 | ~ character
42000065 |       50 | DN WEB_APPS Magento XMLRPC-Exploit Attempt
42000222 |       47 | DN SCAN Open-Proxy ScannerBot (webcollage-UA)
 42000031 |       20 | DN SCAN Muieblackcat scanner
42000043 |        8 | DN SCAN WhatWeb Web Application Fingerprint Scanner Default User-Agent Detected
42000126 |        8 | DN WEB_APPS WordPress Uploadify-Access
42000151 |        8 | DN SCAN Scanner whatweb
42000229 |        7 | DN APP_SERVER ColdFusion - Vuln-URL-Access administrator
42000230 |        7 | DN APP_SERVER ColdFusion - Vuln-URL-Access adminapi
42000248 |        7 | DN SCAN SQL-Injection Scanner CZxt2

On normal days, an average of between 50 and 150 blocked requests occur per IP per domain, where the attacking IPs are blocked after several attempts; otherwise, the number of attacks per day averages between 300 and 500.

Performance

A number of benchmarks have compared Nginx with other prominent web servers [13] [14], and Nginx asserts itself as one of the fastest open source web servers. A server with Nginx normally remains responsive even under heavy load, so that login and admin tasks are possible if something should go wrong. In comparison, rogue Apache servers quickly become difficult to manage on Linux.

With Naxsi in its armory, Nginx's performance drops, as you might expect; in our test on a quad-core server with 2GB of RAM, performance dropped by about 30 percent. The result of a corresponding benchmark appears in Figure 6.

Conclusions

Nginx is ideal for use as a front end for application servers, regardless of whether they are based on Apache and PHP, Tomcat, JBoss, Rails, Django, Flask, Node.js, or ${insert your favorite application server here} . Using Nginx can speed up existing web applications by several orders of magnitude, and the more visitors a website has, the more likely Nginx is worth using. Naxsi equips the web server with web application firewall functions, without sacrificing too much performance. Whitelists and a learning function on Naxsi ensure that all requirements are met to protect the server against automated exploit attempts.