Nuts and bolts NSE: Nmap Scripting Engine

Exploring Nmap's scripting engine

Security Script

The Nmap security scanner comes with a built-in scripting engine. We'll take a look at Nmap scripting and how the scripts are organized. By Ron McCarty

The Network Mapper, or Nmap [1], is one of the most popular network scanners available. System administrators and security specialists use Nmap for network scanning, discovery, and inventories. Nmap is useful for these security tasks, but it is also quite useful for network troubleshooting, You can use Nmap to determine whether a service is running or to determine whether a service version has changed. Nmap is also very good at identifying application and operating system versions through fingerprinting of protocols and applications. Although Nmap comes with a graphical user interface, Zenmap, many administrators prefer to work from the command line. Nmap lends itself very well to scripting in a shell interpreter scripting language such as the Bourne again shell (Bash) or a text-processing language such as Perl, sed, or awk. However, Nmap also has a native scripting feature. The Nmap scripting engine (NSE) has several advantages over Bash or a text-processing language:

Phase awareness – Nmap organizes the scanning process into phases. NSE is aware of Nmap phases, which let the programmer avoid complex recursive or if-then algorithms to determine the state of the scan.
Common language for portability – A common language makes the script portable to Nmap systems on other platforms. (A Bash script, on the other hand, would work well in Linux but might not work on a Windows system.)
Community-based distribution – A common language allows for a common standard and a system for distributing scripts either within Nmap or separately.

Nmap's scripting language uses the Lua programming language. Lua is an interpreted language that is "powerful, fast, lightweight" [2]. It is probably most well known as the scripting language used by the popular game World of Warcraft (WoW) [3]; however, Lua is also used with Adobe Lightroom and other tools.

A large collection of scripts is available online at the Nmap.org website [4]. Some of these scripts are also bundled with the Nmap application. The ls command will give you a list of locally installed NSE scripts (Listing 1).

Listing 1: Local NSE Scripts

root@dal01:~# ls -l /usr/share/nmap/scripts
total 564
-rw-r--r-- 1 root root 15655 2009-11-06 03:39 asn-query.nse
-rw-r--r-- 1 root root  1864 2009-11-06 03:39 auth-owners.nse
-rw-r--r-- 1 root root   705 2009-11-06 03:39 auth-spoof.nse
-rw-r--r-- 1 root root  5582 2009-11-06 03:39 banner.nse
~
~
-rw-r--r-- 1 root root  6636 2009-11-06 03:39 sslv2.nse
-rw-r--r-- 1 root root  5376 2009-11-06 03:39 telnet-brute.nse
-rw-r--r-- 1 root root  5780 2009-11-06 03:39 upnp-info.nse
-rw-r--r-- 1 root root 89999 2009-11-06 03:39 whois.nse

The Nmap Reference Guide [5] provides a breakdown of the scripts that are currently distributed with Nmap.

Understanding Nmap Scripts

Nmap organizes the scanning process into five phases (Figure 1). NSE scripts fall into categories associated with the phases of the scanning process as:

Pre-rule scripts – executed before Nmap has run any scans against the network. Pre-rule scripts (called prerules) can take care of housecleaning activities, such as creating tables, determining environmentals, or even generating a list of addresses Nmap will scan against.
Host scripts – executed against each host specified for Nmap once the host's scan is complete. A script to grab the MAC addresses of the LAN hosts is a good example of a script that could run during the host script phase. Host scripts are only executed against existing hosts. The host scripts are referred to as hostrules.
Service scripts – run after host scripts and only run if the particular service is running. Nmap includes many service scripts, and this is likely the phase that will receive the most focus in community-supported scripts. Service scripts are only run for matching service ports. The term portrule identifies this phase.
Post-rule scripts – executed after all the other phases. These scripts (called postrules) support cleanup, reporting, or other logic that should execute after the rest of the scan is finished.

Figure 1: NSE divides the scanning process into five phases.

NSE scripts are divided into the following sections:

Description
Categories
Phase
Action

The description section pro-vides a description of the script included inside double square brackets:

description = [[
Example for Admin magazine
]]

The categories section aptly defines the categories the script is executed in (see "Script Categories.") Associating a script with a category allows it to be executed as part of a category scan. A script can belong to multiple categories. The format for the categories section is

categories = {"default", "safe"}

which specifies that the script will run any time the administrator executes scripts in either the default or safe categories.

Script Categories

The Nmap community classifies scripts by category. If you are looking for a script, or if you plan to write your own scripts, it is important to understand the methodology for how scripts are classified:

Auth – The Auth category comprises 30+ scripts for testing authentication protocols, configuration, and controls on the targeted host. Most of the scripts are related to brute force and vulnerability testing, so the scripts should only be run against networks where intrusion detection testing methods are allowed.
Broadcast – The Broadcast category is currently very small, with a set of six scripts that use broadcast and multicast methods of discovering information on the network. Scripts include Dropbox client identification, host service discovery using DNS service discovery, and multicasting for web services.
Default – The Default category includes 50+ scripts. The Nmap team uses six criteria for placing a script in the Default category: speed, usefulness, verbosity, reliability, intrusiveness (very little), privacy (lack of).
Discovery – The Discovery scripts lend themselves to using network and operating system information to map out network hosts. More than 70 scripts come with Nmap's scanning engine.
DoS – Scripts that can cause service failure or major issues are categorized as denial of service (DoS) scripts. Their use should be weighed carefully against the possibility of crashing essential services. Currently, only two DoS scripts are distributed with Nmap.
Exploit – Exploit scripts try to execute an exploit on the basis of Nmap having identified the exploitable host. Nmap only includes one exploit script in its package.
External – External scripts communicate with hosts other than the local host and target. For example, a script that automatically updates a database with heuristic information is classified as an External script.
Fuzzer – Fuzzer scripts are used for discovering new vulnerabilities or identification techniques and generally work by sending random data to a service.
Intrusive – Intrusive scripts can either crash services and systems or take up large amounts of bandwidth while performing the script.
Malware – Malware scripts are scripts designed to identify malware running on a remote host.
Safe – Safe scripts are network friendly and generally not regarded as intrusive.
Version – The Version category covers scripts that are designed to explicitly identify particular versions of services.
Vuln – The Vuln (vulnerability) category is for identifying (but not exploiting) vulnerabilities.

The phase section identifies the phase when the script will run: hostrule, portrule, postrule, or prerule. (See the discussion of phases earlier in this article.) Note the hostrule and portrule settings require parameters and receive the parameters from Nmap's scanning engine. The Phase section ends with the Lua end command. (The phases are implemented as Lua functions.) Listing 2 shows the portrule code from the html-title.nse script.

Listing 2: Portrule Code

01 portrule = function(host, port)
02     local svc = { std = { ["http"] = 1, ["http-alt"] = 1 },
03                   ssl = { ["https"] = 1, ["https-alt"] = 1 } }
04     if port.protocol ~= 'tcp'
05     or not ( svc.std[port.service] or svc.ssl[port.service] ) then
06         return false
07     end
08     -- Don't bother running on SSL ports if we don't have SSL.
09     if (svc.ssl[port.service] or port.version.service_tunnel == 'ssl')
10     and not nmap.have_ssl() then
11         return false
12     end
13     return true
14 end

The portrule in Listing 2 creates the variables svc.std and svc.ssl as associative arrays (lines 2 and 3) and then checks to see whether the transport protocol is not TCP (line 4) or whether the ports are not running HTTP or HTTP over SSL (line 5). If either of these constraints is not true, the script will simply return false (line 6) and Nmap will not print any results.

On the other hand, if the port is running TCP, the logic continues and the script determines whether Nmap has access to the SSL client libraries. If it does not have the SSL client libraries, it will return false (line 11) and not print anything during execution. If all of these tests pass, the function returns true, and the logic defined in the action section is carried out.

The action section is the algorithm containing the logic of the script once the phase logic is executed. The logic uses standard Lua variables, syntax, and functions, combined with the libraries included with Nmap. The presence of the Nmap NSE libraries save the developer from having to deal with functions such as connecting to the port or reading from a network-connected service. The code in Listing 3, which is from the anonymous FTP Nmap script ftp-anon.nse, shows how easy it is to connect and send data to a socket and check for a status code.

Listing 3: Taking Action

01 socket:set_timeout(5000)
02 try(socket:connect(host.ip, port.number, port.protocol))
03 try(socket:send("USER anonymous\r\n"))
04 try(socket:send("PASS IEUser@\r\n"))
05         while status do
06                 status, result = socket:receive_lines(1);
07                 if string.match(result, "^230") then
08                         isAnon = true
09                         break
10                 end

Using NSE

Many basic NSE scripts are members of the default category. Use the -sC option to run Default scripts (Listing 4).

Listing 4: Running Default Scripts

root@dal01:~# nmap -sC www.mcwrite.net
Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-06 19:35 CST
Interesting ports on blogs.typepad.com (204.9.177.195):
Not shown: 998 filtered ports
PORT     STATE SERVICE
80/tcp   open  http
|  robots.txt: has 7 disallowed entries
|_ /t/trackback /t/comments /t/stats /t/app /.m/ / *
|_ html-title: Ron McCarty's Blog
2909/tcp open  unknown
Nmap done: 1 IP address (1 host up) scanned in 12.93 seconds
root@dal01:~#

The -A option runs the Default scripts and executes several other scanning options. To run a specific script, use --script with the name of the script. For example,

nmap --script html-title

runs the Whois script included with NSE (Listing 5). NSE also lets you run several scripts by separating them with commas, or complete categories by including the category name. The command in Listing 6 runs all Malware category scripts.

Listing 5: Running the html-title Script

root@dal01:~# nmap --script html-title www.mcwrite.net
Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-06 19:18 CST
Interesting ports on blogs.typepad.com (204.9.177.195):
Not shown: 998 filtered ports
PORT     STATE SERVICE
80/tcp   open  http
|_ html-title: Ron McCarty's Blog
2909/tcp open  unknown
Nmap done: 1 IP address (1 host up) scanned in 8.47 seconds
root@dal01:~#

Listing 6: Running Scripts in a Category

root@dal01:~# nmap --script malware www.mcwrite.net
Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-06 19:23 CST
Interesting ports on blogs.typepad.com (204.9.177.195):
Not shown: 998 filtered ports
PORT     STATE SERVICE
80/tcp   open  http
2909/tcp open  unknown
Nmap done: 1 IP address (1 host up) scanned in 11.02 seconds
root@dal01:~#

Wrap Up

In this article, I introduced you to the Nmap scripting engine. I hope this discussion gave you a good basis for determining when you need to use scripts.

Gaining a solid understanding of the various Nmap categories and scripts, as well as some background on the scripting engine itself, will help you use Nmap to its fullest potential. Have fun!