Backup Battle

When Graham Keeling became annoyed with the complexity of network backup software program Bacula [1], he fired up his program editor. The result was the compact network backup solution Burp [2], which runs either in client or server mode, can be controlled completely with parameters, and is therefore also suitable for use in scripts.

Burp's server mode is only supported on Unix/Linux machines, whereas a client installer is also available for 32- and 64-bit Windows systems.

Among other things, Burp uses Librsync to create space-saving delta backups. The main difference between ordinary incremental backups and delta backups is revealed when you back up data that is subject to continuous change. Then, each incremental backup data set comprises the complete contents of the previously created incremental backup data set, as well as all the changes and newly added data since the last full backup. In contrast, delta backups only contain data changed or added since the last backup (full or incremental).

Burp Features

To back up Windows hosts, Burp uses the Windows Volume Shadow Copy Services (VSS), thus guaranteeing non-disruptive and consistent backups of Windows computers. It also supports junctions [3], a type of symbolic link on NTFS filesystems.

On Unix systems, Burp can back up hardlinks, symlinks, fifos, and device nodes over the network, as well as files and directories. As of version 1.1.70 from last year, Burp also backs up Extended Attributes and Access Control Lists (ACLs) under FreeBSD.

Burp is free software and is available under the AGPLv3 on SourceForge [4]. The current stable version (1.3.0) dates back to January of this year, whereas the current version (1.3.6) was released at the end of May.

The associated Windows client is available in the form of an installer file for 32- and 64-bit Windows. Optionally, admins can check out the Burp files from the GitHub repository:

git clone git://github.com/grke/burp.git

Additionally, Burp 1.3.1 has for some time been available from the Ubuntu Precise (12.04) Universe repository and as a package for Debian Sid.

To install the server on Linux, you first need to install the necessary packages; the example here uses an Ubuntu distribution:

sudo apt-get install librsync-dev libz-dev libssl-dev uthash-dev

Then, in the directory with the unpacked sources, type the command:

sudo ./configure ; make ;  make install

Instead of make install, you could also issue a checkinstall to build an installation package, which simplifies the uninstall process later, if needed. To start the server just enter

burp -c /etc/burp/burp-server.conf

where -c defines the path to the configuration file. This step also ensures that Burp generates the necessary certificates when launched (Figure 1).

In the default setting, the server runs in the background as a daemon, but it can also be launched in the foreground using the -F option. For a first test, you can use the supplied server configuration. The Burp server uses port 4971 by default. In a production environment, you will definitely want to customize the backup directory (Figure 2).

Unix Client

The server assigns each client subdirectory in its backup directory. Burp uses it to store the last five backups, although you can change this to your own liking in the server configuration template /etc/burp/burp-server.conf.

In the Unix variant make install also sets up a cron job for the client; it runs every 20 minutes by default. You define the data to be backed up in the respective client config file /etc/burp/burp.conf with the help of an include statement.

The exclude path instruction specifically excludes directories. For a more granular approach, you can exclude individual file types with

exclude_ext = type

as in:

exclude_ext = img

Or, you can use exclude_fs to exclude different types of meta-data, such as temporary filesystems like proc, devfs, devpts, or ramfs from the backup:

exclude_fs = sysfs

With burp, you can exclude files of a certain size (e.g., those of 0MB). The Burp configuration language also provides two size parameters: min_file_size and max_file_size; for example:

min_file_size = 0 Mb

A special feature can help when dealing with files such as fifos and block device nodes. In the default setting, Burp only backs up the names of the pipes but not their contents. However, you can use the following two options to save selected (or all) fifos explicitly:

read_fifo=/path/to/fifo
read_all_fifos=0

The same applies to the block device nodes; use the settings

read_blockdev=/path/to/blockdev
read_all_blockdevs=0

to include them in the backup.

Windows Client

The installer file for the 32- or 64-bit Windows client is available, as mentioned, from the subdirectory of the respective Burp version on SourceForge below Files. In Windows, just double-click the installer file; this is sufficient to install the client. A Wizard queries the server IP address and additionally wants a client name and a password, which the server users identify the client (Figure 3).

Like the cron job on the Unix clients, the installer in the Windows version subsequently sets up a Windows Scheduler task with a polling frequency of 20 minutes and then completes the installation – unfortunately, without the possibility of changing the location of the configuration file. You will find this file in C:\Program Files\burp\burp.conf.

Just as in the Unix version, the directory to be backed up must be added to the default configuration C:\Users through the use of an include statement. After defining further directories with additional include or exclude statements, there is nothing to keep you from using the client.

Using Burp

As mentioned previously, Burp can be controlled completely at the command line and with the use of your own scripts. This functionality is reflected in the extensive parameter set. The easiest way to trigger a backup, b, on the client is to enter the command:

burp -a b

where parameter a stands for "action" and b for "backup." The backup command also understands the options in Table 1.

For example, burp -s l shows you the existing backup on the server ("list"), whereas burp -a l -b 2 lists all the files in backup number 2. If you want to see all the files in all existing backups, enter:

burp -a l -b a

Of course, more complex queries are also possible. For example,

burp -a l -b 3 -r <regular expression>

lists all of the files in backup 3 that match the "regular expression" specified by -r. Regular expressions, of course, also can be used with the restore command. The command:

burp -a r -b 2 -r <regular expression>-d /restore dir

restores all files from backup number 2 that match the specified regular expression to the directory specified in -d. You can easily check the last backup performed with burp -a v.

Bare-Metal Restore of Windows 7

As a practical example, Keeling describes the workaround for a complete bare-metal restore of a Windows PC using the Automated Installation Kit (AIK) for Windows 7 (current version 3.0 is available for download [5]). The starting position for the following steps consists of a complete Burp backup of drive C:. To do this, you just need the option

include C:\

in the Burp client configuration in the file C:\Program Files\burp\burp.conf. However, it is highly recommended, for reasons of space, to exclude pagefile.sys as follows with the exclude option:

exclude = C:/pagefile.sys

The process for starting backups is the same as on the Unix client, either in the command prompt or in PowerShell. You then need to download the almost 1.6GB ISO image of AIK, a collection of tools that supports the configuration and deployment of Windows operating systems. You can use them to automate the Windows installation process, for example. However, AIK also lets you capture deployable Windows images with ImageX, configure images using Deployment Imaging Servicing and Management (DISM), and create Windows PE images.

Additionally, you can use the User State Migration Tool (USMT) in AIK to migrate user profiles and data. Windows AIK also contains the "Volume Activation Management Tool" (VAMT), which gives OEM manufacturers or integrators the ability to automate the activation process for volume licensing using a multiple activation key (MAK). After installing Windows AIK, run the following commands with administrator rights.

copype.cmd amd64 c:\winpe_amd64
cd c:\winpe_amd64
copy winpe.wim iso\sources\boot.wim
dism /mount-wim /wimfile:iso\sources\boot.wim /index:1 /mountdir:mount

Now, you can either pack the complete C:\Program Files\burp directory into the WinPE disk by copying it into the mount\windows directory, or you can store the directory on a file server and enter

dism /unmount-wim /mountdir:mount /commit
oscdimg -n -betfsboot.com iso winpe_amd64.iso

to create an image that you then burn onto a DVD/CD.

The WIM format is used by Microsoft for high-compression images and consumes up to two-thirds less memory than the original data. The copype.cmd script creates a bootable Windows PE ramdisk on CD-ROM. Then, you need to create a bootable disk with the WinPE ISO image, with which you can boot the computer to be restored; this takes you to a spartan prompt of the form X:\windows\system32. You can now create a new C: partition,

select disk 0
create partition primary
active
assign letter=C

and format it:

format fs=ntfs quick

After doing so, you can trigger the restore process by entering

cd C:\Programme\Burp\bin
burp.exe -a r -f

if Burp exists in WinPE. Alternatively, you can start the client installer from a file server, for example, and again configure it with the appropriate parameters (IP address, client name, password).

Next, you can shut down the machine and boot with an ordinary Windows 7 installation disk. To begin, select the language, time zone, currency, and keyboard as usual. Then, in the Setup Wizard, select Repair Your Computer instead of Install Now, and in the subsequent step, choose the last entry, Prompt. You will need to run the diskpart call again, but only to activate the C: partition and to restore the boot sector:

select disk 0
select partition primary
active
bootrec /rebuildbcd

Then, boot a second time from the Windows 7 installation disk and select Repair Your Computer again. However, the system should now be able to boot on its own when you select System Repair. According to Keeling, this workaround does cause some strange entries to appear in the main directory of the restored system, such as C:\$Recycle.bin, C:\Documents and Settings, C:\System Volume Information, C:\Temp, and C:\C:, as well as an additional small partition, E:. Apparently, these directories also exist as hidden elements of the original system.

Alternatives

Alternatives to Burp include Obnam and Backshift, both of which recently released stable versions. Backshift [6], by Dan Stromberg, is a relatively simple program that doesn't support client/server mode or encryption out of the box; however, the tool impresses with its simple handling, and because it compresses and deduplicates the data during backup, it achieves enormous space savings. The author emphasizes that Backshift is not just another tar front end, because the deduplication algorithm leads to much greater space savings than mere compression.

The relatively fresh Backshift version 1.03 [6] is available as a source code (Python code) archive. To install, you need to configure before entering make install in the usual way. There is no need to build because this is Python code. During the configuration, you can choose between the CPython interpreter – in this case, you just need to enter enough ./configure commands – or Pypy [7]. The current 1.8 version of Pypy is regarded as slightly faster and more stable. Then, you can configure like this:

./configure --python /usr/local/pypy-1.8/bin/pypy

The configure parameter --prefix specifies where Backshift is installed; by default, it's in /usr/local. To create a list of data to be backed up, Backshift needs an external program such as find. Calling Backshift, for example, to back up the complete root filesystem, looks like this:

find / -xdev -print0 | backshift --save-directory /path/to/save-directory --backup -subset slash -init-savedir

For further, more complex examples, you can check out the documentation. Additionally, Backshift author Stromberg has gone to the trouble of comparing the current free network backup solutions in an informative study you can find on the Backshift website [6].

Obnam [8] can already look back on six years of development and has been maintained by Lars Wirzenius for the past six years. However, it was not until June of this year that version 1.0 was released under the GPLv3 license. The author's motivation for developing Obnam is stated in the announcement with 1.0. Apparently, the author originally wanted to set up a company for online backups but could not find any software to suit his requirements. He developed Obnam as a simple, manageable backup program.

Obnam is also written in Python and creates backups exclusively using the SFTP protocol. Additionally, Obnam encrypts each backup, which from the viewpoint of the user, is a complete snapshot of the data. To save space, Obnam deduplicates the data. Backups optionally can be launched on the client (which then sends them to the server) or on the server.

The deduplication algorithm is one of the highlights of the software and relies to a great extent on B-trees, in much the way the new Linux filesystem Btrfs [9] does. The software breaks down the data into blocks but only stores identical blocks once, even if they occur in different files in different generations of backups. For a simple call to Obnam, just type:

./obnam backup --repository </path/to/backup> $HOME
According to Wirzenius, the current version can be used in production, but he does want to accelerate the program significantly in the future.

Conclusions

Even with such old topics as backups, the open source world constantly comes up with new solutions; so the major league players, such as Bacula and Amanda, need to keep on their toes. Not only do commercial producers like Arkeia attract customers with free Linux versions, but newcomers assert themselves by offering programs with interesting features such as encryption or deduplication.

The tools presented in this article are perhaps not exactly the tools of choice for use in large companies, but they are perfectly suitable for small businesses. Strategies for cloud or virtualization infrastructures are no more a replacement for backups of individual hosts than are existing high-availability solutions or long-term archiving systems. Also, the question of backup medium and of the selected backup strategy is something a company must consider carefully.

I particularly liked Burp for its command-line control option, which allows admins to develop individual backup strategies. What speaks in favor of commercial solutions, such as Arkeia or SEP sesam, from an organizational point of view is the ability to manage backups as a Virtual Tape Library and to use any tape drive, tape library, or autoloaders as a repository, whether on SCSI, SATA, iSCSI, FC, InfiniBand, SAS, or USB. If you do not need such features or prefer to avoid the cost, you will find that the solutions presented here are powerful backup utilities in dependable Unix style.