Jailbreak

Virtual machines give the impression of a small jail, but administrators should not be fooled by this idea. As early as the Black Hat security conference in 2011, Nelson Elhage presented a breakout vector [1] that exploited vulnerabilities in the contemporary versions of KVM or Qemu.

Installing the Virtio driver on the guest also allowed a breakout by exploiting existing or undiscovered bugs. Once malware gains control of the host system, it can also directly hijack and control the other virtual machines running there. Administrators should therefore take care to keep KVM, Qemu, and the Virtio drivers up to date at all times. This is especially true for Windows guests, which – in contrast to Linux distributions – cannot update Virtio drivers automatically.

Walled In

To guard against a breakout, admins need to build virtual safety perimeters around their virtual machines. It helps that the virtual machines appear to be normal processes from the host system's point of view. These processes in turn can be regulated by SELinux, AppArmor, or some other mandatory access control system. The sVirt component in libvirt 0.6.1 and newer actually does some of the work for SELinux and AppArmor [2]. For example, in SELinux, sVirt attaches labels to virtual machines, which can then be isolated selectively.

You also can lock virtual machines in cgroups and thus control their resource consumption and access. Incidentally, this practice protects you against a crashed machine running wild or using too much CPU time, or a DoS attack blocking network access to other virtual machines. Normally, libvirt automatically produces a corresponding cgroup hierarchy [3]; the access to resources can be controlled in a targeted way using the virsh tool.

The following command, for example, restricts the computational time for a virtual machine called webserver to 100 shares (this is often equivalent to about 10 percent of the computing power):

virsh schedinfo --set cpu_shares=100 webserver

However, virsh and KVM do not let you regulate all of your resources. The virsh man page lists the options below schedinfo.

HR Department

If you boot the virtual machine directly using sudo qemu-kvm -m 512  …, you are giving the VM – and any program that breaks out of jail – system privileges (Figure 1). Fortunately, root privileges are not necessary: Most distributions give normal users from the kvm group access to /dev/kvm. It is therefore perfectly okay to add your own user account to the kvm group and then use this account to run qemu-kvm … in the future. If you give the virtual machine access to the network via a TAP interface, you must also create the interface for the user and group under which the virtual machine will be running. Usually, this is qemu in the qemu group.

Alternatively, you can use the libvirt tools to manage your virtual machines. They automatically start a virtual machine as the non-privileged qemu user. A number of pitfalls exist here, however: On some distributions, such as Ubuntu, all users in the libvirtd group are allowed to run virsh. However, if you have access to virsh, you can use it not only to manage all of the virtual machines, you can also use virsh nodeinfo to query the host's hardware specs (Figure 2). Thus, virsh and the like should only be used by selected administrators. If you integrate virsh into your scripts, you should also check the rights these scripts have.

Furthermore, the libvirtd daemon running on the host system likes to have root privileges. If attackers were able to contact libvirtd, they could hijack the virtual machine and, in the worst case, the whole system. Consequently, you need to limit access to libvirtd. In particular, you must make sure that the /etc/libvirt/libvirtd.conf configuration file does not have a listen_tcp = 1 line. In this case, libvritd would accept TCP connections from anyone – assuming the firewall on the host system does not block the appropriate port (by default, 16509).

Personals

Administrators also should use encryption. libvirtd and its tools optionally can tunnel their communication via SSH, use authentication via SASL/Kerberos, or open an encrypted SSL/TLS TCP connection. In most cases, an SSH connection is probably the best bet, which means running an SSH server on the host system. Because root privileges are needed, the connection is opened with, for example:

virt-manager -c qemu+ssh://root@example.com/system

For security reasons, the SSH server should require a user login and reject root. As a compromise, you could go for certificate- or public/private key-based authentication. In case of SSL/TLS encryption, you should also change the port number in the /etc/libvirt/libvirtd.conf configuration file so that at least direct attacks are difficult.

Finally, admins should definitely consider the contents of the LIBVIRT_DEFAULT_URI environmental variable. It specifies the URI that virsh and virt-manager use by default to establish a connection. If the variable contains, for example, the value qemu://host/system, virsh opens an unencrypted connection if no parameters prevent this.

Smugglers

It is dangerous simply to pass host devices through to the virtual machine. If you mount a filesystem on the virtual machine that other computers on the LAN can also see, the infected guest could use this pass-through to distribute malicious programs, change configuration files, or destroy existing documents. The valuable project files on the NAS would then be goners. Administrators should therefore carefully consider what filesystems depend on which virtual machine. Ideally, the virtual machine uses only its own image files.

Selected USB devices – and even a complete USB port – can pass through to the virtual machine (Figure 3). In the following example, all devices end up at port 1.1 on bus 2 of the virtual machine:

qemu-kvm -usb -device usb-host,hostbus=2,hostport=1.1 ...

However, the risk of someone plugging an infected USB flash drive into the host computer and then smuggling malicious software onto the virtual machine must also be considered. The same concerns apply to DVD drives  – which can even boot a Live system under certain circumstances.

Hidden Control Center

If the virtual machine is allowed to write its screen output to a window, pressing Ctrl+Alt+2 changes to the monitor (Figure 4). Once there, you can restart the system or change to the virtual DVD, among other things. If the virtual machine is running on a server with no graphics output, the monitor can also be redirected. The following command serves up the console available via a TCP connection on port 4444:

qemu-kvm -monitor tcp::4444,server,nowait ...

All communication is unencrypted here (Figure 5). Consequently, anyone could use Telnet to dock with the virtual machine and make desired changes. Admins should therefore at least restrict access to one computer, for example, the computer with the IP address of 192.168.100.21:

qemu-kvm -monitor tcp:192.168.100.21:4444,server,nowait ...

Ideally, however, you would want to prevent this kind of access and instead use the libvirt tools – in particular, virsh. The latter lets you call monitor commands directly via:

qemu-monitor-command

By default, each newly created virtual machine is given a serial port, which administrators can use to connect via virsh console.

You can even redirect the console, like the monitor, for example, to the physical serial port on the host or via TCP [4]. The (boot) messages seen on this console depend on the guest. Depending on the configuration, the administrator could even use this method to log in. Although this approach is useful for remote maintenance of a screenless virtual machine, again all communication is unencrypted. You should not redirect the output from the host system to the serial interface, but use a secure method such as SSH instead.

Figurative

By default, Qemu writes the screen output from the virtual machine to a window on the host desktop. Alternatively, you can enable a built-in Qemu VNC server. The desktop on the virtual machine then displays a VNC client. By default, this VNC client must run on the same computer as the virtual machine; Qemu only allows connections from the local host IP address 127.0.0.1. However, admins can change the default using the -vnc parameter:

qemu-kvm -vnc :1 ...

With this setting, which is mentioned by many guides, anyone could connect to the VNC server. If the host computer has an IP address of 192.168.100.21, the following command displays the screen from the first virtual machine on the administrator's desktop (Figure 6):

vncviewer 192.168.100.21:5901

If you forgot to log out of the guest system, the attacker is now gifted with the system. For the same reason, you should not simply share the VNC server by using virt-install with the -graphics parameter, even if this facilitates remote installation. The TCP ports are known, and KVM uses numbers from 5900 upward by default (the port is calculated as the number 5900 plus the display number).

Very Important PC

Administrators should at least restrict access to a trusted computer. The following command tells the VNC server to accept connections from the IP address 192.168.100.30 only:

qemu-kvm -vnc 192.168.100.30:1 ...

Ideally, you also want to filter the VNC connections through the firewall. If you work with the libvirt tools, check the settings in the /etc/libvirt/qemu.conf configuration file. You can also specify a password there and at the command line. Unfortunately, this password is stored in the clear in the configuration file and can even pop up in the logfiles under certain circumstances. You should therefore always encrypt VNC connections with SSL/TSL or tunnel them with SSH. If the virtual machine is running as a headless server, the video output also can be completely disabled for safety reasons (e.g., via the -nographic parameter).

Incidentally, the warnings relating to VNC also apply to graphics output via SPICE. Again, the built-in KVM SPICE server listens on the first available port above 5900. Unlike VNC, however, SPICE supports encryption in the connection [5].

Data Tombs

By default, disk images are not encrypted. If they reside on a freely accessible hard drive or (NFS) share, an attacker could create a copy and then analyze the installed system at leisure – this is particularly true for the RAW format. Administrators should thus ensure that no one gains access to the images. It also makes sense to encrypt the images. To do this, you can store the images on an filesystem encrypted by the host system and let the guest system encrypt its own (virtual) disk.

Furthermore, KVM or Qemu also encrypt an entire image with the AES algorithm. Encryption itself is completely transparent for the guest. However, the image must be in the qcow2 format, and Qemu currently only uses a 128-bit key. An encrypted image is created, for example, using qemu-img (Figure 7).

Caught in the Net

Once a virtual machine accesses the intranet or Internet, it can fall victim to DoS attacks, port scans, and intrusion attempts, just like any physical machine. Therefore, the same security measures should be taken: The web applications or services running on the virtual machine should be always up to date.

Even if a break-in succeeds, the malware must not spread via the network to the host computer, to other physical computers on the LAN, or to other virtual machines. The latter, in particular, is easy to achieve if all virtual machines are connected to the same (virtual) network (Figure 8). Firewalls must thus check the flow of data to, and especially between, the virtual machines. This practice is supported by the libvirt tools – in particular the nwfilter network filter driver [6].

In the normal NAT operating mode (user mode, default network), only the guest system on the virtual machine accesses the network; remote attackers only see the host computer, which they can attempt to compromise. Conversely, however, the guest by default reaches its host on 10.0.2.2, which is also the address on which the built-in DHCP server runs. Although this setup is useful for exchanging data, malicious programs on the virtual machine can easily attack the host. The firewall on the host usually does not intervene here because the queries are running on the host system itself. It is therefore important that no vulnerable services are running on the host system. Ideally, the host only runs the virtual machines, and nothing else.

Qemu also lets you connect virtual machines via TCP and UDP sockets. However, these connections are not encrypted, and anyone who has access to the sockets can send data to the connected virtual machines. If you do use sockets, you should therefore at least choose a setup where the connections use the access restrictions offered by Qemu and only allow connections from specific IP addresses (parameters -net socket,connect=hostname,port).

Conductive

If you have set up port forwarding on the virtual machine, it should be limited to a single network interface (Figure 9). This approach makes it easier to distinguish between and identify the connections to the virtual machine and the host. Maintenance on the host computer then exclusively uses the eth0 interface, whereas access to the virtual machines only uses eth1.

The libvirt tools and the MacVTap driver can help here by directly connecting virtual machines with a network interface on the host system. Private and bridge modes of operation are of interest here, where the virtual machines cannot access the host system. In Private mode, even the virtual machines can no longer communicate with one another [7]. However, these actions do not mean that you can do without a firewall.

Qemu offers a built-in TFTP server that lets you quickly and easily share files between the host and guest system. However, communication is unencrypted, and there is no user authentication. Administrators will thus want to disable the built-in TFTP server and, instead, set up a properly secured FTP server or, preferably, SSH access on the guest.

Fide Sed Cui Vide

Administrators should always keep the guest system up to date and install only known software. Caution is also advised with prebuilt virtual machines (appliances): They could include malware from the outset. Anyone planning to migrate an existing system should examine it up front for malicious software. Otherwise, you could also infect the new host system. If you boot to the virtual machine over the network via PXE, the server contacted for this purpose must be trustworthy. This is especially true if you automated the process of creating new virtual machines by scripting.

In a live migration, only the host systems involved should be allowed access to a shared filesystem or an NFS share. After the migration, you would then subsequently restrict access to the migrated system. Live migration should take place over encrypted connections, such as SSH.

A virtual machine that goes haywire can hog computing time, whereas a DoS attack on a virtual machine blocks the physical network connection. Both cases also paralyze all other virtual machines at the same time. It is consequently advisable to set up a monitoring system and to organize the virtual machines in cgroups, as already mentioned.

Virtual machines can be automatically started using the start scripts when booting the host system or using libvirtd. However, you should think twice about this approach: If you have a malicious program on the virtual machine, and the program manages to break out, it can access the system at a very early stage. In extreme cases, you have no opportunity to rein in and isolate the rogue virtual machine.

Conclusions

A virtual machine is not a practical jail. On the contrary, it even increases the number of attack vectors. Administrators should think of virtual machines as additional, physical computers on a network and thus initiate the same security measures or include the machines in their (existing) security concept. Guests also need regular updates and the same care as the host system. Finally, most defaults in KVM, Qemu, and libvirt are designed for security, and administrators should therefore change them only for a good reason.