News from BSD

For 20 years, FreeBSD [1] has been an efficient, secure, and stable open source operating system. Now version 10 has been released with some innovations.

FreeBSD is part of the Unix family, although it is not called Unix for legal reasons. FreeBSD is one of several systems descending from the Berkeley Software Distribution (BSD) Unix system developed at the University of California Berkeley. Although the BSD license is a Free Software license, unlike the GPL, it does not provide a "copyleft" protection that forces developers to make their changes available to the community. For that reason, companies can mix BSD with proprietary code or even take the code out of open source, which is one reason why BSD formed the basis for Mac OS and other proprietary operating systems. Authorities and companies in the fields of technology and IT use FreeBSD as an internal developer and server platform.

FreeBSD upholds the traditional Linux virtues of security and reliability. As an example of the trust that corporations place in FreeBSD, Sony based its PlayStation 4 on FreeBSD 9. Likewise, Juniper relies on FreeBSD as a robust operating system for network routers, and Apple uses parts of FreeBSD for its Mac OS X operating system.

FreeBSD claims to be the most mature Unix-like operating system for x86 servers. In the 1990s, FreeBSD established itself as an operating system for all sizes of web and FTP servers with its speed, reliability, and often years of uptime. This tradition became a commitment and was rarely interrupted.

With multiprocessor and multicore systems asserting themselves in the server segment of the market, the FreeBSD development team had to respond. The FreeBSD 5 kernel published in 2004 was completely remodeled in a time-consuming process for symmetric multiprocessing. This radical redesign led to problems that were largely not remedied until version 7.

The next, not quite as radical, upheaval took place with the introduction of virtualization and ZFS on FreeBSD 8 in 2009. The filesystem developed by Sun Microsystems with integrated volume management is designed for very large volumes of data and high data security. The port to FreeBSD required some adjustments to the kernel, but it also contributed to further stability and security. In many ways, the new FreeBSD 10 is quite revolutionary.

GCC Has Had Its Day

FreeBSD 10 uses a new compiler to build the kernel sources and the associated tools: LLVM/Clang [2] [3]. The GCC project's move to the new GPL version 3 license prompted the developers to remove GCC from the FreeBSD base system. (See the box titled "Stallman on Clang.")

LLVM, originally an acronym for Low-Level Virtual Machine, not only contains a compiler but also a number of tools and libraries. Traditional compiler systems perform a number of optimizations by compiling the source code. Then, they link the compiled modules together, ignoring some optimization potentials arising in this area. The reason is that the linker only sees the individual modules and not the entire program.

LLVM resolves this problem by first creating virtual bytecode, as found in RISC instruction sets (Figure 1). This bytecode can be optimized again during the link process.

As the name LLVM suggests, part of the framework's architecture is based on a virtual machine that virtualizes a processor: either a main processor (CPU) or a graphics processor (GPU). At run time, the virtual machine converts the internally generate intermediate language (LLVM Assembly Language) to that of the current system. Its efficiency also allows for just-in-time compilation. Additionally, LLVM allows a flexible back-end system and supports a variety of processor architectures.

As an LLVM suite front end, Clang parses the C or C++ code and converts it to LLVM Intermediate Representation (LLVM IR). The LLVM back end then generates efficient machine code. At this point, the virtual machine is not used, because LLVM (i.e., x86, PowerPC, IA64) takes its place as a compiler back end for the current architecture. In addition to C and C++, front ends exist for Fortran, Haskell, and Ruby, and others are in development.

Because of the structure of the LLVM/Clang system, compiling FreeBSD with make build kernel and make buildworld now takes about 20 percent longer. However, this extra time is well spent; Clang checks the kernel's and the tools' C code in more detail and has thus revealed some errors that GCC had not found.

This structure also has an effect on the ported software (ports) in the FreeBSD ports tree. Most programs can be compiled with no or just a few code corrections using LLVM/Clang. However, other ports require the old GCC compiler suite. To use this, you need to install GCC parallel to LLVM/Clang and use it exclusively for the ports. Unfortunately, until all ports will build in Clang, the port makefiles do need some TLC, as follows:

CC=gcc47
CXX=g++47
CPP=cpp47

Because the FreeBSD back end is only available for the amd64, arm, armv6, and i386 platforms right now, use of the compiler suite is limited to these architectures. The other target platforms continue to rely on a slightly older version of the GCC suite.

Kernel World

The replacement of the compiler suite is not the only change in the new version of FreeBSD; new features also have been added. For example, FreeBSD 10 has a tickless kernel [4]. A normal kernel, as used by FreeBSD versions up to and including 9.2, listened for events. Events trigger, for example, read and write operations on the disk. To handle this kind of interruption, the system stores the CPU state to continue working at the same place after processing the event.

This procedure also takes place even if there are no read or write operations in idle mode. However, backing up the CPU state costs computing power, and thus time, energy, and  – on laptops – battery power, even though this is unnecessary in idle mode. The new kernel fixes this vulnerability by not listening for events at fixed times but only when they occur.

The advantage of this method is important not only for laptop power savings but also means an increase in overall performance. However, on servers with a high load, the energy savings are not substantial, because many events and read and write operations will typically occur.

Graphics

On FreeBSD, traditionally the X Window System, Xorg, takes care of switching to graphics mode. Graphical user interfaces rely on it, whereas the graphics card usually works in text mode until they are launched. Mode setting occurs when an X server is started in user space using the startx command, for example, or by starting a login manager. Unfortunately, this mode change leads to ugly screen flicker as well as security issues.

In contrast, the newly introduced Kernel Mode Setting (KMS) switches the operating system kernel to graphics mode, thus eliminating the previous drawbacks. FreeBSD uses Linux code for the transition, following the move by the Xorg project to KMS; this means that Xorg graphics drivers only work with KMS.

After the introduction of KMS, FreeBSD now uses the xf86-video-ati driver for AMD graphics processors and thus supports chipsets by this manufacturer. The FreeBSD wiki page [5] has published a table listing AMD graphics cards that work with the new KMS driver.

On the downside, the suspend and resume feature, which is particularly important for laptops, does not currently work with KMS. If you still want to use KMS, you need to add the following to the /etc/make.conf configuration file,

WITH_NEW_XORG=YES

and then recompile Xorg.

Virtualization

The new version of the FreeBSD operating system offers its own virtualization solution: Bhyve [6], a type 2 virtualization software. That is, it is based on a full operating system and uses its device drivers. In contrast, a type 1 hypervisor runs directly on the hardware.

Bhyve (Figure 2) is a hardware virtual machine (HVM). Thus far, the hypervisor only supports Intel's VT-x technology; support for Secure Virtual Machine (SVM) by AMD is still work in progress. Bhyve uses Intel's Extended Page Tables to manage the memory addresses of virtual machines.

The FreeBSD hypervisor emulates I/O APIC (Advanced Programmable Interrupt Controller) and thus supports APIC for guest systems as well as an AHCI emulation, which currently only partially works. The developers are currently working primarily on non-blocking read and write access and support for suspend and resume.

Bhyve consists of a kernel module, vmm.ko, the libvmmapi.so library, and the applications bhyve(8), bhyveload(8), and byhvectrl(8). The components can manage with as little as 250KB of memory.

The bhyveload(8) tool loads a FreeBSD guest directly on the virtual machine. This is done quickly with a simple command that starts FreeBSD from an ISO image:

bhyveload -m 1024 -d ./freebsd.iso freebsd-vm

The performance of FreeBSD as a guest in a virtualized environment is improved by the virtio driver package, which has made its way from the ports to the base system in the new version of FreeBSD. The kernel module contained in the virtio package, virtio-kmod, provides virtualized FreeBSD direct access to the host resources, thanks to paravirtualized APIs. Without virtio-kmod, the host would need to emulate a network card, hard disk controllers, and other hardware components for the guest operating system. Emulating this functionality and implementing the back end consumes time and resources and, thus, slows down the guest.

The virtio package also supports memory ballooning. This technique provides the memory released by the guest system to other guests.

The entries shown in Listing 1 from the /boot/loader.conf file enable virtio. This changes the names of the virtual disks and network cards to, say, /dev/vtbd0 and /dev/vtnet0, which means adjusting the /etc/fstab file to match. Figure 3 shows the typical kernel boot messages.

#Init VirtIO package
virtio_load="YES"
virtio_pci_load="YES"
# Block devices
virtio_blk_load="YES"
# network hardware
if_vtnet_load="YES"
# Memory Ballooning
virtio_balloon_load="YES"
# SCSI support
virtio_scsi_load="YES"

VirtualBox also works with virtio. To do this, you need to enable the VirtualBox Manager in the advanced network settings as a Paravirtualized Network (virtio-net)--type adapter (Figure 4).

FreeBSD 10 also introduces some innovations in storage medium management. In addition to performance optimizations, useful tools from ports have made their way into the base system. This includes the growfs(8) command. It lets users change the size of a UFS2 filesystem – the default on FreeBSD. This tool is especially useful if you need to transfer a backup of a smaller slice to a bigger one. growfs then offers the option of growing the filesystem up to the slice boundary without unmounting. Of course, a backup is recommended before the change.

Even the iSCSI system (Internet Small Computer System Interface) has made its way into the base system in the form of a kernel module. The system consists of an iSCSI target and initiator.

iSCSI transports SCSI data over IP networks, encapsulating the data in TCP/IP packets and using ports 860 and 3260. iSCSI enables access to a storage area network via a virtual point-to-point connection without setting up separate storage devices. Also, existing network switches can be used for iSCSI; iSCSI does not require special hardware for node connections.

Disk access is blockwise, which makes it suitable for databases. Access via iSCSI is also transparent: For example, iSCSI devices show up on FreeBSD as normal SCSI block devices (/dev/da*) and can be used as local SCSI disks.

FUSE (Filesystem in Userspace) has also been promoted from the ports to the base system. FUSE is a kernel module that shifts filesystem drivers from kernel mode to user mode. This approach allows non-privileged users to mount their own filesystems.

Because FUSE uses user mode, just like any other application program, a variety of drivers have emerged. Some of these filesystem drivers create completely different data structures in the form of filesystems instead of hard disks and other storage media.

FUSE is used on FreeBSD to mount the following filesystems:

• Windows NTFS-3G: This is a FUSE implementation of the Windows NTFS filesystem, which is used by Windows XP and Windows Server 2003, among others. The FUSE driver supports read and write operations, as well as almost all POSIX filesystem functions. Only file permission and ownership changes are not provided.
• Linux-ext4: FreeBSD supports the popular Linux ext4 filesystem with FUSE; however, the module only offers access.
• FUSEPod: This extension can be used to mount an Apple iPod or iPhone and provide access to all the files on the device.

ZFS

ZFS, as a filesystem with integrated volume management, has been part of FreeBSD since version 7; it has been suitable for production use for some time [7]. FreeBSD 10 introduces the ZFS NOP (No Write operation) function, which substantially increases the speed of the filesystem.

The performance gain is a result of saved writes. Without NOP, ZFS creates a checksum for each data block during the write operation, even if the content of the data block remains the same. With NOP, however, the system compares the checksum of the block to be written with the existing block on disk. If the checksums are identical, the system does not write the data block. Cryptographic methods are used here to create checksums, thereby enhancing data integrity.

Furthermore, the ZFS developers have added data compression to the Level 2 cache. The Level 2 cache (L2ARC) provides fast read and write access. For optimal coverage of a ZPool – a kind of virtual grouping of several block devices on ZFS – the size of the L2ARC grows in proportion to the ZPool size. On very large systems, however, this quickly results in storage capacity bottlenecks. For this reason, FreeBSD 10 compresses the data in the cache and thus reduces its size.

ZFS already compressed the stored data in previous versions of FreeBSD on request, previously using the Lempel Ziv Jeff Bonwick method (lzjb). FreeBSD 10 uses the faster LZ4 algorithm with significant performance gains: For easily compressible data, the compression speed is 50 percent higher, and during decompression, an 80 percent boost is achieved compared with lzjb. When processing non-compressible data, the newly implemented algorithm is still about three times faster. On systems with slower CPUs, especially, the speed gains are clearly noticeable.

The following command enables compression in line with the new standard for a ZPool named users:

# zfs set compression=lz4 pool/users

Fast solid state disks (SSDs) are especially well suited to caching. To fully benefit from these drives, ZFS implements support for the ATA trim command. ZFS uses trim to tell a solid state disk that deleted or otherwise vacant blocks are no longer used. Without trim, ZFS only uses the administrative structures to show that the corresponding regions are available again, but the SSD controller does not receive this information.

The ATA trim command tells the drive to mark the affected blocks as invalid and that the data is obsolete when deleting files. As such, the data is not written, which reduces access to the SSD and its wear effect. The labeled blocks are then finally released during the next delete action.

ZFS support for trim includes the following configuration options (sysctl Management Information Base, or MIB):

• vfs.zfs.trim.enabled: If this sysctl MIB is set to zero, trim support is disabled. It is enabled by default.
• vfs.zfs.trim.max_interval: This sysctl MIB defines how many seconds must elapse between two trim calls.
• vfs.zfs.trim.timeout: This parameter sets a delay value in seconds for the first execution of trim.
• vfs.zfs.trim.txg_delay: Specifies the maximum period the data of a Transaction Group (TXG) can remain in memory before it is written. This is an important value for ZFS because the filesystem delays write operations until sufficient data is available. Initially, this value was set to 30 seconds; however, this caused problems on slow systems with only one hard disk. Writing of data and processing of the trim command coincided frequently, thus blocking the system.

Network

The FreeBSD developers have also done justice to the rise of multicore and multiprocessor systems by adapting the kernel and drivers. To exploit the performance potential of the future, they have revised the pf packet filter.

This tool, which was originally ported from OpenBSD, is designed for single CPU systems and briefly stopped a data stream at the beginning of the filtering process. The filter rules were then applied and, at the end of the process, the data stream was released. The SMP-friendly version changes this sequential process. Now, multiple parallel threads process the data stream, which increases the processing speed significantly and reduces system load.

FreeBSD 10 not only eliminates old problems with multiprocessor operation in the field of wireless LAN but also adds new hardware components to the driver for Atheros wireless cards. ath(4) now supports all Atheros PCI/PCIe network adapters up to and including the AR9287 chipset. However, support for network cards with AR5513 MIMO 802.11abg chips and AR5523/AR5212 chips, which are used on USB WiFi sticks and plugin cards, are still missing, as are the AR7010 and AR9271 series, which are used for USB WiFi sticks.

FreeBSD 10 also includes a number of enhancements for the IEEE 802.11n standard – the latest version of the WiFi standard. These enhancements also serve as the basis for meshed WLANs, which is a network of communicating wireless nodes as per the IEEE802.11s standard, also known as an ad hoc network.

Ad hoc networks connect mobile devices (nodes), such as mobile phones and notebooks, without relying on a fixed infrastructure, such as wireless access points. The data are passed from node to node until they reach their destination. Thus, the data load in such networks is distributed better than in networks with a central node. Special routing methods are used to implement this principle; they allow the network to adapt continuously as nodes move, join, or leave (Figure 5).

Nothing prevents practical use of the implementation of the IEEE 802.11s in FreeBSD 10; however, Linux compatibility has not yet been added.

In terms of support for the IEEE 802.11s standard, the following picture emerges: Drivers for wireless cards with the Atheros chipset (ath(4)), Ralink chipset (ral(4)), and chips by Marvell (mwl(4)) support WLAN meshes.

Things are not as rosy for WiFi adapters with IntelPRO wireless chipsets, because much of their functionality lies in closed-source firmware, which also prevents improvements looking forward. This situation also applies to drivers with Intel PRO wireless chipsets ipw(4), iwi(4), iwn(4), and wi(4). The developers are working hard on all other WiFi drivers to make them fit for wireless meshes.

The following example shows how just a few commands can be used to build a wireless mesh. The commands run on each node and set up a mesh that runs on channel 36 and goes by the name admin-mag-mesh:

# ifconfig wlan0 create wlandev ath0 wlanmode mesh channel 36 meshid admin-mag-mesh
# ifconfig wlan0 <IP address/netmask>

The following command prints a list of all nodes in the mesh:

# ifconfig wlan0 list sta
ADDR   CHAN ... STATE RATE ...
<MAC0> 36   ... IDLE  0M   ...
<MAC1> 36   ... ESTAB 6M   ... WME MESHCONF
<MAC2> 36   ... ESTAB 6M   ... WMEMESHCONF
<MAC3> 36   ... ESTAB 6M   ... WMEMESHCONF

The <MAC0> line shows your own mesh node; the other lines list the other nodes.

Package Management

FreeBSD 10 has also moved to a new, previously optional, package manager [7]. The pkg tool now replaces the tools used previously: pkg_add, pkg_info, and pkg_delete.

The new package manager especially improves the resolution of dependencies that arise when an application needs other programs, libraries, or scripts. The pkg command transfers dependency management to a SQLite database and, thus, improves speed and stability significantly.

In the future, pkg will also process signed packages and thereby add a safety feature. However, pkg does not to manage the ports; this is left to the portmaster and portupgrade tools, which reside in the /usr/ports/ports-mgmt directory. To keep the management of ports and packages consistent, add the following line to your /etc/make.conf file, if it does not already exist:

WITH_PKGNG=yes

Operations are simple and command-line oriented, like before. Figure 6 shows the dependencies using the portupgrade tool.

The new package management software does not just install and uninstall applications, it also takes care of upgrades. Typing

# pkg version

retrieves a list of all packages, whereas the following command updates all outdated software, including all dependencies:

# pkg upgrade

The pkg tool also takes care of security. The command

# pkg audit -F

displays all packages with security vulnerabilities.

Installation

Even in the new version, FreeBSD installation is still via the text interface of the bsdinstaller tool. Text-based installation prevents potential problems with Xorg detecting the graphics hardware. The new version of bsdinstaller lets you install the system in a ZFS pool (Figure 7), which previously involved some effort [8].

The FreeBSD default installation generally runs smoothly, both on laptops and on servers. However, the weak points still remain: support for ACPI and graphics hardware. On a laptop, you will definitely want to remove the vesa module from the kernel; otherwise, the notebook will crash the next time it resumes.

Furthermore, the experimental installation of FreeBSD in a ZFS pool caused some trouble. A virtual machine created with VirtualBox regularly crashed or became unresponsive with this configuration in our tests.

Generally, the new version, FreeBSD 10, is still a stable and secure operating system, and it adds some useful new features. Apart from resolving the problems mentioned above, I would like to see wider hardware support, for example, in the WiFi department. That said, FreeBSD 10 is recommended because of its great quality for supported hardware.