Nuts and Bolts OpenSwitch Lead image: Lead Image © higyou, 123RF.com

Data center networking with OpenSwitch

Switched On

When it comes to network devices on Linux, legacy firmware still seems to have a competitive edge. Lately, however, some highly promising approaches have shown up, including the OpenSwitch project. By Udo Seidel

OpenSwitch is an open source, community-based platform for building customized data center networks. OpenSwitch [1] entered the stage in Fall 2015 [2] when Hewlett-Packard (HP) announced the new network operating system in cooperation with Accton Technology Corporation, Arista, Broadcom, Intel, and VMware. The series of Altoline switches [3] by HP Enterprise (HPE), as the group is now called, were supported from the very beginning. Less than a year later, more advocates were found, such as Mellanox, Cavium Extreme Networks, and even LinkedIn and Microsoft. Since the beginning of June 2016, OpenSwitch has been an official project under the auspices of the Linux Foundation [4] [5].

Difficult Beginnings

OpenSwitch can be regarded from two perspectives: the user's or the developer's. To explore OpenSwitch as a user, you need compatible hardware, such as a suitable switch. For the developer, the entry point is the OpenSwitch Gerrit page [6]. For development work, the prospective programmer needs a GitHub account and the Git [7] and Gerrit [8] tools. The project page [9] explains the individual steps. If you have dealt with Yocto [10] in the past, you should be able to find your way around pretty quickly. Yocto is a Linux distribution for small systems and is the foundation for OpenSwitch (Figure 1).

Figure 1: OpenSwitch is based on Yocto. The current version comes with a 3.18 kernel.

In this article, I approach OpenSwitch from the end-user perspective, because getting started from the development side would be difficult without user experience. However, not everyone has access to the required compatible hardware; fortunately, the project provides an alternative for virtual servers. The image is available in the Open Virtualization Appliance (OVA) format 11-13. The tarball contains a prebuilt image in virtual machine disk (VMDK) format [14], which describes the associated virtual machine (Listing 1). If the existing infrastructure does not allow automatic integration of OVA images, you will need to take the manual approach.

Listing 1: Installation for Virtual Servers

$ tar tf openswitch-appliance-image-appliance-ops-0.4.0-master+2016080600.ova
OpenSwitch.ovf
OpenSwitch.vmdk
$ tar xf openswitch-appliance-image-appliance-ops-0.4.0-master+2016080600.ova
$ ls OpenSwitch.*
OpenSwitch.ovf OpenSwitch.vmdk
$ qemu-img convert -f vmdk -O raw OpenSwitch.vmdk OpenSwitch.img
$ ls OpenSwitch.*
OpenSwitch.img OpenSwitch.ovf OpenSwitch.vmdk

KVM was used as the hypervisor for this article. I took the specification of the virtual machine from the OVF file: two CPUs, 768MB RAM, and eight network interfaces. Converting the hard disk image is a matter of taste, because KVM can also deal with VMDK files. By the way, the first network interface is used for managing OpenSwitch or the underlying hardware itself; therefore, this interface must have an IP address that is reachable over the network. The assignment may be static or via DHCP.

Whether manual or automated, the installation of the virtual instance of OpenSwitch is quite painless and unspectacular. A few seconds after starting, the system is available (Figure 2).

Figure 2: The virtual instance of OpenSwitch is available for use after a few moments.

Three users are predefined: root, admin, and netop. Only admin and netop are protected by a password, which is the same as the login name. Access to the all-powerful root user is not protected by additional authentication. For interactive sessions, users can use SSH over the default ports as HTTP or HTTPS. A root login is not allowed for access through the browser. Of course, you should change the passwords for protection against unauthorized access.

To install OpenSwitch on physical hardware, the project relies on the support of the Open Network Installation Environment (ONIE) [15]. ONIE provides methods and concepts for installing custom firmware on network devices. The term "firmware" might sound somewhat misleading here, because it refers in most cases to Linux-style systems, which typically refer to themselves as a network operating system (NOS). Interestingly, ONIE is also based on Linux.

The NOS installation is via the usual Linux bootloader mechanisms. The box "An Installer for Everything" provides further information.

An Installer for Everything

The goal of ONIE is to provide a uniform and simple way to equip network devices with the firmware, or NOS, of their choice. In the background, a modern Linux kernel and BusyBox [16] go about their work. The project even goes so far as to state that it would like to support parallel installations in large environments to the extent possible: Puppet [17], Chef [18], or Ansible [19] provide capable and reliable tools. The switch behaves much like a Linux server; adding administration tools is quite simple.

ONIE is an initiative of the Open Compute Project [20] [21]. Its founding members were, among others: Accton, Agema, Big Switch Networks, Broadcom, Cumulus Networks, and Dell. An overlap with the OpenSwitch proponents of the early days is not accidental. The driving force was Cumulus Networks [22], which is also active in the NOS market. ONIE seeks to serve individual end users, managers of large network environments, and switch manufacturers and vendors equally well.

The workflow of the NOS installation is straightforward (Figure 3). The low-level bootloader of the switch initializes the hardware and finally ONIE, (i.e., a Linux kernel with BusyBox on top). ONIE configures the management interface, loads the NOS installer, and executes it. ONIE can use various methods and configuration options [23] for finding and loading the installation program.

The NOS installer location information can be configured directly in ONIE. Alternatively, the data can be provided via DHCP, multicast DNS, and similar zero-conf methods, including the PXE boot mechanism known from server installations. Both TFTP and HTTP are suitable for transporting the binary data, and it is possible to save the installation program on a locally attached USB disk, which is where ONIE looks first.

Figure 3: The installation of a NOS via ONIE (left) and a normal boot process (right).

Digging Deeper into the Rabbit Hole

As mentioned, the first step after you complete the install, should be to set up new passwords for the root, admin, and netop users. Creating your own users with the useradd command is also a good idea. The normal Linux commands ps, top, and ifconfig give you an overview, and systemd handles system startup.

If you look at the list of standard processes, you will see that many services with names that start with ops are represented. Each is responsible for a specific OpenSwitch function. For example, VLAN management is handled by ops-vland, whereas ops-ospfd manages events relating to the Open Shortest Path First (OSPF) routing protocol. The project page has detailed documentation for each function [24].

Management requires OpenSwitch admins to start a special command interpreter (Listing 2), vtysh [25], known from the Quagga routing software. Users in privileged mode then have the ability to copy configurations, for example. If you want to change the configuration, you need to switch to configuration mode, which is done by entering configure or configure terminal.

Listing 2: Client Configuration for NTP

root@switch:~# vtysh
switch#
switch# show ntp status
NTP is enabled
NTP authentication is disabled
Uptime: 137 second(s)
switch#
switch# show ntp associations
----------------------------------------------------------------------
 ID NAME REMOTE VER KEYID REF-ID ST T LAST POLL REACH DELAY OFFSET JITTER
----------------------------------------------------------------------
----------------------------------------------------------------------
switch(config)# ntp server ntp.linux-magazine.com
switch(config)#
switch(config)# exit
switch#
switch# show ntp associations
----------------------------------------------------------------------
 ID NAME REMOTE VER KEYID REF-ID ST T LAST POLL REACH DELAY OFFSET JITTER
----------------------------------------------------------------------
 1 ntp.linux-magazine.com - 3 - - - - - - - - - -
----------------------------------------------------------------------

Changes made in this way do not survive a reboot. For the experimental stage, this is quite useful, because a restart restores the system to a plain vanilla state. In normal operation, admins will usually want any changes to persist. With a simple command, you can send the run-time configuration to non-volatile memory,

copy running-config startup-config

which is done directly in the vtysh.

Freeing the Way for Automation

The configuration of a single switch is only one piece of the puzzle, compared with the actual ambitions of the OpenSwitch project. The target is automation in larger configurations. The project prefers Ansible [19] and provides the documentation for getting started [26]. One argument in its favor is that it requires no installation on the target system (i.e., the OpenSwitch device).

The example given earlier for the NTP configuration of a single switch required logging on, changing to the routing shell, switching to configuration mode, and finally issuing commands. No one wants to do this manually on a large number of devices. The critical reader might argue that the NTP configuration is static in most cases and only needed during initial setup of a new unit, but the described process – login, vtysh, configure, <a command> – can be compared with configuring a new VLAN.

This task is part of the admin's everyday life, and you will perform it in most cases on a series of switches. However, manual implementation doesn't scale. With OpenSwitch and Ansible, the new VLAN configuration can be rolled out to many systems in seconds; it is simple, repeatable, and supported on the hardware regardless of the manufacturer. The Ansible module ops_template also supports the complete reconfiguration of many switches in a single step.

The core area of Ansible [27] included four modules when this issue went to press. Table 1 lists them with a short description. Of course, you can use other Ansible modules – after all, OpenSwitch is based on Linux. For the actual switch commands, you would need to configure the change to the special shell (vtysh) via constructs such as:

echo 'show version' | vtysh

Tabelle 1: OpenSwitch-Specific Ansible Modules

Module	Description
`ops_command`	Runs an arbitrary command on OpenSwitch
`ops_config`	Manages the OpenSwitch configuration at the command line
`ops_facts`	Collects device information for the switch
`ops_template`	Installs a run-time configuration on OpenSwitch

HTTPS almost has a double function: It is also the entry point for a REST interface [28]. Given the appropriate infrastructure, the OpenSwitch environment can be managed using GET, POST, DELETE, and PUT.

What Remains at the End

OpenSwitch is currently available in version 0.4. Much has happened since its release just over a year ago – and not just in the technical details. In particular, OpenSwitch, being under the auspices of the Linux Foundation, ensures sustainability and further development. HPE is a renowned manufacturer of switch hardware backing the project.

OpenSwitch is not the only player in the NOS market. Dell, for example, cooperates with Cumulus Networks and has a development of its own on the roadmap: OS 9 (Dell NOS 9) [29] based on NetBSD. Another is VyOS [30], which has a history in the Brocade camp.

The project documentation is quite extensive, with just a few open construction sites. Some examples are even misleading, because they fail to mention much-needed parameters. Elsewhere are references to outdated versions.

At the end of the day, OpenSwitch is a complete open source operating system for switches based on well-known projects such as Yocto or ONIE that offers Linux users easy entry to data center networking.