House of Cool

OpenStack release planning is a smooth process, and release manager Thierry Carrez has the reins firmly in hand. The development of OpenStack 2014.1, alias "Icehouse," progressed with very few glitches. OpenStack  [1] enjoys an extraordinarily good reputation within the FOSS developer, system administrator, and IT architect community because of its reliable release cycles. In the past two years, the OpenStack cloud platform has gained a reputation for being extremely reliable – with both the timeline and the quality of work. The Icehouse release is both a solid maintenance version and a motor for future innovation. What are the changes? Does the OpenStack cloud framework deliver what its developers promised? Read on for a closer look at the new version.

Trip to Oslo

One of the most important innovations in OpenStack is the Oslo Common Libraries Project [2], which has been around for some time but only truly became an integral part of the OpenStack project in recent months. Oslo's goal is to eliminate the often annoying redundancies in OpenStack source code – and prevent them in future.

OpenStack currently consists of several individual projects, such as the Glance virtual image repository, Nova controller, Cinder block storage, and other components that perform specific tasks. Because of mandatory project standards, however, many components need to continually re-implement individual functions themselves: RPC communication with various AMQP services is a good example. In the past, one OpenStack project often has adopted and modified source code from another  – which is bad for several reasons: A new release does not automatically ensure that improvements in one part of the source code filter down to the copies; a manual merge takes time and causes trouble. Additionally, copying source code among the various components unnecessarily bloats the code.

The problem is already known in virtually any programming language, and the solution is a library. Libraries provide a unified API, which other programs then use to incorporate key components into the code.

Oslo, which brings a library of frequently used Python routines to OpenStack, is truly a core project; in contrast to the other components in OpenStack, however, admins rarely have any direct dealings with Oslo modules. In Icehouse, the developers have extended and spruced up various Oslo modules; a complete list is available on the OpenStack wiki at [3]. Also, even if the project looks unspectacular at first glance, it has the potential to change the face of development in the OpenStack project in the long term.

Incidentally, the OpenStack developers are quite willing to look outside the box: If they find a specific functionality outside of OpenStack relevant, they publish the module outside the Oslo framework as a standalone Python module, thus giving other projects the possibility of benefiting from the development work at OpenStack. Once again, the OpenStack project shines as an F/LOSS flagship project.

Detail Work

Elsewhere in OpenStack, much has happened since the last release. In the past, the Glance image repository spoke only English. OpenStack developers had not spent a lot of time worrying about multilingualism. For seasoned admins who are non-native English speakers, dealing with English-only software is usually only a minor problem. End users, however, often find it hard to follow computer programs that speak another language. The call was for better localization, and this has now happened with Glance: The gettext module in Glance now supports localization to any language that gettext can handle. Other OpenStack components will soon follow this example and offer the ability to present logs and API service messages in other languages.

Cloud Block Manager

The OpenStack Nova compute controller also introduces some new features, of which the "Callback API" is probably the most significant. Thus far, the Nova API was mainly responsible for the functions that support starting or stopping a virtual machine. After the VM launch (Figure 1), Nova's work was done – or at least, Nova did not let admins communicate directly with their virtual systems or modify them after starting. For instance, if you wanted your virtual machine to have more than one network connection, or even if you wanted two network ports to reside on different private networks, you used to have to reboot.

The Callback API solves this problem by allowing other components in OpenStack to retroactively change the configuration of a VM. Another benefit of this new feature is creating a snapshot of a VM with Cinder. On request from the admin, Nova informs Cinder then locks down the VM to avoid it becoming a "moving target." Once Cinder is finished, it sends a message to Nova, telling it to stop blocking. The Callback API shows OpenStack's trend toward increasingly integrating the individual components and making them more interactive. Icehouse is the first version of OpenStack to offer this feature.

Neutron: The Eternal Construction Site

OpenStack's Neutron networking service also saw many innovations. The component responsible for software-defined networking (SDN) is well-known for implementing many useful SDN features.

Neutron is also notorious for its complexity, particularly among cloud newbies: The learning curve is intense, and Neutron throws many well-known network conventions overboard. Many cloud admins are likely to be heartily unimpressed by the hustle and bustle of Neutron, because more functionality inevitably comes at the price of more complexity.

In Icehouse, the order of the day seems to be IPv6. To date, IPv6 support was virtually non-existent; but more support appears with the Icehouse release, and Neutron will continue to support IPv6 networks in the future. Routing advertisement will make it possible to connect virtual machines directly with an IPv6 address. (In the past, the lack of IPv6 support has been a showstopper for many enterprise environments.)

The second major Neutron innovation is built-in SDN support. Neutron supports the OpenDaylight SDN environment – more reading for admins to catch up on. In OpenStack Icehouse, the legacy Open vSwitch module is officially tagged "deprecated"; the old vSwitch will likely be completely missing in the Juno release that will follow Icehouse in October. In the previous Havana release, the Neutron developers gave their baby a new framework that ultimately emerged from the wish to avoid code redundancy. Earlier, virtually every module implemented its own basic functions; duplicate implementations were thus inevitable.

The "Modular Layer 2" framework circumnavigates this problem by using an API to provide basic functions that all modules can use. In Havana, the bulk of the Neutron module had already been ported to ML2; the developers have now pushed on with this effort in Icehouse and are preparing to remove non-ML2 plugins for Open vSwitch. In this context, it is almost grotesque that the granddaddy networking component "nova-network" is still not "deprecated" in Icehouse, despite repeated announcements to the contrary; the developers expect the severely out-of-date nova-network to hang on in OpenStack for at least another 12-18 months.

OpenStack Icehouse is the first OpenStack version in which the Neutron service has rudimentary support for "Regions." Regions are used in OpenStack to logically group individual computers on the basis of specific, predetermined properties, but previous versions of Neutron completely ignored region membership. Icehouse offers the option of adding a "region" flag to the database for individual ports in Neutron; Juno will draw considerably more value from OpenStack regions. Additionally, the latest Neutron comes with new plugins and includes significant updates for several existing plugins.

Horizon Keeping Pace

Many of the changes in Neutron and Nova required adjustments in the Horizon web interface or the dashboard (Figure 2). The dashboard developers clearly aimed for continuity: The Icehouse dashboard enhances the functionality of features that already existed in Havana. The VPNaaS plugin has been updated; and the options for editing the "flavors," that is, preconfigured profiles for virtual machine hardware, are also improved in Horizon.

Cinder and Keystone

The block storage service Cinder and the Keystone authentication service typically putter around unnoticed in the background; but, that is no reason to ignore them in this discussion of new features. In Keystone, the new API (Version 3) has seen some updates, and it looks likely the new version will oust the currently more popular version 2 API in the long term. The developers are still not forcing users to make a decision; the old API will continue to work in Icehouse without restriction, even if it is tagged as "deprecated." (It is still unclear whether the legacy Version 2 API will be dropped in the upcoming Juno release.)

Another security-related change is that tokens are now only valid for one hour, as opposed to 24 hours with previous versions. OpenStack's own components can easily task Keystone with issuing a new token if in doubt. However, if you build your own software on top of Keystone, you will need to ensure that the software can cope with the shorter token validity period.

As has often occurred during recent release cycles, Cinder sees some new drivers and updates for the existing drivers: You can use HP's MSA 2040-SAN storage system, as well as additional QoS parameters for HP's 3PAR storage and SolidFire storage solutions. Fibre Channel support, which was already introduced in Grizzly, is enhanced to handle Fibre Channel storage zoning directly from within OpenStack.

Feature Balancing: Heat and Ceilometer

The Heat orchestration service was first introduced in the previous version of OpenStack; and while many observers point to a certain degree of immaturity in the first version, the service has made some great advances in Icehouse. The developers have added tweaks in every possible nook and cranny: You can set a "service" type for a template. Authors of Heat templates have the choice to offer generic templates or stipulate from the outset that a template is designed for use with a specific service. For OpenStack, this option is of great importance because more and more components are offering specific services: Trove is responsible for DBaaS, and Sahara provides Hadoop as a service. Ready-made templates make it easier for admins to use Heat effectively.

A template interface for Heat dynamically expands Heat's features to reflect the customer's needs, with customers definitely having the option of installing plugins. The core of Heat, the engine, provides only a few basic functions, according to the developer's unanimous decision. These functions include starting or stopping VM environments ("stacks" – Figure  3) and automating the network configuration for virtual systems. If you want to automate other functions, use the Heat plugin function.

Some template functions have also been added to the Heat core; the "software configuration" framework for Heat templates will, in the future, use a standardized format to define the configuration of individual services that run on Heat VMs. The design decision suggests that the "small number of basic functions" in the eyes of the developers involved with Heat will clearly go beyond merely starting and stopping VMs. An interesting question in the future will be where the limits lie. What features are generic enough to belong to the Heat core, and what features need to be swapped out into plugins?

The heat developers describe the Adopt feature in a somewhat confusing way in the changelog: In Icehouse, it will be possible for a user to relinquish control over a running Heat stack, thus allowing another user to take over the stack. This feature is designed to simplify the management of VMs.

In the Ceilometer camp, most of the changes that took place will not directly affect users. Granted, Ceilometer runs in the background and acts as an accountant in the cloud; users only have contact with the software via the dashboard. Despite this issue, the developers still introduced some interesting features in Icehouse. For example, Ceilometer understands performance data from VMware's VCenter. Those who use OpenStack clouds with VMware hypervisors now have access to a full-fledged metering application with Ceilometer in Icehouse. If you query the Ceilometer database from your own app, you will have better filter options in the future to determine which results to display. Ceilometer also has a new filter parser that can handle complex regular expressions in Icehouse.

Off the Beaten Track: Swift

The latest version of Swift also offers some exciting features: the Object Replicator, which ensures that binary objects within the Swift cluster are copied back and forth and thus redundantly maintained, has been rewritten and is now faster and more reliable.

When this issue went to press, it was not clear whether the Storage Policies feature would make the cut-off for Swift at the last minute. The aim of this functionality is a tiering-like configuration of content that distributes different parameters within the cluster without making it more complex to call them. Even if the feature did not make it into the Icehouse version of Swift, there is nothing to worry about: Swift releases appear very regularly, usually after an OpenStack release, and during the regular OpenStack release cycle. No need to wait another six months.

New Kid on the Block: Trove

A milestone for any OpenStack subproject is when it officially becomes part of the OpenStack core and OpenStack commits to providing long-term care for it. The Trove database-as-a-service makes its debut in the core release. Through its own API, Trove accepts commands for new VMs and, after processing the parameters of the API command, uses Nova to start a separate database instance, which then comes up with a complete, ready-to-use database. In fact, Trove is the first OpenStack service that operates a helper in the virtual machine itself: trove-guestagent fields commands from Trove and then configures the database on the VM to match the user's specifications. Right now, the service only supports MySQL, but because it has a generic framework in the background, you can safely assume that support for other databases will soon follow.

Conclusions

OpenStack Icehouse (Figure 4) is both a solid maintenance release and a new version with great innovations. You'll find many, very practical innovations in Glance, Nova, Neutron, and Cinder that add new tricks without compromising compatibility with previous versions. After the compatibility disasters of the first OpenStack versions, the attention to compatibility is a good thing.

Heat and Ceilometer float between the worlds: Although both were already aboard in Havana, it is evident in Icehouse that the predecessors offer only a small taste of the true potential that the solution for metering and automation holds. Heat, in particular, generates much excitement in Icehouse because it comes with a whole cornucopia of new features geared to attract users. If the development of the program continues at the same pace, we are likely to see Heat a few steps ahead of its role model, Amazon CloudFormation, in the foreseeable future.

Then, there are the newbies: Trove offers a genuine DBaaS services, and Ironic enters the scene as the successor to TripleO and orchestrates the deployment of VMs as well as that of the hosts running the VMs. Ironic is still not an official core component in Icehouse, but a quick look at the tool is already worthwhile, and it likely will join the ranks of the core components in the near future. More projects are looming on the horizon: Sahara seeks to offer prebuilt Hadoop on the basis of OpenStack, and Mistral will be a sort of cronjob for OpenStack that handles specific commands on a time basis. Things are happening at OpenStack, and Icehouse offers a good insight into the philosophy and the work ethic that dominate the OpenStack scene. First and foremost, Icehouse is a successful release, which impresses with its feature richness.