Pitching the Tent

With corporations like SUSE, Red Hat, Canonical, HP, IBM, and Intel investing time and money in OpenStack, clearly this cloud computing platform is here to stay. On the other hand, OpenStack technological advances do not rank highly with many administrators.

The criticisms are always the same: OpenStack is not mature enough as a project to meet the challenges of the provider's daily grind. Some developers believe the money that comes from corporate funding is not always useful: Precisely because so many stakeholders want a say in OpenStack, the project is finding it difficult to formulate and implement its technological objectives. Therefore, the tension always builds up in the community whenever a new OpenStack version is announced, which typically happens twice a year, as was the case in April. OpenStack 2015 "Kilo" is the successor to OpenStack Juno, but what features does the revised OpenStack offer, and have the developers succeeded in solving some of the major problems?

The Big Tent

Possibly one of the most important innovations in OpenStack has nothing to do with technology: the OpenStack Foundation's Big Tent initiative. The image of a big tent underscores the fact that OpenStack wants to be understood as a catch-all for any project that is related to cloud computing. The implementation of the Big Tent initiative was preceded by lengthy discussions, especially within the OpenStack Foundation itself. Release manager Thierry Carrez describes the problem in an exhaustive but easily understandable paper [1].

Up until Juno, OpenStack comprised one major release, and only the officially integrated OpenStack components were allowed to use the title. Anyone wanting to integrate their component needed to pass through the incubation process and thus survive a number of tests. After finally completing a number of intensive checks, the path for promotion to an official OpenStack component was then clear.

However, so many sub-projects have requested admission to the project that the task of coordinating releases is becoming increasingly difficult. The backlog creates the impression that programs not yet integrated are undesirable, or at least not firmly anchored in the OpenStack community. This impression is not always accurate or desirable.

The policy applied thus far has actually prevented development: For example, two alternatives exist for the Ceilometer component (Figure 1), both of which offer metering for OpenStack. So far, neither of the projects – StackTach and Monasca – has gained official status, because they offer an alternative to previously existing functionality.

Four-Point Program

The Big Tent initiative will mean that any project can call itself an OpenStack component as long as it observes four very important rules:

• The project must identify with the OpenStack goals, and it must be interested in improving OpenStack's reputation.
• The licensing and development models and the developer tools (e.g., Gerrit for code review) must be identical to those used by the OpenStack project.
• Basic API compatibility, for example, to OpenStack's Keystone identity service, must be in place.
• The projects accept the authority of the OpenStack Technical Committee.

Big Tent has already led to heated discussions in the community. Although proponents of these rules are happy to see the project moving in what they consider the right direction, opponents are throwing their hands up in horror. The big tent is normally full of clowns, they say, and the Big Tent initiative is equivalent to the OpenStack Foundation's giving up all control over the technical architecture of OpenStack. They equate the initiative to an end-of-line sale of the OpenStack brand.

The project is looking to counteract this risk with tags. In the future, administrators will be able to tell whether projects are part of the six-month release cycle, or are following their own cycle, simply by looking at the tags. A list of tags is available online [2].

In the case of Kilo, you can say that things are never as bad as they seem: Kilo mainly comprises components that were integral components of previous versions, with the Ironic provisioning tool added on top. Diversification in line with the new rules is thus only likely to take effect slowly after the Kilo release. The time to talk about this topic again will be after the OpenStack Liberty release in October 2015.

Ironic: Finally!

Ironic is one of the major highlights and the only official, new component in Kilo. Ironic only works in combination with Nova, which is the component that manages computing power in OpenStack clouds. If you want to use a virtual machine, Nova finds a matching host and launches the virtual system on it. So far, Nova has been restricted to virtual machines. Ironic adds the ability to handle physical systems. Metal is handled in OpenStack just like a virtual machine: Both can be managed centrally in a web interface.

How much sense Ironic makes remains to be seen. It was created by HP and developed in combination with Triple-O, which stands for OpenStack on OpenStack. The actual hardware in a setup of this kind is an "Under-Cloud" that is controlled by OpenStack on which separate "Over-Clouds" can then be launched for customers. You can try out Ironic with little effort, but if you want to leverage the solution's full potential, you need Triple-O.

Keystone: Less Overhead, More Federation

Keystone does not feature heavily in release messages; this central user management component in OpenStack typically runs in the background. Kilo introduces two major innovations for Keystone. On one hand, Keystone instances of multiple OpenStack clouds can be connected to create a federation network. Users can log into multiple clouds with the same credentials – a genuine convenience feature.

Almost more important, however, is the concept of Keystone Lightweight Tokens, a.k.a. KLWT. When users log into Keystone with a combination of their name and password, they receive a digital key, or token. They use this token to talk to other OpenStack services. Keystone stores each token it issues in a separate database. When users log into other services in the cloud with the token, these services query Keystone to see whether the token is still valid (Figure 2).

The need to save persistent tokens has led to unfortunate side effects, however. Besides performance hits, the Keystone token table tends to fill up in large OpenStack clouds, leaving administrators with no alternative but to clean up manually.

This is where Kilo's KLWTs enter the fray: The idea is that other services can validate tokens without the need to store the tokens centrally. A digital signature is used for this purpose: A KLWT contains user information, the user's project, the token issuing timestamp, and a digital signature.

Based on this signature, a service contacted by the user validates the token independently of Keystone. Morgan Fainberg, who was the Technical Lead for the Kilo release cycle, hopes that KLWTs will improve performance and make Keystone more reliable.

Nova as an Object

The Nova computing service offers several new functions in Kilo. The developers have been busy under the hood and have pushed on with converting to database objects (i.e., to the data model that Nova uses for its own database). Earlier Nova versions used SQL Alchemy to write individual records directly to the database.

That said, if you have a large computing framework with dozens of servers, it can be difficult to coordinate these writes in an orderly way. Conversion to objects was introduced back in the Havana version of OpenStack (2014). Since then, Nova has viewed instances as objects that it can modify directly in the database and using RPC calls across the whole cloud. The layer model therefore practically introduces an abstraction layer, which not all Nova components have used so far.

Kilo comes with many updates for this object functionality and facilitates updating Nova as a useful side effect. Thanks to the abstraction layer, increasingly smaller numbers of Nova components depend on the Nova database schema not changing between versions.

Cell Refresh

In Kilo, the OpenStack developers finally started to expand the Nova cell extension. Nova developers understand cells to be autonomous computing units interconnected via a central location. In Nova, it was impossible to use local databases and RPC services in the individual cells, which can cause a bottleneck if the individual cells are not adjacent on the network. The 2.0 cells in Kilo resolve this problem, thereby boosting their utility value.

Felling Trees in Neutron

Some cloud admins still freeze when they hear the word Neutron. Little wonder, given that this OpenStack SDN component has a reputation of being very complex and hard to manage. As if the developers had heard this criticism, they reacted in Kilo and cleaned up the giant Neutron project to a great extent. This does not mean that any functionality has been dropped, but that, in the future, the single, large Neutron project will be replaced by multiple small projects with specific functions (e.g., L2 or L3 network connectivity).

The parts that were outsourced, such as the LBaaS or VPNaaS plugins, will also benefit from this step: In the future, they will use defined APIs to talk to Neutron instead of directly accessing Neutron code. A separate Neutron library with separate code elements also helps. Kilo is only the first phase of the paradigm shift: By the time the M version is released, the other changes should be in place; each Neutron add-on component will then have its own API.

Perestroika has been dropped for the time being, it seems. This was the code name for a new plugin interface planned for Kilo and intended to offer more functionality for SDN plugins like Open vSwitch or OpenContrail. At the end of February, the developers had to ditch the plan; however, because much work has been completed, you can expect Perestroika to reappear in the OpenStack Liberty version.

If you use OpenStack with Windows, you will be happy to hear about the Neutron OVS agent for Windows, which also comes with Kilo. Cloudbase has again put in some excellent work, successfully managing the Open vSwitch port to Windows and delivering OpenStack integration.

Glance: Minor Changes, Major Effect

For the Glance image service, the Kilo developers only had 21 features on their to-do list – compared with twice this amount for Neutron or Nova. Compared directly with the other components, life in the world of the OpenStack image service seems uneventful. This situation was to be expected: Glance supports practically all the features that are relevant in a cloud.

To tackle the most urgent problem for Kilo, the developers have implemented an improved sorting order, including user specifications for outputting all images in Glance and support for version numbers in line with the Semantic Versioning (SemVer) standard [3]. Performance improvements, such as storing images in the OpenStack Swift object store and new performance tweaks, round off the changes to Glance.

More Drivers for Cinder

The service for persistent volumes, Cinder, is following Nova's lead and also moving towards an object-oriented setup. Based on this, it will be possible in the future to address each Cinder in the way that is already possible for Nova instances.

The reasons are identical and, in part, the Cinder developers are also copying Nova details. What administrators are likely to find far more interesting, however, is the fact that there will be new drivers for Cinder and a new driver interface. The ABC Python library provides the basis for this development. Initial drivers have already been ported to the new ABC interface, such as the rbd driver for Ceph, and others are likely to follow in the course of the Liberty release.

New drivers and driver extensions are available for HP 3PAR, StorPool, or Open vStorage. The Cinder developers have massively extended their iSCSI implementation. The NetApp driver in Kilo now also officially offers Fibre Channel support.

Billing: The Eternal Construction Site

If you use OpenStack in a production environment, you will probably want to bill your customers for the service you provide. To do this, you need a reliable and powerful billing module, which is precisely what OpenStack has lacked for years. Following the public confession by Julien Danjou as the first author of Ceilometer last year [4], it looks like Ceilometer is on the right track.

Very few changes will be made to the Ceilometer architecture in the new Kilo version. Across the board, introduction of Gnocchi as a time-based database will take some time to achieve. If you prefer not to wait, you should take a closer look at the competitor products StackTach [5] or Monasca [6], which offer at least some of the required features.

Spring Cleaning at Horizon

The end user's point of contact with an OpenStack cloud will typically be Horizon, the OpenStack web interface (dashboard). In recent years, very little has been done for Horizon's appearance, and the tool is starting to look a little dated (Figure 3). Kilo resolves this by introducing a totally new wallpaper for Horizon's start instance dialog. The aim was not just to freshen Horizon's look but also to make it much more efficient to use (Figure 4).

Graphical interfaces may be a question of taste, but the Horizon revamp between Juno and Kilo is an undeniable improvement. All told, Horizon feels much more elegant thanks to the new Launch Instance dialog.

For vendors like Canonical or HP, Horizon has some other pleasant surprises. A separate theme interface as of Kilo gives administrators the option of implementing themes directly and in an orderly fashion in Horizon. Although themes were available prior to Kilo, the process required the administrator to overwrite the default theme with a local theme using a Python override. Kilo comes with its own interface for this so that you can, for example, use multiple themes and even lets users choose from various themes in a drop-down menu.

Conclusions

Kilo is not one of those OpenStack versions that impresses with trailblazing new features. The Big Tent discussion raged stubbornly throughout major parts of the release work, and thus consumed much time – although the effects of Big Tent will not genuinely be felt in Kilo. We won't know until a later release whether this initiative actually pays dividends for the project, or just causes more chaos.

From a technology standpoint, the changes in OpenStack Kilo are not exactly epochal, but this makes them all the more understandable. Kilo is a meaningful update of Juno – and one without major destructive potential.