Motor for the Cloud

VMware provides a platform for building an internal enterprise cloud environment with the vRealize Suite. The heart of the suite is vRealize Automation, a cloud management platform, which has now reached version 7. It promises fast provisioning via the self-service catalog, automatic provisioning of complex services, and multi-vendor management of cloud resources (Table 1).

The main field of application for vRealize Automation is automated and standardized provisioning of infrastructure resources – from individual virtual machines (VMs) through complex environments for developers and users. In many companies, this is still a manual process where provisioning such resources can take several days or even weeks. In contrast to this, vRealize Automation users can order desired environments from a web-based self-service catalog and then sit back and wait for vRealize Automation to fully provision this using automation techniques. Such development environments are typically generated automatically for the customer within a few minutes.

With this automation solution, VMware is focusing primarily on generating infrastructure components – that is, individual or whole clusters of VMs and associated networks and firewalls but also components such as virtualized load balancers and routers. Additionally, it is possible to publish virtually arbitrary services, such as resetting passwords or creating new users in Active Directory in the self-service catalog. This is achieved with the help of vRealize Orchestrator – an orchestration engine provided to vSphere customers free of charge by VMware. This engine plays an important role as the mediator for the vRealize products and now enjoys widespread support in the VMware community; many additional plugins are available for it.

A further field of application for vRealize Automation is supporting complex DevOps environments. This no longer simply involves creating individual infrastructure components; instead, it is about deploying complete applications and managing them throughout their life cycles – from creating and updating an application until its final deletion.

In addition to automating the process of creating resources, the focus in many enterprises is also on governance. The product lets you manage the entire life cycle of provided resources via policies and processes. Among other things, this includes multilevel approval processes or precise guidelines on the deployment duration for resources.

Installation Vastly Simplified

Because of the complexity and the history of the product, the installation of vRealize Automation was always a challenging task in previous versions. For three years, VMware has gradually ported the original Windows-based product to Linux. Although deployment of the supplied Linux appliances is easy, much manual configuration work was required for the remaining Windows components in the previous version. VMware has addressed this point of criticism in the new version and provides a comprehensive web-based installation wizard (Figure 1) that can perform a distributed installation for various deployment scenarios (small, medium, or large environments).

For admins, this means that the install itself is dramatically simplified – on the whole, administrators only need to handle the preparatory work, such as deploying Microsoft SQL Server and the basic configuration of the Windows machines, which is a big relief compared to previous versions. Even larger environments can thus be set up within a short time.

Architecture and Operation

Individual vRealize components can be based both on Windows instances as well as on Linux appliances, due to the history of the product. In version 6, VMware started to gradually port features based on Windows in the previous version to Linux. A minimal deployment – but sufficient for a small lab or training environment – therefore requires at least one Linux appliance and one Windows node along with Microsoft SQL Server.

In medium-sized and larger environments with high availability requirements, the number of nodes in the vRealize Automation environment increases significantly. Figure 2 illustrates this logical design for an enterprise deployment: In addition to a cluster of Linux appliances including load balancers, two further Windows clusters need to be managed. First, there is the Model Manager Cluster, which is responsible for storing the Infrastructure as a Service (IaaS) data and deployment workflows. Second, the Manager Service handles communication with external systems, such as vSphere, Hyper-V, or even public cloud environments like AWS. Communication with the underlying hypervisor and cloud providers is handled by the "DEM worker" and agents.

Despite the complexity of the product, operation management in the vRealize Automation environment involves remarkably little effort – this works especially because VMware lets you manage and maintain the installation centrally.

Deployment with the Self-Service Catalog

The most important building block of the vRealize Automation portal is the self-service catalog (Figure 3). This is the centralized interface between cloud consumers and service providers and can be most easily described as an online store for IT services. The requirements for a service catalog can be simply summarized: A catalog should be easy to use and provide a well-defined selection of ready-to-order services. On the other hand, the catalog location should capable of publishing ready-to-order services in a way that is as granular as possible. In the case of VMs, this means, for example, that consumers can make adjustments to the CPU, memory, and storage before ordering.

It is, of course, also beneficial for admins to have the ability to customize existing forms as well as create new ones. Because the catalog is used by different user groups, simple and centralized administration of user permissions is also essential. A customizable service catalog of this kind has always been vRealize Automation's strength. In conjunction with vRealize Orchestrator, the deployment options extend to arbitrary IT services, assuming they can be automated by Orchestrator – a major benefit compared to competitor's products.

Another important feature – especially for large enterprises and cloud providers – is multitenant capability. It is important to provide each client with a dedicated organizational unit with its own directory service, permissions, and services, as well as isolated hardware resources for deploying VMs. vRealize Automation scores points here, too. One small drawback remains, however: vRealize Automation does not support client-specific data store libraries, for example, for storing ISO images.

Easy Management of Cloud Resources

The main task of any cloud management platform is to bind the underlying cloud resources to the cloud platform and thus allow consumers to provision services. vRealize Automation allows all of this. It is not only possible to integrate VMware resources, such as vCenter clusters, but also resources from other vendors. For example, administrators can easily integrate Microsoft Hyper-V, Microsoft System Center Configuration Manager (SCCM), KVM, Citrix, vCloud Director, vCloud Air, or Amazon Web Services. It is not for nothing that VMware positions vRealize Automation as multiplatform cloud management. Nevertheless, the tool plays out its full strength in VMware environments, not least because of the good integration with other VMware products, such as NSX (for virtualization and security automation), vRealize Operations (for monitoring and capacity planning) or vRealize Log Insight (for log analysis).

To be able to manage cloud resources and make them accessible to users on the self-service portal, vRealize Automation implements a kind of logical layer model abstraction. The lowest layer is where the hypervisors or cloud providers reside. For vRealize Automation to be able to access them, admins need to configure end points. The discovered resources then form the fabric. Cloud administrators can then group fabric resources by specific criteria. For example, it would be possible to pool all resources at one location in a fabric group. Many other criteria are also possible, however. Fabric groups can be used to isolate hardware resources, create service tiers, for licensing reasons (e.g., when creating a fabric group for Microsoft SQL Server), or they can be created based on other criteria.

Tenants are in turn divided into business groups. These are typically organizational units such as departments or customer projects that require dedicated user management and resource allocation. Reservations give business groups the ability to leverage the underlying resources when provisioning of infrastructure services. A reservation is a one-to-one mapping between an underlying resource and a business group. Reservations are also important because administrators use them to control what operating resources vRealize Automation can provide to the individual business groups. Computing resources, memory, storage or available networks are considered operating resources. Once the resources configured in the reservation for a business group are exhausted, no further provisioning is possible with the reservation.

All told, the model implemented by vRealize Automation is impressive. It abstracts the underlying virtualization and cloud platforms and thus enables broad support for a wide range of hypervisors and cloud providers as advertised by VMware. At the same time, the approach is sufficiently flexible to make it easy to implement a wide range of customer requirements and extensions.

Simple Provisioning of Complex Services

Before the service catalog can be populated, administrators first need to define the services to be published in vRealize Automation. The central element of publication in vRealize Automation is the "blueprint," which handles all aspects of a service. In the case of VMs, this includes the actual hardware resources (CPU, memory, disk space), but also the deployment procedure. For example, vRealize Automation can produce a machine by means of (linked) cloning, but also supports techniques such as booting from ISO images, Linux Kickstart, SCCM, or Windows Imaging (WIM) file format.

VMware has given special attention to the process of creating computer networks and applications in the most recent version. The new Blueprint Designer (Figure 4) stands out here; it lets admins create complex design templates for application environments using drag-and-drop techniques. Thanks to integration with VMware NSX, admins can also provision applications along with a dynamically generated network stack and various network tiers.

The latest version of the intuitive Blueprint Designer sees VMware keen to deploy VMs as well as install or configure software. This is achieved with the new software components, reusable modules for running cmd, PowerShell, and Bash scripts on VMs that can be easily applied to VMs using drag-and-drop in Blueprint Designer. To avoid the need for cloud operators to reimplement all software components, VMware provides a range of ready-made components (e.g., for Microsoft SQL Server, Microsoft SharePoint, JBoss application server, or MySQL) on its Solution Exchange Marketplace. If this is not enough, you can easily build a configuration management tool like Chef or Puppet into the provisioning process.

The ability to save blueprints as Infrastructure as Code (IaC) is also new. This means that admins can export or import any blueprint as a YAML file using a command-line tool. This is especially interesting, as this approach lets admins create different versions of a blueprint in a software management tool, such as GitHub, integrate them with other installations, or revert to an older blueprint version.

Creating Additional Features

Although vRealize Automation already contains many features out of the box, in most corporations, the need arises to implement additional features or at least adapt the product such that corporate policies can be observed. The list of possible adaptations is long.

Simple changes only relate to the forms in the service catalog where you might need to add additional fields or to introduce restrictions for existing fields. It is often necessary to customize the life cycle of a VM. For example, it might be necessary to integrate a machine into an enterprise-wide IP address management tool, to ensure a valid IP address and hostname, or define which network to use, before actually provisioning the machine. After the deployment is complete, it is customary to create an entry in a configuration management database (CMDB) in many companies. It may also be necessary to archive the data before you delete a machine. vRealize Orchestrator handles such changes in the life cycle of a service (Figure 5).

Extensions may also be required for previously provisioned resources. Users may conceivably want another button in the user interface, for example, to perform a fully automated backup or install a virus scanner on demand. VMware refers to such operations as Day 2 Operations, because their execution time does not coincide with the original deployment of the resource.

VMware generally recommends the use of its own Orchestrator to implement workflows in a VMware-centric ecosystem. The vRealize Automation service catalog is precisely designed to publish developed workflows in Orchestrator. Companies do not need to develop such workflows from scratch; instead a powerful ecosystem has now formed around Orchestrator. Many manufacturers already offer prebuilt plugins for Orchestra whose workflows can in turn be published in the self-service catalog. As an example, consider the plugins by various storage vendors that support automated building and mounting of additional logical unit numbers (LUNs). Because such services can include virtually any functionality, VMware also refers to this as Anything as a Service (XaaS) in contrast to infrastructure services.

Although it was already possible to make such adjustments in previous versions, VMware has tried to make development work as simple as possible in the most recent version. This means a further simplification of the API as well as centralized administration and linking of events with workflows. The final result was the Event Broker, which allows centralized triggering of Orchestrator workflows. In addition to the life cycle events of a VM (e.g., requesting or deploying a VM), this also includes approval workflows, configuration changes, Day 2 Operations, or event logging. It's also still worth noting that the product has a well-documented REST interface and its own CLI.

Cost Overview Included

In addition to managing technical deployment of VMs and services, administrators must keep an eye on the financial aspects. IT departments are always encouraged to ensure cost transparency and thus to be able to justify their spending. To help admins calculate the total cost of the cloud and allocate these costs to the services to be deployed, VMware bundles the "vRealize Business" costing product with the Standard Edition of vRealize Automation.

In the Standard Edition, vRealize Business can be rolled out as an appliance and connected to vRealize Automation. After completing the configuration, administrators can handle pricing in the catalog in relatively simple ways. Cloud management portal users thereby benefit from cost transparency for the provisioned resource and can determine the total cost for themselves at any time.

Interaction with Other VMware Products

Establishing an enterprise cloud infrastructure is a major undertaking. Although vRealize Automation already implements a large feature set, interplay with other products is essential to meet all customer requirements. VMware, especially the current version, makes a genuine effort to facilitate integration with other in-house products and thus keep the integration overhead manageable.

The integration of VMware NSX is exemplary; it ensures a quick and easy approach to dynamically deploying network components. This includes the production of entire networks as well as the automated deployment of load balancers, as well as defining firewall rules to control network communications. Unfortunately, the current version still lacks a mechanism similar to Amazon's autoscaling, which lets you add more nodes and use a load balancer to scale out applications. For this, VMware points to its roadmap.

VMware has a solution for the issue of monitoring and capacity planning. vRealize Operations can monitor both the vRealize Automation infrastructure and the workloads generated by consumers. The built-in reporting engine in vRealize Operations also supports quick creation of reports on the cloud's performance and load state.

Besides monitoring, VMware offers LogInsight, a dedicated log monitoring tool. Again, integrating this with vRealize Automation is child's play. This means that both the vRealize Automation infrastructure and the deployed VMs can quickly be covered by the centralized log monitoring.

Total integration with other VMware products rounds off the good impression, and it is easy to see that VMware is keen to coordinate the individual products well. This is a massive benefit, especially over comparable products, where integration can be a major headache caused by missing interfaces or different versions.

Conclusions

Earlier versions of vRealize Automation meant that VMware was already a market leader in cloud management, and this has been massively improved with the current version 7 (see Table 2). The advantages of the solution lie mainly in the large feature set, which keeps possible extensions and modifications manageable for most businesses. If you rely on your own developments, vRealize Automation supports you with impressive extension options and tools, such as vRealize Orchestrator.

The product's architecture is complex because of the history of Windows, but VMware has managed to keep the operating overhead in limits with good tool support. Also the integration with other VMware products is convincing – whether this be NSX, vRealize Operations, or even LogInsight. Meanwhile, a genuine ecosystem has also formed on the market centered on VMware's cloud solution. One thing holds true for any implementation of a cloud management platform, however; do not forget the concept design overhead required to implement the product in a larger environment.