Package Delivery

Our organization currently follows a systematic manual approach to patch management. This approach includes identification of the need for, acquisition and testing of, resolving issues with, and implementation of the patch.

In our organization, systems administrators typically receive notification of a patch release through research or vendor notification via email. The Information Assurance (IA) team also releases security alerts about a week after the vendor makes a public statement.

After receiving notification of the need for a system patch, the admin downloads the patch from the vendor and saves it to a network repository. Once the patch is in the repository, the admin then tests the patch in a test or development environment. If any issues arise with the patch, the admin contacts whoever issued the patch and resolves the problems. When testing is complete, the patch is ready for implementation; the patch is then deployed in production during the next maintenance window. Once implemented, the patch is kept in a network repository for historical purposes.

A Satellite Server is Red Hat's way of automating the application of software updates. The role of the Satellite server is similar to the role played by Microsoft's Systems Management Server (SMS).

Implementing a Satellite server does not necessarily change the current patch management process; however, the Satellite server will automate the retrieval, storage, and deployment of patches.

The Satellite server also keeps an inventory of the various software packages that are on each server it manages.

Satellite server automatically acquires the appropriate patches, along with the documentation on why the patch was released.

Research Methodology

The primary tools used for this research were the various manuals for Red Hat Satellite Server 5.4. Red Hat's Solutions Architect was also a very helpful resource. We tested Red Hat Satellite Server over a three-month period, with Satellite Server installed on an HP Proliant DL380 G5 Server, using RHEL 6.1 with just the base packages and no modifications. The two client servers were RHEL 6.1.

The first test server was an HP Proliant DL380 G5 server, and the other was an HP Proliant WS 460 G6 located in an HP BladeSystem C7000 Enclosure. The drawing in Figure 1 illustrates the lab environment.

Patching with Satellite Server

Red Hat issues patches as they are needed – sometimes three or four times a week for individual programs. These patches often have multiple dependencies. If a sys admin on a busy network were to implement Red Hat patching manually, it could take hours, if not days, to check and download the necessary dependencies and transfer the files to the server that needs patching.

In theory, a Red Hat system can keep up to date using nothing but the Yum package manager (see the sidebar titled "Patching with Yum"). But for a network with several servers requiring updates, the time involved in the process and the possibility of a critical patch slipping through the cracks lead to the need for a tool that tabulates, manages, schedules, and implements patches. Red Hat Satellite Server is a semi-automated mechanism to implement patches (bug, security, or enhancement fixes) to servers. Those servers do not have to be Red Hat Linux servers; Satellite Server supports Solaris and Microsoft systems as well. The Satellite Server is connected to the Internet and is registered with the Red Hat Network Server that provides patches. It is the only server connected to the Red Hat Network (RHN) [1]. The system administrator defines an idle time at night for Satellite Server to log in to the Red Hat Server and check for updates.

Satellite Server has an inventory of software packages on each client server. When the patches are downloaded, Satellite Server determines which system needs which patch. The inventory is kept in an Oracle database that can be embedded into the Satellite Server or installed on a standalone database system.

The Satellite Server system schedules patches for each client server. It is up to the system administrator of the client server to plan the installation of the patch. Satellite Server provides a solution to organizations requiring absolute control over and privacy of the maintenance and package deployment of their servers. It allows Red Hat Network customers the greatest flexibility and power to keep servers secure and updated [2].

Satellite Server requires a database system for storing package information. As stated previously, the Satellite service can operate with either a standalone database on a separate machine or an embedded database installed on the Satellite Server machine. These two deployment options are functionally similar, although some differences do exist. These variations are primarily isolated to hardware requirements, installation steps, and maintenance activities.

You have the choice of three different topologies for implementing RHN Satellite Server on your network: Single Satellite, Multiple Satellites Horizontally Tiered, and Satellite-Proxy Vertically Tiered.

The Single Satellite Server topology (Figure 2) requires one Satellite Server for the entire network. This configuration is ideal for a lab environment, but it is not very practical for implementation on a network with multiple subnets and many individual departments that administer their own systems. A Single Satellite Server configuration is inexpensive and easy to administer, but it is designed for a flat network and does not scale well to larger environments.

The Multiple Satellite Horizontally Tiered topology has two or more Satellite Servers that are connected to RHN through the Internet (see Figure 3). Each Satellite Server is synced with the others. This topology enables each subnet or project office to have its own Satellite Server. Additionally, the redundant configuration provides failover functionality in case one of the servers goes down. However, this topology also increases the cost of implementing a solution because each server needs its own full license, and the complete system requires more hours to configure and maintain.

The Satellite-Proxy Vertically Tiered topology, illustrated in Figure 4, implements a single Satellite Server and a set of multiple Satellite-Proxy Servers. A proxy server is a package-caching mechanism that reduces the bandwidth requirements to connect to the RHN server. This is accomplished because only the Satellite Server is connected directly to the Internet. The Proxy Server only communicates between the client servers it manages and the Satellite Server.

Although the packages are served by the Proxy, the client system profiles and user information are stored on the single full-version Satellite Server. The proxy acts as a go-between for client systems and Satellite Server. When a patch is released from Red Hat, the Satellite Server downloads the patch. After the patch is downloaded, Satellite Server identifies which system needs the patch. Satellite Server will then send the patch to the correct proxy server that maintains the server that needs to be patched [3].

Only the package files are stored on the proxy server. Every transaction is authenticated by the Satellite Server, The Red Hat Update Agent checks the GNU Privacy Guard (GPG) signature of each package retrieved from the local RHN Proxy Server and the Satellite Server. The GPG signature is implemented at installation. Each system has a copy of the Satellite Server and proxy server signature files for positive identification.

In addition to storing official Red Hat packages, the RHN proxy server can be configured to deliver an organization's own custom packages from private RHN channels using the RHN Package Manager.

For instance, an organization could develop its own software, package it in an RPM, sign it with its own GPG signature, and have the local RHN proxy server update all of the individual systems in the network with the latest versions of the custom software. (Actually, the other two topologies also support a version of this local delivery process.)

A vertically tiered proxy server configuration is scalable and secure. It also conserves bandwidth, because packages are downloaded from the RHN only once, and it lends itself to easy customization for specific architectures and OS versions.

The proxy server license is less expensive than a full Satellite Server license, which results in savings over other multiserver alternatives. On the other hand, you do have to buy a license for each proxy server and train the IT staff of each department to manage it.

Figure 5 illustrates how an organization could use a vertically tiered proxy server configuration to ease the administration of updates and even the delivery of custom packages to different locations.

Test Results

In evaluating Satellite Server, we learned that Red Hat's Satellite Server documentation provides a good overview of the technology, but it should not be taken too literally. The documentation includes a number of errors and omissions. For instance, Red Hat changed all their command names from the previous version to this one, and the documentation was not updated to reflect those changes.

See the box titled "Satellite Server Hardware Requirements" for a summary of system requirements. Keep in mind that you will also need to open a number of ports to use Satellite Server, and you will need to modify firewall rules so that the Satellite Server can communicate with the RHN servers.

The ports in question are 80, 443, and 4545 inbound and outbound, and 5222 inbound only. If a proxy server is being used, you must also open port 5269.

Additionally, a time source needs to be in place, and NTP port 123 must be configured and opened on the servers. Satellite Server and the proxy and client servers all need to be able to synchronize their time with a central time server. Also, a Fully Qualified Domain Name (FQDN) of the Satellite Server has to be registered with the DNS servers.

Red Hat requires the Satellite Server to be installed completely before the application of any site-specific security configuration.

By default, the following services are not enabled: Telnet, FTP, and TFTP. To log into the server remotely, use the SSH protocol.

SELinux on RHN Satellite Server 5.4.1 must be in permissive mode. The system will not work without errors in enforcing mode. (According to Red Hat, this problem will be addressed in the next release.)

For this test, the embedded database solution was implemented. The Satellite Server ran virtually using VMware or Red Hat's native KVM.

Looking Around

The Red Hat Satellite Server is administered through a web interface (Figure 6). The Satellite Server installation script installs an Apache 2.2 web server. Figure 6 shows the Overview page, which provides a single view of tasks and critical systems. To view the Systems page, where the Satellite administrator will spend the most time, select Systems in the top menu. The Systems page has an overview of all systems managed by the Satellite server (Figure 7).

The legend on the left side is important for managing the systems at a glance. The icons shown in the legend identify the current status of a system in the system list. As you can see, the server SAS-TESTING has some critical updates that need to be scheduled for installation. For a closer look, select the name of a server in the list or click on one of the parameters under Overview for details. For instance, after selecting the number 4 under Errata for the server SAS-TESTING, the Errata page appears (Figure 8) showing that two security patches need to be scheduled for installation on the server. Select the patch to install or click Select All, then click Apply Errata. The page that appears (Figure 9) lets you schedule a time for the patches to be applied or to apply the update now; click Confirm.

Figure 10 shows that the two errata updates have been scheduled and two related bug fixes need to be scheduled for installation. You can either select each individual line or click on Select All and then Apply Errata to schedule the installation.

Another useful summary of system information is available through Systems | Details (Figure 11). This page displays the types of updates available (Critical, Non-Critical, and Packages), the IP address of the server, the Kernel version, and more.

If you click on the Packages link (any text that is in blue is a link), the available package updates will be displayed, as shown in Figure 12. This page lists all the packages, and you can click on the package name to get a description of what each package contains. Select the packages you want to install, or click on the Select All button, then press Install Selected Packages to start the update.

Red Hat Satellite Server Licensing

Red Hat Licensing is more like a subscription – you are required to pay the subscription price every year you have the server running. We requested price quotes from three authorized Red Hat resellers. Table 1 shows a summary of the price quotes for one full Satellite Server subscription, plus two proxy server subscriptions and two Management/Provision module subscriptions.

Conclusion

Satellite Server offers an automated way of providing patches to the servers. It can help streamline the process, which currently takes, days if not weeks, to complete. However, you still need to test all patches. Satellite Server provides a mechanism that makes the system administrator's job a whole lot easier.

In our study of the costs for implementing a RHN Satellite solution for our organization, we came across a whitepaper published by IDC titled "Linux Management with Red Hat Network Satellite: Measuring Business Impact and ROI," authored by Tim Grieser, Randy Perry, and Eric Hatcher. The authors state that "…based on data from interviews with IT managers from 10 organizations using Red Hat Network Satellite, IDC's ROI analysis yielded an average 338% ROI – more than three times the initial investment – and an average payback period for the initial investment of a short 4.8 months for the IT organizations in this study" [4].

We recommend certifying RHN Satellite Server in the Satellite-Proxy Vertically Tiered topology configuration. This solution allows granularity in administration of patches. We also recommend you implement Satellite Server in a virtual environment to save hardware costs using either Red Hat's KVM solution or VMware. Red Hat fully supports virtualizing Satellite Server; however, you still need to test your virtual configuration.