![Lead Image © Kae Horng Mau, 123RF.com](images/Kae_Horng_Mau_123RF-Water.png)
Security issues when dealing with Docker images
The Crux with Leaks
Docker Hub is easy for users, and the docker
command-line tool can directly tap into it. You can easily pick up prebuilt images for CMS, databases, or distributions and import them into your local infrastructure. But what guarantees do users have that the software running in the container is also free of vulnerabilities?
Threat Modeling
To start, you need to distinguish between threats; security experts refer to this as a threat model. In this case, there are three threat scenarios:
- The manufacturer embeds malicious code and offers infected images.
- Attackers tamper with the software en route from the manufacturer to the user.
- The manufacturer fails to eliminate known security vulnerabilities in its images.
Users need to select software vendors they trust for effective protection against the first case. Well-known and reputable companies would be reluctant to compromise their reputations, but a distant dubious download service should inspire some skepticism. Finding out who actually offers an image on the Docker Hub is important, because potentially anyone could upload it. Docker, Inc. [1] does not check uploads and typically leaves this responsibility to the user.
A good image usually contains a note on its build instructions – the docker file. Repository sites such as GitHub typically host these descriptions and let you download them. Thus, every user can reconstruct how an image was created. Of course, a review of this kind takes time, but it is worthwhile if the image in question will be playing a central role in your own infrastructure. Examples of this would be, for example, basic images for a Java application server or a preconfigured CMS in a container.
Official Images
The name of the image on Docker Hub consists of two components: The username and the actual name of the image, such as johndoe/mysql
. If the first part is missing up to the forward slash, it is an official image. Official images play a very prominent role on Docker Hub, they are usually from the upstream projects themselves.
The official images help to ensure quality. Docker employees examine the related content and collaborate with upstream developers, security experts, and the community, according to company statements. Docker, Inc. defines some best practices for construction images with a view to security and other aspects [2]. This includes, for example, making sure that you only download the necessary Docker files via encrypted TLS or correctly validate Open PGP signatures.
Ultimately, Docker, Inc. accepts no liability whatsoever for the image. The projects themselves push their build instructions into a GitHub repository, from which Docker's service then assembles the images. It is very instructive to take a look at the help texts, scripts, and patches in the form of comments and pull requests hosted there. Currently, the figure is just over 100 official images, only a fraction compared to the tens of thousands of images to operator specs that Docker Hub offers.
But if you do not trust Docker Hub, official images or not, you need to look around for alternatives. Some providers of container platforms offer their own registries, Red Hat and Amazon, for example. Conceptually, this changes nothing in the threat model.
For production use in the enterprise, admins would do well to set up their own registries. This is something that you can do amazingly quickly, because there are also Docker images for this. The system administrator then only uploads in-house images or personally tested and verified images from other registries, to this registry. At the same time, you would exclusively allow the run-time environment access to this registry instead of the default.
Damaged in Transit
The second main threat can be quite easily contained with current Docker tools, because the communication between Docker and the client, the Docker host, and the registry is usually TLS encrypted and protected by SSL certificates without requiring too much work from the user.
The Docker Notary [3] project offers an approach for verifying end-to-end whether an image actually comes from the provider who the image claims created it. To describe this in a greatly simplified way, The Update Framework (TUF, [4]) on which the project is based computes a checksum on the manufacturer's side, and users then verify the image. The problem that this project faces is the immense complexity of some of the details, which makes it difficult to understand and thus difficult to use for many users.
Obsolete Software
If you manage computers, you will be aware that the flood of updates and fixes for bugs and vulnerabilities shows no sign of slowing down. Thus, it's an essential task to find out faster than attackers whether an image contains vulnerable libraries or executables.
This task proves to be all the more complex in a container environment because each container has its own set of libraries and tools. At the same time, patching is considered to be anti-pattern, because each instance actually needs to be ephemeral. You want to be able to use the newer instance of an image to update an older one at any time.
When you learn of a vulnerability, you need to build a new image based on the template on Docker Hub. But, how can you ensure that the template is not vulnerable? To do this, Docker, Inc. operates a supplementary service, known as Docker Security Scanning. It screens official images for paying customers, so far, only when specifically hired to do so, and more recently as part of the Advanced Enterprise Edition (see the "Docker's New Licensing Model" box).
To allow this to happen, Docker operates a database of files with known security vulnerabilities. During the building on the Docker Hub, the image maker matches the files generated in any filesystem layer against this database. A web front end of the hub shows the results, and every visitor can view these – provided they log in with a free Docker ID.
The Docker Hub website [5] offers a search box in which users can enter the container name (e.g., mongo
for the NoSQL database). If the attribute official
appears in the results list, you can click on the image name and select the Tags
tab. Now, a color-coded map of the image's security status (Figure 1) appears for each version of the image. Critical errors are marked red, major vulnerabilities in orange, and minor vulnerabilities yellow.
![Users who log onto Docker Hub with their Docker ID to search for official images see a little more than anonymous users: The results of the security scan appear for each version. Users who log onto Docker Hub with their Docker ID to search for official images see a little more than anonymous users: The results of the security scan appear for each version.](images/abb1.png)
Clicking on the Tag
name shows more details, such as the name of the affected package, a classification, and the CVE number that provides more details of the discovered vulnerability (Figure 2).
![In each image layer, the Docker Security Scanning service uses static analysis to check whether it can find a file with a known vulnerability. In each image layer, the Docker Security Scanning service uses static analysis to check whether it can find a file with a known vulnerability.](images/abb2.png)
Independent software vendors can also offer certified containers in the new Docker Store, which Docker continuously scans for security weaknesses.
Sobering
The results of the scans are sobering. An analysis I performed some time ago revealed that just seven percent of the 114 official images at the time were completely free of warnings. The worst offender had no less than 69 safety warnings. Current sampling does not substantially change these findings. Either manufacturers are not taking the tests seriously, and not focusing on the messages quickly enough, or the scanner is so sensitive that it returns too many false positives. Both presumptions do little to justify the user's trust in the production maturity of the service.
Other manufacturers offer a very similar approach. At CoreOS, the security scanner goes by the name of Clair [6]; it scans local and remote container images, including Docker images. In Red Hat's Project Atomic has an Atomic scan, which uses a plugin system [7] and the OpenSCAP local policy scanner [8] for the scan.
Security Included
No single tool automatically secures container environments, not even if you wave money around to entice it. The recommended approach is to see what your own environment offers in terms of specific tools for enabling the described mechanisms.
Before starting an instance of the image, administrators should check – manually or preferably in an automated process – whether it comes with known vulnerabilities, and only continue if these are patched. Unproblematic containers today can easily fall foul of effective attacks tomorrow. It helps to keep track of the installed components, scan your own system regularly (see box "There are Scanners, and Then There Are Scanners"), and use a container management system that supports your efforts to build and deploy new images quickly in an automated workflow.
Good Sources, Quick Scans
Many system administrators retrieve software from a variety of sources. It is always advisable to carefully consider the providers of these sources – this also applies to classic software without containers. In this regard, Docker and others are no less or more secure than traditional installations.
If you provide images for Docker, but also offer rkt or Project Atomic, you should disclose how you handle updates and known vulnerabilities. This is the only way to ensure permanently secure operation of such software. Whether a commercial product can be linked with higher quality has been the subject of much discussion since the early days of open source, because some promises relating to security quality can genuinely be purchased with commercial applications.
The different static security scanning offerings usually test signatures. The results of the current implementations are mediocre and contain many false positives. The question arises as to whether the additional overhead of reworking actually justifies the amount of money you pay.