Features Windows PKI Lead image: Lead Image © vlue, 123RF.com
Lead Image © vlue, 123RF.com

Windows security with public key infrastructures


A rarely used feature for improving security in Windows environments relies on certificates issued for various applications, services, and procedures that is based on a public key infrastructure. By Marc Grote

Establishing and maintaining public key infrastructures (PKIs) is considered complex and time consuming. Moreover, they can be expensive if you want to use public certificates to improve security. That said, no other system provides an equivalent level of safety. In this article, I introduce the main components of a PKI and show practical implementation options using examples such as email encryption in Outlook and VPN access.

Microsoft's operating systems and applications provide many safety features for improving data protection. Very often, these measures focus on enhanced login security through a stronger and improved password policy, additional software for login security, or the use of firewalls on servers and clients or centrally on gateway servers to the Internet. A feature rarely used to improve security in Windows environments is based on a PKI that relies on certificates issued for various applications, services, and procedures.

All too often, sensitive data transfers are still transmitted without protection, email with sensitive content is not encrypted, or the latest business report is transferred in the clear over FTP. Logging on to the web interface of a third-party solution to inventory software relies on Active Directory (AD) authorization, but the username or password are transmitted in clear text via HTTP. These examples show how a lack of understanding or expertise in the field of encrypted connections can cause security problems.

PKI Components

A PKI comprises the certification authority (CA), a registration authority (RA), and a validation authority (VA). The CA issues certificates, which are requested by the RA and validated and approved by the VA. Additional components include, for example, a revocation list distribution point in the form of a certificate revocation list (CRL) and the Online Certificate Status Protocol (OCSP) to verify the validity of certificates.

Certification authorities come in private and commercial flavors. A private – or internal – CA is operated by the IT department on your local network, and the trust extends only to the computers that are located on the corporate network. In the case of an enterprise CA with a Windows server, for example, all the members of the AD forest trust each other.

A commercial CA typically provides services against payment. In contrast to a private CA, the trustworthiness of a commercial CA is ensured by the fact that most manufacturers of operating systems, web browsers, and applications have already implemented their CA's root CA certificates, so that the client can validate the certificates without seeing error messages.

Certification authorities can be hierarchically structured, as well. The root CA is the root of a CA hierarchy, which can include intermediate CAs. Below the intermediate CAs, you then find the issuing CA.

For many versions now, Windows servers have provided functionality for setting up a CA infrastructure. For the optimal integration of a CA in an AD environment, an AD integrated enterprise CA should be used, because the integration also allows issuing certificates automatically; ensuring the trustworthiness of the root CA certificate by its automatic addition to the certificate store of all AD domain members; and other features, such as automatic key archival and storing of the public certificates of users in AD.

Requesting a Certificate

Commercial CA certificates are typically requested through a web interface and issued after final verification of the requester's identity. When you use a Windows CA, the certification authority and the certificate template settings determine how certificates are issued. By default, certificates are almost always issued without the interaction of the CA manager. You should modify this behavior in the certificate templates or in the properties of the CA issuing the certificates, so that the approval of the CA administrators is necessary for each certificate request.

Certificates can be requested in different ways from the internal CA. The typical approach in previous versions of Windows server – requesting certificates in a web interface – is outdated. The recommended approach is now a Certificate Microsoft Management Console (MMC) snap-in, or the use of PowerShell or command-line programs. Many applications, such as the Internet Information Server (IIS) provide options for requesting certificates in the graphical front end. Some applications can only create a certificate request process in the form of a certificate request file. The generated request file must then be submitted manually to the CA administrators, and the certificate request completed in the CA management console. Then, the system transmits the generated certificate to the requester.

Encryption and Digital Signatures in Outlook

Two approaches have been established for digitally signing or encrypting email: PGP (Pretty Good Privacy) and S/MIME (Secure/Multipurpose Internet Mail Extensions). Whereas PGP is not natively available in Windows systems and applications such as Microsoft Outlook, S/MIME is included in the current versions (Figure 1).

S/MIME is the only on-board tool for encrypting email in Outlook.
Figure 1: S/MIME is the only on-board tool for encrypting email in Outlook.

A Windows Server CA is capable of issuing email encryption certificates for users and distributing them automatically via Group Policy, as well as automating the configuration of the Outlook client. The public key for the email certificate can be stored for each user in AD. This means that any user can exchange email encrypted or signed with other AD users. Optionally, you can configure the CA to archive private encryption certificates: If a user no longer has access to their private key, a trusted CA administrator, a key recovery agent (KRA), can restore the certificate with the private key and make it available to the user.

To set up email encryption for Microsoft Outlook users in an AD infrastructure, take the following steps:

Testing Encryption and Signing

To check whether the encryption certificates were issued correctly and the appropriate Outlook settings are in place per Group Policy, start Microsoft Outlook 2016; in the settings, navigate to Trust Center | Trust Center Settings | Email Security, where you should see a certificate for the logged-on user account, as well as the settings for automatic signing and encryption of email.

Even if you can largely automate the distribution of certificates and the Outlook configuration, end-to end encryption in Outlook has several disadvantages:

These and other arguments contradict end-to-end email encryption and have meant that most companies that require email encryption on a larger scale opt for centralized encryption gateways to eliminate almost all the disadvantages mentioned.

Using the Encrypting File System

The Encrypting File System (EFS) technology encrypts individual files or entire directories on a volume formatted with NTFS; it is available for almost all Windows operating systems (Figure 2). In contrast to full volume encryption (FVE) with Microsoft BitLocker, you can selectively encrypt data using EFS on the local device, as well as on a file server on the network. However, files encrypted with EFS are not encrypted for transferring across the network. EFS provides a multiuser function that lets users give others the right to access files encrypted with EFS.

Certificates encrypt files with EFS.
Figure 2: Certificates encrypt files with EFS.

Users encrypt files and directories with EFS by selecting the desired data in Windows Explorer and then enabling encryption in the properties. The data is encrypted with the EFS certificate stored in a user's profile. If EFS does not find an enterprise CA in AD, the EFS process creates a self-signed certificate with a longer lifespan. This certificate is now used for encrypting and decrypting data.

Administrators can control the use of EFS in a corporate network with Group Policy. By default, all users can encrypt data using EFS. If no PKI is available and you cannot ensure that encrypted data can be restored with EFS, then when the user no longer has access to the EFS certificate, you should disable the use of EFS globally.

To recover files encrypted with EFS, an EFS recovery certificate is created in an AD environment. The recovery certificate is located by default on the first domain controller in the AD forest and used as the recovery agent for EFS-encrypted files. Only users who possess the EFS recovery agent certificate, can decrypt files. For this reason, you should protect the EFS recovery certificate with your private key, store it in a safe place, and renew it at regular intervals.

Before using EFS, you should plan well, document your steps, and provide functional recovery scenarios. A PKI makes the use of EFS more efficient and more secure, because EFS certificates are centrally issued by the CA, private key certificates can be archived, and administrators have an overview of the EFS certificates issued.

Certificates for Web Servers

IIS is still a commonly installed server role in the current server versions of Windows, because it is often used as an administrative front end by third-party applications or applications that use the IIS web services. Often IT managers then issue self-signed certificates during the installation of these applications to allow IIS to encrypt web server communication. Unfortunately, one still often sees unencrypted access to sensitive data and information or unencrypted system logins, because the manufacturer or administrator is apparently not aware of the extent to which this configuration is exposed to data misuse and hacker attacks.

With a Windows PKI, you can simply replace the self-signed certificates with CA certificates or encrypt the previously non-encrypted web server communication. When using a Windows CA, you can copy the "Web Server" certificate template type, configure your own settings in the template regarding key length and validity, and distribute certificates based on the template automatically using Group Policy. Therefore, you can extend your web server by adding HTTPS bindings or replace existing self-signed certificates with CA certificates.

Using the IIS Management Console, you also can request certificates manually from the certification authority and then bind them to the HTTPS protocol. If you also run IIS functions such as the central SSL certificate store, you can create an efficient solution for encrypting web server communication with HTTPS.

Securing VPNs with Certificates

When using site-to-site VPN connections with protocols like L2TP over IPsec, it is standard procedure to work with certificates, thus avoiding insecure Pre-Shared Keys (PSKs). Given the increasing trend toward the use of mobile devices (which are not always company property) to set up a VPN connection to the corporate network, it is important to ensure that only desired devices can establish a VPN connection. In addition to solutions such as network access control (NAC), you can also use client-side certificates, on the basis of which the devices or users then authenticate. Only users or devices that have access to the certificates can successfully establish a connection to the corporate network.

Current Microsoft client operating systems such as Windows 10, as well as some older versions, offer the possibility of establishing a VPN connection on the basis of certificates. Windows 10 provides a wizard for users that lets you use smart cards as credentials (Figure 3); these can be classic smart cards in the form of a USB stick or a virtual smart card [2]. Windows 8 was the first operating system to support the creation of virtual smart cards that act like traditional cards by storing certificates for authentication. Like conventional smart cards, access is protected by a PIN.

On Windows 10, VPN connections can be secured with a certificate on a smart card.
Figure 3: On Windows 10, VPN connections can be secured with a certificate on a smart card.

Certificates for DirectAccess

DirectAccess is still a fairly recent Microsoft VPN technology, which saw the light of day for the first time in Windows Server 2008 R2. It became more widespread in Forefront Unified Access Gateway (UAG) and is still used as a server role in Windows Server 2012 and R2 after Microsoft discontinued the standalone product. The special feature of DirectAccess is that you do not need a traditional VPN client; instead, the DirectAccess capabilities are built in to the client operating system as of Windows 7. After initializing the network stack, the computer establishes a VPN connection and is already connected to the corporate network without a user login.

DirectAccess relies heavily on the use of certificates, both on the client and server sides; if the simplified DirectAccess Setup Wizard is not used, that eliminates the need for a PKI. An internal Windows PKI is recommended for a professional implementation of DirectAccess. If you are using Windows 7 as a client or using multisite DirectAccess, the use of a PKI is imperative. The Network Location Server (NLS) needs a web server certificate issued for the internal DNS's fully qualified domain name (FQDN). The CRL distribution point must be accessible via HTTP from the Internet, and the certificate must be trusted on all of the components involved in DirectAccess. For IPsec communications with DirectAccess clients, the DirectAccess server needs a computer certificate from the internal CA that is issued to the FQDN of the internal DNS name. For the IP-HTTPS communication of DirectAccess clients with the DirectAccess server, you need a web server certificate whose common name (CN) matches the external DNS FQDN; this is used by DirectAccess clients on the Internet to reach the DirectAccess server. For an advanced DirectAccess configuration of the IPsec connection, DirectAccess clients need a computer certificate issued for the name of the computer.


An internal PKI provides the basis for secure communication of almost all infrastructure components. If you use AD, you should always choose to use the built-in certification authority. If you need to issue certificates in an environment without AD or in a demilitarized zone, you should consider a standalone CA. For all forms of PKI, you should always plan and consider both the technical aspects of and the management processes for establishing the PKI. A carefully implemented PKI can then be the basis for protecting many processes and procedures; it is guaranteed to improve security significantly in Windows environments.