Identity Transfer

The synchronization of users, groups, and contacts between directory services can be a challenge. Windows Server and Active Directory (AD) do not provide any functions for this out of the box. Microsoft Identity Manager (MIM) 2016 can help to sync not only identities in the local AD, and between a variety of sources, but even in Azure AD and Office 365.

MIM 2016 is a classic identity and access management (IAM) tool. It is strongly linked with AD and Exchange and offers features for provisioning identities, role-based access management, certificate management, and self-service tasks, such as resetting passwords, managing smart cards, and synchronizing objects. All MIM tools deal with the life cycle of identities. The key feature is the MIM portal. This is where many of the suite's identity management-related functions and tools are controlled.

The synchronization service in the background is used, for example, to implement the rules defined in the portal for creating new users. But even without the portal, the synchronization service can be meaningfully used. Starting from a central location, it lets you synchronize various connected sources without having to use the parent portal. This article looks at how to handle local synchronization between directory services and how to include Azure AD, if you happen to use Office 365.

The product family was launched in 2007 under the name Identity Lifecycle Manager (ILM) 2007 and was a merger of the Microsoft Identity Integration Server (MIIS) and Certificate Lifecycle Manager (CLM) tools. In 2010, Microsoft integrated the products into the Forefront range and Forefront Identity Manager (FIM) was born. Because the Forefront portfolio was discontinued, Microsoft announced in 2014 that it had renamed FIM and that it would now be developed under a new name as MIM. The current version, released in 2015, is MIM 2016, which includes the above-mentioned synchronization service.

This has survived all product cycles almost unchanged. The history of MIM or FIM is also important, because the technical documentation for the synchronization service on TechNet still often refers to the "FIM synchronization service," although the description is just as applicable to MIM because the service remains the same at its core. If you install Azure AD Connect, then a version adapted to Azure AD ends up on the intended server to handle the user information in Office 365.

Structure of the Synchronization Service

If you look at the synchronization service's elements, it soon becomes clear that it is more than a minor service. It is installed on a member server in AD; you also always need an SQL server. SQL can run on the MIM server if you like. Later, I will look at the requirements for commissioning in more detail.

In many cases, AD is likely to be where you will find most of the objects to be transferred, but MIM can also work with other data sources. This does not need to be a directory service such as the Microsoft AD or Novell eDirectory. Text files, or an SQL Server that contains user data, are just as conceivable. Wherever the objects to be synchronized reside, a corresponding management agent (MA) is always needed to access a data source (Figure 1).

Each MA has its own connector space, containing all the imported objects, on the SQL Server. Depending on the requirements, several MAs can be set up so that multiple connector space exist. In this context, the "metaverse" also enters into play; there is only one of these, independent of the scenario. The metaverse is a conglomeration of tables in SQL Server and basically forms the heart of the MIM synchronization service. It is deployed at a central point to keep outbound and inbound-defined objects logically in sync with the connector space. This depends on how the "run profiles" are set up – another important element for the administrator. Via a scheduling service, run profiles are executed regularly; they define imports, exports, and also synchronization for each MA. You will find some very good articles on TechNet [1] with in-depth coverage of the dependencies.

You can thus set up constructs, such as directly creating corresponding user objects in AD from the data of an SQL-based HR application. Or, in case of implementations that have a separate resource forest for an Exchange environment, you could keep the forests synced and populate several ADs. The corresponding disabled user accounts are automatically synchronized and end up in the resource forest – to mention just two examples.

Synchronizing with the Cloud

Office 365 uses an internal directory service in the form of Azure AD that is similar to the local AD. To use Office 365 in a meaningful way in a larger context, AD synchronization is essential. For example, a user account created in Azure AD must be able to license users for the Office 365 services.

Synchronization does not depend on whether the users log on to Azure with the same username and password (same sign-on) or whether a single sign-on is possible via an AD FS implementation. Although you could create the users manually, this makes little sense in larger environments. The administrator's life is simplified if the local AD is coupled with the Azure AD and newly created users are automatically synchronized with the Azure AD.

If this includes synchronized passwords, it simplifies user logins in Office 365. Changes and deletions are also naturally kept in sync. Microsoft offered several tools for this in the past. The current tool is called Azure AD Connect. It is somewhat confusing that Microsoft does quick product changes and frequently changes the name of the synchronization tool. The first version was called DirSync and was replaced at the end of 2014 by its successor, Azure AD Sync. Less than a year later, it was again obsolete and Azure AD Connect (June 2015) was released. Under the hood of Azure AD Connect, you will find a customized version of the FIM or MIM synchronization service, by the way.

One downside is that the MIM synchronization and Azure AD Connect services cannot be run together on a server – they use the same technology at the core. If you already have an FIM or MIM-server and want to additionally set up synchronization with Azure AD, there would be the possibility of using the Windows Azure AD connector on the MIM server. This saves you a server license, but you need to bear in mind that the connector is no longer under active development. Support is still provided, but it does not make much sense to use the connector.

Apart from that, the last new features only made their way into Azure AD Connect (e.g., the ability write passwords back to the local AD) if users change these in Office 365. It remains to be seen how Microsoft will manage the software zoo of synchronization tools in the future. The aim can only be to have a single solution, but it is still not foreseeable today.

MIM in Action

Suppose you have a scenario with an AD forest named KBCORP.DE and a second forest named KBDEST.DE. KBCORP is the user domain in which the users log in. In the KBDEST domain, you need to keep selected user accounts for testing purposes, for example. The domains are not connected by a trust relationship. You also want users from KBCORP to be able to log into Office 365 with their normal user accounts and passwords.

All told, you'll need three servers: one for Azure AD Connect, one for the MIM 2016 synchronization service, and one for SQL Server. Azure AD Connect would also be fine with Microsoft SQL Server 2012 Express LocalDB, for MIM needs SQL Server. It could be installed on the MIM server, but in this example, the two synchronization servers share a single SQL Server 2012; this has no influence on the setup for the two servers. For the software and hardware requirements, see TechNet for Azure AD Connect [2] [3] for the synchronization service.

For this example, you can assume that the SQL Server is available, the KBCORP.DE domain is already registered in Office 365, and the AD domains are also available. Thus, you can turn directly to the Azure AD Connect server, assuming that .NET Framework is installed there. Its version depends on the version of the server operating system on which Azure AD is installed.

Installing Azure AD Connect

In addition to the .NET Framework, service accounts are required in the AD. For the example of Azure AD Connect, you just need the service account to be a member of the domain users group. Depending on which edition of Azure AD Connect you install, however, other rights are needed – especially if information from Azure AD needs to be written back to the local AD. An overview of the necessary setup configurations with the respective rights is also available on TechNet [4]. The Azure AD Connect setup on the server is unspectacular and does not involve any real pitfalls.

After the installation wizard has completed, you will find various programs on the server. They include Azure AD Connect. You are offered the possibility to view the current configuration or edit it, if necessary. The Synchronization Rules Editor allows fine tuning of what it synchronizes and how; transformations are also possible here. The most important program is the Synchronization Service Manager, your command center. This is where the MA and run profiles are defined; you can also browse the connector spaces and the metaverse.

You must be absolutely sure of what you are doing before changing anything in Synchronization Service Manager. In the case of the Azure AD Connect server, all the important settings are taken from the setup routine, and the synchronization service is set up with the local information in place and ready for action, unlike at manual MIM setup, which I will show later. If changes are necessary, again use the Azure AD Connect wizard, which configures the synchronization service with the new settings, this actually eliminates the need for direct adjustments – fine-tuning apart.

The setup also configures the scheduling service and is thus synchronized by default every three hours. In the case of urgent changes that must be immediately available in Office 365, you can use the DirectorySyncClientCMD.exe tool to trigger an immediate sync. After the first run, the users in the online portal are synchronized and can be enabled and provided with licenses there.

Clean up Before Running Setup

It is recommended that you clean up your AD before you start synchronizing with Office 365. Duplicate email addresses or non-standard characters in important user attributes, for example, can cause problems, which will be time-consuming to eliminate later. An Office 365 article [5] shows which preparations are useful. It describes the pitfalls and lets you download the IdFix tool. This is a utility that not only checks AD but also draws attention to potential problems and offers steps for eliminating them. It installs without a time-consuming setup and does not require extensive rights, apart from write privileges for any corrections.

Installing MIM Synchronization

Commissioning the MIM synchronization service on the second server requires more manual work. I'll look at the service accounts first. Because the MIM setup takes place independently of Azure AD Connect, and the two services run separately on their own servers, you need to choose a service account in the KBCORP domain for the MIM server. Again, this does not require wide-ranging administrative rights, either in the domain or on the MIM synchronization server. A domain user is sufficient. For access to the two domains that you want to synchronize when configuring the MA, a separate user account is also needed in the respective domain. Both accounts must have the "replicate directory changes" privilege in their domains. You can assign this in the security settings at domain level. Other rights are not necessary at this point.

The MA in the KBDEST domain additionally requires permissions to create the replicated user accounts in the target organizational unit (OU) of KBDEST. Before you start the MIM setup, you need to install the SQL client on the server. The installation wizard is largely self-explanatory. It is launched via splash file on the MIM Setup DVD. This is also where you will see the remaining products from the aforementioned MIM portfolio.

The setup dialog itself is worth mentioning; you use it to set up the security groups so that you can later associate administrative tasks with user roles. It makes sense to prepend the domain name to the group; this ensures that the setup uses the groups in the domain. Otherwise, these groups are created locally on the server. This is not a big deal, but group management – in particular of administrative groups – should be part of your AD. When you install a hot-standby server for MIM, as described later, you will not get far with local groups anyway, because they are not automatically created and populated with accounts on the second server. AD thus makes far more sense.

Otherwise, the installation wizard is self-explanatory. Even a backup of the encryption key is created at the end. You will want to keep this backup file because it is used for a new installation with the same database or if a second MIM synchronization server, such as a hot standby, enters the scene.

Setting up Synchronization

The Setup Wizard installs Synchronization Service Key Management on the server; this is launched on completing the installation and used to manage the encryption keys on the server. You will also find the Synchronization Service Manager on the machine; this is the main admin tool for dealing with synchronization.

The functions are selected via buttons at the top. In addition to the Joiner, which allows you control what happens with objects in the connector space that are no longer linked, you can browse the metaverse or even edit the metaverse schema with the Metaverse Designer.

The Operations feature tells you the status of the synchronization at a glance (Figure 2). Additionally, you can manage the MAs via the same button. All settings for the connected data source and synchronizing the two AD domains with Metaverse are defined here. The Create command starts a new definition; this requires a fair amount of input. After some experimenting, things start to make sense and the elementary principles are quickly revealed.

The Configure Join and Projection Rule and Attribute Flow steps are especially important here. Projection in MIM-speak simply means creating new system objects, whereas Join refers to changing an existing object. The idea that the new system (projection) would need to be configured at the MA for the KBDEST domain seems obvious – but this is not the case. Because the New process relates to objects in the metaverse, which is handled by the MA for KBCORP, the setup also occurs here. Accordingly, only a Join occurs for the MA in the KBDEST domain at this point.

The administrator sets the mappings of the individual attributes in the Attribute Flow. The attributes previously defined for the MA from the data source are mapped with specific attributes in the metaverse. Using the Flow Direction option, you then specify whether this is an import or export, as can be seen from the direction of the arrow (in the MA for KBCORP from left to right on importing into the metaverse). The handful of attributes from this example are only for illustration purposes. In practice, far more attributes are likely to be necessary. The important thing here is that the attributes have correct mappings from the data source to a metaverse object. The arrows in the MA for the KBDEST domain again point from right to left, reflecting an export with the same attribute mappings.

The settings for the MA are quite complex, which can cause frustration – especially at the beginning – if synchronization is not working as it should. At this point, you might like to check out the help function, which you can launch bottom right in each dialog. Relative to the context, it explains in detail the aspects that are now offered in the dialog. This is quite useful, especially when you are just getting started. Now that both MAs are completed, you need the run profiles for the two agents. They control the flow of data between the directories. The import and the synchronization should exist for both MAs, the Export step only exists in the Run Profiles for the KBDEST MA.

Provisioning Users

One process controlled by the portal – unless you use the synchronization service via the portal – is that of creating or deleting user objects, that is, provisioning. Because you're not using the portal, but just the synchronization service, you'll need to find an alternative. MIM has an interface named Rule Extension for this. This is a way to embed custom code in C# or Visual Basic (VB) that lets you intervene with the data flow of program logic. In the Synchronization Service Manager, you will find this under Tools | Options. At this point, you can also create a code template for C# or VB, which is, however, a one-off measure; you can find many how-tos for the implementation online. Not every administrator wants to be a developer, however, and this is ultimately also a matter of time.

For a simpler approach, you can turn to the Codeless Provisioning Framework [6]. The setup consists of only two files: a DLL, which you need to integrate with Synchronization Service Manager, and an XML file that contains the ruleset. The author includes various provisioning rules in the form of XML files; you therefore have a fair amount of reference material for quickly familiarizing yourself with the subject matter – and without using Visual Studio.

Scheduling Run Profiles

If you navigate in Synchronization Service Manager to Run Profiles, you can press the Script button to create a VBScript, which – when wrapped in a batch file and handed over to the scheduling service – ensures that regular synchronization runs [7]. This is quite convenient, but you will find an even better option in the form of the FIM/MIM MARunScheduler [8], which you might want to consider.

The scheduler comprises two files: MARunScheduler.exe and an XML file that contains all the parameters. You can thus create a plethora of Schedules and, for example, restrict the time frame for each run profile, or you can stipulate that the synchronization cycle run permanently and that changes are only written from the source to the destination in case of changes:

<C>(parameter:) LoopIndefinitely<C>

In this way, the two domains from the example would be permanently synchronized in real time, which offers a variety of opportunities.

Setting up High Availability

As an admin, one question that you need to answer is how to handle high availability. The same applies to the MIM synchronization service. After all, this only synchronizes every few hours (depending on scheduling). If the server fails, a new one is quickly installed, probably between two cycles.

In scenarios where high availability is required, you might prefer a to deploy a second server, on which MIM is installed, in parallel. If you let it run permanently, as a virtual machine with up-to-date patches, but with the FIM synchronization service stopped, it can immediately jump into the breach in the event of a failure. The prerequisite for this is a central SQL server, which is not affected by the failure of the first MIM server, and the file with the encryption keys that was created at setup. Using miisactivate.exe, this server's ID is then registered in the SQL database as the current server. The important thing here is to be sure the first server really is no longer running, and then you can proceed with the synchronization.

Conclusions

The possibilities offered by the MIM synchronization service are often underestimated. In the shadow of Azure AD Connect, which "only" offers synchronization with the cloud, MIM offers unforeseen possibilities for keeping a variety of sources in sync, including data transformation. I have only looked at the MA for AD, but it does not always have to be a directory service. Take the time and experiment in a test environment with the MAs for PowerShell or other MAs. This will certainly result in ideas for everyday administrative practice that can make your work easier.