Hybrid Exchange: Making It Easier And Faster To Move To The Cloud

At Microsoft Ignite 2018 this year Jeff Kizner, Microsoft’s Principal PM Manager in the M365 and Exchange Online Engineering team ran a groundbreaking session on existing and new Exchange Hybrid features for a more efficient and streamlined migration to Office 365. In his session, he discusses the challenges Messaging Admin face when moving to Exchange Online. Such as ongoing communications with Network Admin and security implications. In this blog, I am going to elaborate on these issues further and inform how you can overcome them with the new and enhanced Hybrid Exchange features.

The Hybrid Challenge

The first workload in Office 365 is mostly Exchange because other Office 365 applications like Microsoft Teams or Planner rely on the mailbox already being migrated to Exchange Online for a full functionality. For example, Microsoft Teams voicemail or configure connectors in Teams will only work if the mailbox is migrated to Exchange Online. Another reason is that the users don’t have to learn or use new tools because Outlook and OWA will not be changed after the move. This means you want to migrate your mailboxes as easy and fast as possible to Exchange Online.

For some of you it’s going quickly and for others, you might be having issues to migrate to Exchange Online – it’s necessary, but it’s hard. There are a lot of dependencies first, like configuring Active Directory synchronization, choosing the right sign-in method for your organization, and importantly communicating with your security department why Microsoft needs published endpoint from your Exchange on-premises organization to Office 365 (and back).

What we currently have is two separate organizations managed separately. The admin must go over to the on-premises Exchange Admin Center or Exchange Management Shell for one task, like add a new proxy address to an Exchange Online mailbox , then your admin must go to Exchange Online Exchange Admin Center or Exchange Online PowerShell for another task, like add mailbox permissions for a migrated mailbox, and so on. The idea of hybrid isn’t the reality because currently there are two separate organizations managed separately. The vision of hybrid should be an idea of a virtual organization that brings all your users and data under one view. Therefore, everything should be seamless and transparent for the users. Microsoft is working very hard on their vision of a single virtual organization, however, there is still a long way to go.

Administration and Configuration

The current problem is the administration and configuration between both organizations, Exchange on-premises, and Exchange Online.

To adjust your Exchange Online organization with your on-premises organization, there were a lot of things that you must have done manually prior to migrating your users. Customizations of policies like retention policy, OWA mailbox policy, mobile device mailbox policy, etc. must be created and configured in Exchange Online as well. The management of those policies will be always in both of your organizations if you have mailboxes both on-premises and in Exchange Online.

Organization Configuration Transfer

OCT v1 was released in June 2018, it allows you to do a one time copy of Organization Configuration objects to Exchange Online. The subset of policies and objects are:

• Retention Policy
• Retention Policy Tag
• OWA Mailbox Policy
• Mobile Device Mailbox Policy
• Active Sync Mailbox Policy

But those one-time transfer is limited to New-* actions only.

For example, if you had an OWA mailbox policy on-premises that didn’t exist in Exchange Online, HCW will do a comparison between both organizations. The comparison is done by the name attribute of the policy, which means if there is no OWA mailbox policy matching the name attribute, HCW will read all the configured properties on-premises, saves it in the memory and creates a new one (New-* actions) with the same properties, like you had on-premise. If HCW saw a match based on the name attribute, it will skip the OWA mailbox policy completely.

Now in October 2018, OCT v2 is available which allows you to transfer more Org configurations to Exchange Online:

• All OCT v1 objects
• Active Sync Device Access Rule
• Active Sync Organization Settings
• Address List
• DLP Policy
• Malware Filter Policy
• Organization Config
• Policy Tip Config

As part of v2, if a policy object with the same name attribute already exists both in Exchange on-premises and Exchange Online, you can now choose to either overwrite the values of the objects in Exchange Online or keep them as is (Set-* actions). And, just in case you overwrite the cloud settings and didn’t mean to, Microsoft provided a rollback script to undo those changes. Keep in mind that there is not a synchronization engine built in here, it is still a one-time transfer. Every time you run the HCW it’s doing a transfer, which you can run that as many times as you want.

OCT supports Exchange Server 2019, 2016, 2013 and 2010. OCT requires the latest cumulative update or update roll-up available for the version of Exchange you have installed in the on-premises organization. But if that’s too much of a burden, the immediate previous release is also supported.

Hybrid Setup and Onboarding

Microsoft talked to a lot of customers and wanted to figure out why they couldn’t go from hybrid to Office 365. A prevailing problem was the security implications, especially for financial institutes, because most security departments said they can’t allow any inbound traffic to the Exchange on-premises environment, even if it is locked down to the Exchange Online IP addresses. Another example is organizations that outsource their IT resources and can’t make any changes in an appropriate time frame to run the Hybrid Configuration Wizard, etc.

Microsoft has started to think about how they can fix this problem with a few simple principles:

• Keep it simple: It must be integrated with the experiences that you as an Exchange admin had control over.
• Start simple, prove the model, create a platform: Just like OCT, prove what we were doing was possible. But ensure you create a platform that you could continue to extend on.
• Don’t change Exchange authentication, authorization, or throttling: Work within the framework that Exchange has today. We wanted to make as few changes as possible to Exchange.
• Most importantly: we wanted to fix or eliminate the most common problems we saw engaging with all the customers.
  • No customer DNS changes
  • No certificate changes
  • No firewall or network changes
  • Protect on-premises systems

Hybrid Agent

The Exchange Modern Hybrid will install an agent, built on the same technology as Azure Application Proxy, and publish your Exchange on-premises environment to Exchange Online without requiring any of the changes customers have struggled with.

All you need are outbound connections from that agent to the Internet on port 443 for the data moves and port 80 for the CRL checks. What that does is the agent talks to the hybrid proxy service which runs in Azure with a tenant specific endpoint (or a unique host name URL for your environment). This endpoint has nothing to do to identify your environment because it will not be discoverable and this random GUID will be assigned unique to your tenant. The only place you can find this URL is the on-premises logs, in the migration endpoint in your tenant, and the Organization Relationship in your tenant – it is not discoverable anywhere.

For security reasons, the agent has an IP white list and that’s how connections are controlled. It is locked down to the Exchange Online IP addresses.

V1 of the Hybrid Agent will support the core scenarios of mailbox moves and free/busy for your hybrid deployment and is in private preview now.

There is no change for the free/busy request form your on-premises organization to Exchange Online. This means you still must allow outbound connections to Exchange Online. For the connection from Exchange Online to your on-premises organization, the TargetSharingEpr parameter will be overwritten with the unique URL where the Azure App Proxy runs for your tenant.

The same URL change is used for the migrations where the remote server is the unique URL from the Azure App Proxy. Migrations are always a call from the cloud to on-prem and there is only one inbound direction that goes through the Azure App Proxy.

Hybrid Agent – Setup Details

The Hybrid Agent is built in to the Hybrid Configuration Wizard to make the installation as simple as possible.

1. The Hybrid Agent Installer MSI file will be downloaded in the background to always receive the latest information.
2. Then the installer will be kicked off and will do some basic installation tasks, like copy the bits to the machine you are installing on, register some PerfMon counters, and register a Windows Service (Microsoft Hybrid Agent).
3. Then Azure will be contacted to register the agent. What happens here is that a unique identifier will be created for your tenant. A certificate request (CSR) will be created and sent up to Azure along with your admin auth token to validate your tenant. During this process there will be some other properties stamped on the certificate to make sure this certificate can only be used for your specific environment.
4. The certificate will be handed back to you and installed on the box. It is valid for 180 days; the private key is marked as non-exportable and the certificate will be rolled out again automatically 30 days before it expires. For the configuration the backend URL will be stored, and this is where the agent itself on-premises talks to Exchange on-premises. Is it a hostname, a VIP? Microsoft stores that to match the inbound request to where it must hit on your Exchange on-prem servers.
5. Then the frontend URLs will be registered within Azure and validated for Exchange. The validation is super simple and only Test-MigrationServerAvailability will be executed to pass it to the Azure App Proxy endpoint.
6. To complete the configuration, the Organization Relationship will be configured with the TargetSharingEpr endpoint and set the migration endpoint.

The Hybrid Agent v1 will be contain the following functionalities:

The Hybrid Agent will be automatically updated, this means if Microsoft extends the agent with new features, you will get them automatically.

As a recommendation, put the agent(s) on your Exchange servers because it only talks 443 outbound to Exchange Online. Talk to your security department and if they want to put the agent(s) in the DMZ, you can put them also in the DMZ. But this requires maybe an additional server to be installed – keep it simple.

The Send-As Problem

The Send-As problem can’t be fixed with Directory Synchronization, as Microsoft did this for delegate permissions.

The Hybrid Agent can solve this problem, which was shown in a demo during the session. For example, you can add Send-As permissions on-premises for a cloud mailbox and remove those permissions from the cloud again. This is a very early stage and not rolled out to the productive Exchange Online environment now, which means there are no further details available. This is what Microsoft wants to talk about during the next year – how can they improve the goal to be a single virtual organization, what features are coming, what features are missing, etc.

To conclude

• Exchange Online Hybrid setup has never been easier (still in private preview)
• Your networking and security teams can bother other people now

I’m glad to see that improvements will come to the Exchange Hybrid Configuration Wizard to make the move to the cloud easier. But not only mailbox moves, especially the administration and configuration will be a lot simpler than it was before, because the goal to a single virtual organization is now one step closer.

If you’re considering the move to Office 365, you might like the excellent white paper: Email Migration to Office 365.