Managing Office 365 Quarantine Confusion

Current GUI Causes User and Admin Frustration

The email quarantine process is as old as email itself, and still an essential part of any email service. As far as Office 365 is concerned, Exchange Online is a world-class product and the quarantine mechanism works well, however the current iteration of the Quarantine portal falls short in user experience and performance.

Quarantine, as most of us are aware, occurs when an email or its content is flagged to be potentially harmful to the intended recipient, whether it’s a phishing attempt or a spam email. These emails are automatically corralled and held for review by the recipient or an administrator and then blocked or released.

This protection mechanism, along with the associated user experience, is critical in my opinion because it affects whether one receives legitimate emails or not. Unfortunately, the Office 365 quarantine experience leaves a lot to be desired and deserves some attention from Microsoft.

Management Interface for Admins – Where the Confusion Begins

The quarantine process is hosted by Exchange Online Protection (EOP), a subset of the greater Exchange Online product. EOP performs many of its usual tasks with Artificial Intelligence (AI) and as a result, admins have fewer buttons to push and dials to turn compared to other email protection products. The management interface for Exchange Online Protection may be found within the confines of the Exchange admin center in Exchange Online.

However, the Quarantine settings we need are buried within another portal. As Microsoft works to consolidate the many Office 365 portals, Exchange Online Protection will move from the Office 365 Security & Compliance portal at protection.office.com over to security.microsoft.com, which is currently branded as the Microsoft 365 security center.

While the consolidation of portals is welcome, it does create an extra layer of confusion when admins are trying to solve a problem and can’t find the correct portal to configure a particular policy.

User Interface – The Confusion Compounds

The user experience for email Quarantine isn’t any better, so bear with me as I re-hash the process. I’ll start by logging into Outlook on the Web or OWA at outlook.office365.com. I open my inbox and notice that I have a Spam Notification email (also known as a digest email) which is generated by the Office 365 service and informs me that two items originally destined for my mailbox are now blocked or on hold:

I can either review these messages using the Quarantine page in the Security & Compliance Center, OR I have the option to Block Sender or Review. Note that at this stage we are still unable to release the message, even if we know the sender and are confident that this is NOT a phishing attempt.

*Although the service behaves like this for phishing messages, messages classified as spam can be released immediately by simply clicking the Release button. Microsoft states that “quarantine holds potentially dangerous or unwanted messages,” and, “By default, end-user spam notifications are disabled in anti-spam policies.” For the admin unfamiliar with the default setting for end-user spam notifications in EOP, this means that users may not even notice that emails are held up in quarantine or that they can expire after 30 days.

In this instance, since I know and trust the email address and want to release the message, I click Review, noting that the URL for Review starts with https://protection.office.com//quarantine. My action takes me to Office 365 Security & Compliance portal. Well, it does eventually, after some animation appears, communicating that something is happening:

After the Office 365 Security & Compliance portal finally renders, it redirects to the Microsoft 365 security center as the URL changes to security.microsoft.com. There I’m greeted again by the same animation, a loading notification, and then I’m presented with the quarantine view where I notice that there are surprisingly more messages listed along with the one in question I’m trying to interact with:

At best, this sequence of events lasted well over five seconds, and it was a good 30 seconds before I was able to interact with the message in quarantine. Having used several different vendors for email quarantine functionality, I can say that the time it took between clicking on Review to then arrive at this point, is not on par with other security vendors in this space. Most vendors offer a condensed view with only those emails requiring a review before being released from a performant portal or micro-site.

Now our next step is deciding what to do with the message. I must say, the view of my options is not particularly user-friendly, especially when you consider the non-technical nature of the average person using email:

Hovering over each option reveals more detail and lets the user know what will happen should they decide to click on it. Not every button shows this extra detail though, which can cause user confusion over what they should do:

In my case, I have chosen to release the message, so with that, I click on the Release Message button and the Release message dialogue appears:

A lot is happening with this view, it’s busy and easy to overlook detail such as the option to “Report Messages to Microsoft for analysis,” which is already ticked by default. Noting the accompanying text, we underscore the AI-based nature of the service.

Moving further down the dialog screen, we see our message and see the default option to Release messages to all recipients. As a normal user, I would change nothing here and click Release message. Next, dots will begin to move at the top of the dialog, indicating that something is happening.

After a few seconds or more a notification appears, confirming the message has been released:

Clicking Done should close the dialog and revert to the grid view, but it doesn’t and returns us instead to the same screen we started with:

Now, if the user wasn’t confused and/or frustrated before, they will be soon. Not only does this appear to be the same screen we started with, but all the buttons under Details display the same options and without any next steps. Since there’s no indication that the message was already released, many users will assume there was a glitch in the process.

The user is then presented with three options, but only able to select the third option (which would make sense as the message has already been released) but again, this could be confusing for the average user:

Finally, I click Close, since that seems to make sense, and I see a grid view that includes the release status of my message:

After far too many steps, it appears that I’ve now successfully released the message. And while some of these options in the various dialogs do make sense, I still find the overall experience cumbersome and more painful than it needs to be.

Administrator Grid View – Peak Frustration

If we logon to the Quarantine portal in an admin role, we notice that it is identical to the user view but with the ability to see messages held for all users. The messages may be sorted according to personal preference using the Filter option:

Unfortunately, the default grid view for admins does not display the intended recipient of the held email. The Modify Columns button gives a brief glimmer of hope, but ultimately increases frustration by only offering seven modification choices such as Direction and Message ID and Policy Name. Since the grid is sortable based on columns, having more than these seven arbitrary options available while triaging reasons for the email quarantine would be incredibly useful.

Additionally, the view options do not allow me to remove previously released items from the list, which also limits the view to 100 items at a time. It’s important to note that choosing a large message count and releasing them at once often causes the Bulk actions dialog to eventually error out. This can make bulk actions very exhausting for customers who need to triage a large quarantine view.

Make Your Life a Little Bit Easier with PowerShell

More often than not, I avoid the Quarantine grid view entirely and move to PowerShell so I can efficiently execute bulk actions. The Quarantine cmdlets are available using the Exchange Online PowerShell V2 module. They support modern authentication, and feature well-documented examples:

Delete-QuarantineMessage
Export-QuarantineMessage
Get-QuarantineMessage
Preview-QuarantineMessage
Release-QuarantineMessage

Here’s a quick example of the cmdlets in action, displaying the first 1000 unreleased messages for a scoped domain:

Get-QuarantineMessage -pagesize 1000 | where {$_.SenderAddress -like 
"*@*.practical365.com" -and $_.ReleaseStatus -eq "NOTRELEASED"}

If I trust this domain, I can combine a bulk release action. Note that I would not do this idly, but only once I understand the reasons why all email from a particular domain would be blocked:

Get-QuarantineMessage -pagesize 1000 | where {$_.SenderAddress -like 
"*@*.practical365.com" -and $_.ReleaseStatus -eq "NOTRELEASED"}
| Release-QuarantineMessage -ReleaseToAll

Solutions for Both Users and Admins

To minimize user frustrations and pain points, admins have a couple of not-so-great options. For one, they could allow more spam and low confidence phish emails to flow into spam folders, have users deal with them there. Chances are Outlook will also classify the item as spam as it flows into the mailbox and with that method, users won’t experience two different delays, but their mailbox will be more cluttered with spam.

Another option is to train users on the Microsoft 365 Security portal, which would hopefully enable the user to comb through their quarantine queue for items that appear to be missing. While a more tech-savvy user is adept enough to successfully navigate the portal experience, support staff should still expect frustration from many users.

Conclusion

Microsoft Exchange Online, like its on-premises predecessor Microsoft Exchange Server, tends to function pretty smoothly when built and configured according to best practices. However, some aspects of the Exchange Online user experience are less than pleasing, sometimes downright painful, and the admin experience around email quarantine definitely falls into the latter category.

Not to underscore the need for Quarantine, because a security net that filters out potential phishing emails is a necessary business function – I just personally find the Office 365 experience confusing and unnecessarily technical. A big part of the problem is the fact that the Quarantine portal is also being utilized by normal users who are not accustomed to navigating such a technical UI, which is intended more for seasoned admins.

Compared to all the other vendors in this crowded space, the Exchange Online Protection Quarantine experience is sub-par and disappointing. Actions are slow and often time out, especially during bulk operations. The user experience is complex and not performant at all. Administrators have powerful options but are often hamstrung by the same slow portal experiences which users suffer under.

This is an area which requires some attention and a significant investment from Microsoft, both for product performance reasons as well as obvious flaws with the user experience. Improving the user and administrator experience in the Quarantine portal, as well as streamlining the release workflow, would be changes well overdue and welcomed by all.