Resolving the Five Most Common Conditional Access Misconfigurations

Avoid Common Problems and Control Access Efficiently

Successful implementation of Conditional Access policies is far from simple. A complex interface mixed with balancing organizational requirements makes it quite easy to misconfigure policies. Angry end users often catch policies that are too restrictive but it’s the policies that are too permissive that go unnoticed and introduce security concerns.

Microsoft states, “Conditional Access policies at their simplest are if-then statements.” Anyone with programming experience recognizes the if-then statement as a foundational concept. It’s necessary, it’s polished, and it’s easy. That is, until the AND/OR operators are added, NOT starts to appear, and nested statements are introduced.

This article discusses how to implement MFA effectively through the deployment of Conditional Access policies. It exposes possible holes when implementing location and device platform conditions. Followed by an explanation of how to implement Risk policies and clarification on the difference between when to use a grant vs. block access control. Next, an explanation of how policies interact in an additive way. Unlike a firewall, there is no explicit deny rule. Finally, we explore several crucial tools when evaluating a Conditional Access policy’s effectiveness.

Legacy Authentication and MFA

Legacy (basic) Authentication has been “going away” for the last three years. Originally targeting the deprecation of basic authentication for Exchange Online mail connectivity protocols in 2020, Microsoft continually pushed the date back. In January of this year, Microsoft officially disabled basic authentication for all tenants without a specific exception. 

That said, blocking legacy authentication is still a Conditional Access Policy that every tenant should deploy. One of the problems with legacy authentication is that it doesn’t support MFA and is, therefore, susceptible to a number of attacks, such as password spraying. Because of this, I’ve seen some organizations create Conditional Access policies that enforce MFA only for modern client apps. The thought process is that only modern apps can utilize MFA, and therefore there is no need to select legacy authentication clients. Unfortunately, in Conditional Access policies, unselected does not equal disabled. Instead, undefined settings are not applicable. The best method to enforce MFA is to leave client apps “not configured” or select all client apps.

The preferred method to block legacy authentication is to create a policy that blocks all authentication using legacy applications (figure 1).

Conditions: Location

Now that we have a policy to control MFA and legacy authentication, let’s dig into everyone’s favorite condition. In my experience, Named Locations via IP range or country are probably the most used condition when setting up Conditional Access policies. It’s likely that you have a policy configured to enforce MFA when logging in from outside the corporate network. You may even have a policy that blocks all users attempting to authenticate from countries like Russia or North Korea.

These examples highlight the two schools of thought when setting up location-based conditions. The country example blocks, or “excludes,” specific locations and therefore implicitly allows everything else. The problem with the use of exclusions is that locations can be bypassed. A VPN originating from a different country or the use of a GPS spoofer easily gets around this policy. 

The second school of thought is “inclusion,” highlighted in the earlier example of a corporate network. Explicitly naming a location adheres to an “Everything not Allowed is Forbidden” model. Paul expands on this in this article, but he is also careful to point out that defense in depth is highly important. The use of location-based policies aren’t bad; they’re simply part of the solution. Careful monitoring of user sign-in events from untrusted countries helps identify likely compromised users. The Sign-in events page allows for filtering on locations that may quickly identify these suspect login attempts.

Device Platforms (Android, Windows, macOS, etc.) suffer from a similar problem. Modifying a device’s user agent is a trivial task in any modern browser. Unless all device platforms have similar Conditional Access policies applied, it may be another opportunity for an attacker to bypass security controls.  

Microsoft provides additional conditions that make a more secure policy. User and Sign-in Risk are two underutilized conditions that assist in the defense in depth strategy. 

Conditions: Risk

While the lack of using risk conditions may not be a misconfiguration per say, but it would definitely fall into the category of a missed opportunity. These policies essentially provide an artificial intelligence-driven security operations center, which may be one reason they scare people off. The evaluation of a Low/Medium/High risk is a bit of a black box that also may be of some concern. In my opinion, these concerns are unwarranted because of the flexibility Conditional Access policies provide. 

Risk-based policies are best implemented as a standalone policy. Adding risk evaluation to an existing policy may prevent the Conditional Access Policy from triggering. Azure AD only enforces access controls when all conditions match. Another important distinction is that Conditional Access policies evaluate exactly the specified condition. Configuring a policy to Block access when the User risk is Medium will not trigger on High-risk logons unless both Medium and High are selected (Figure 2). 

You don’t need to jump right in and block authentication for any risky user. Start in the shallow end using report-only mode for simple policies that may enforce MFA when High-risk events occur. Microsoft has great content that explains risk and how access policies are applied.

Access Controls: Grant Block

Granting Block access must be one of Microsoft’s most confusing phrases. Perhaps the ambiguous naming convention was an attempt to deter use, as Microsoft provides warnings about using this powerful control. Creating policies with a mix of block and grant access combined with including or excluding users or groups makes it difficult to comprehend the intended implementation. 

One best practice in the realm of Assignments is excluding break-glass accounts from Conditional Access policies. When including/excluding users, groups, or roles, Microsoft provides real-time feedback (Figure 3) that helps prevent full tenant lockout.

Conversely, the Azure AD console will not warn about the deletion of a group included or excluded in a specific policy. It’s not until you attempt to edit a Conditional Access Policy that you receive the following error (figure 4).

Group nesting adds to this already complex configuration. Many administrators managing an identity service are familiar with concerns around group nesting. Evaluating recursive group membership is not a simple task when reviewing Conditional Access policies. Therefore, the removal of a group that appears unused may have dangerous consequences that change the behavior of applied policies.

Complexity surges when an environment has multiple policies that utilize assignments and access controls. In what order are these policies applied? In a conflict, which policy wins?

Policy Sprawl

Conditional Access policies do not function in the same way as other administrative tools. For instance, they have no precedence like Group Policies, and configuring a “deny all” policy does not evaluate access like a firewall. Azure AD evaluates every Conditional Access Policy at the time of authentication. Policies that apply to any given authentication attempt are evaluated using AND logic. 

Another way to think about how multiple Conditional Access policies interact is to think of it in two parts. For each Conditional Access Policy, ask:

• Does the policy apply?
• What controls does the policy enforce?

Before Azure AD applies a Conditional Access Policy, all assignments must match. Is the user account included? Is the cloud app, user action, or authentication context being used? Finally, are all conditions (risk, device platform, location, client apps, and device state) met?

If all the above match, only then does Azure AD enforce the controls specified in policies. Therefore, if an authentication attempt meets all assignments for multiple Conditional Access policies, then the enforced controls are all applied. For example, if one policy requires MFA and a second policy requires a device to be marked as compliant, both access controls are enforced. 

Improving: Test/Evaluate/Report

With so much opportunity to fail, how can you get this right? Several tools can help administrators improve their implementation of Conditional Access policies. The first of which is the built-in “What If” tool. This tool allows you to specify test conditions of logon events and will return a list of the Conditional Access policies that will apply as well as the reason other policies will not.

Once Conditional Access policies are applied and functioning, Sign-in logs provide great troubleshooting assistance to determine why a policy did or did not apply. Selecting a failure event displays a failure reason (figure 6). 

The same pop-out menu has a Conditional Access tab that lists information related to each policy. Selecting the three dots and choosing “Show details”… displays detailed information about matched criteria and enforced controls (figure 7 & 8). 

Azure AD PMs have been developing the IdPowerTools app that includes the Conditional Access Documenter. Once registered in your tenant, this app generates a PowerPoint presentation of all Conditional Access policies that help organizations better visualize and rationalize their policies.

Another incredible tool, described here by Sean McAvinue, provides a full assessment of Conditional Access policies. This assessment yields a comprehensive report that helps identify the misconfigurations outlined here and much more. 

The last piece of advice for improving Conditional Access policies is to use the policy templates. Microsoft currently offers several templates that provide the exact settings recommended for each task. These templates are customizable prior to implementation to fit your environment’s specific needs.

What Now?

Conditional Access policies that are misconfigured or too permissive may introduce unnecessary risk to your tenant. While it may require more than a foundational understanding of if-then statements, the good news is that Conditional Access policies do not need to be complex in order to be efficient. Using the templates as your guide and asking yourself “what if” is a good way to start.

The five examples outlined in this article should prompt you to evaluate your policies with the identified tools. Even if you’ve already implemented policies that enforce MFA, block legacy authentication, and evaluate risk, it may be worth a second look to ensure you haven’t misconfigured your Conditional Access policies.