Datacenter Activation Coordination mode (DAC mode) is a feature of Exchange database availability groups that is designed to prevent split brain scenarios.
Within a database availability group (DAG) each database can have one active copy, and up to 15 passive copies at any given time. Changes that occur in the active database copy are replicated to the passive copies by a process of continuous replication. The active copy can be dismounted, and one of the passive copies mounted to become the active copy, when a switchover or failover occurs.
In a split brain scenario, two copies of a database would be active at the same time, mounted on DAG members that are unable to communicate with each other due to a network problem, causing the databases to diverge. This is a situation that must be avoided, because it creates a difficult recovery scenario and will likely result in data loss.
DAC mode enables a protocol called Datacenter Activation Coordination Protocol (DACP). You can read an example of how DAC mode and DACP work to avoid split brains in this article.
The recommended practice for Exchange Server DAGs is to enable DAC mode if:
- The DAG has more than one member (DAGs start with zero members, and can have one member)
- Third party replication mode is not enabled, or the third party replication vendor specifies that DAC mode can be used
In addition to preventing split brains, DAC mode enables the use of Exchange site-resilience cmdlets (Stop-DatabaseAvailabilityGroup, Restore-DatabaseAvailabilityGroup, and Start-DatabaseAvailabilityGroup). Those cmdlets are used perform datacenter switchovers.
To review the DAC mode configuration for a DAG, use the Get-DatabaseAvailabilityGroup cmdlet.
[PS] C:\>Get-DatabaseAvailabilityGroup | Select Name,Servers,Datace*
Name Servers DatacenterActivationMode
---- ------- ------------------------
EX2016DAG01 {EX2016SRV2, EX2016SRV1} Off
EX2010DAG {EX2010SRV1} Off
To enable DAC mode, use the Set-DatabaseAvailabilityGroup cmdlet.
[PS] C:\>Set-DatabaseAvailabilityGroup EX2016DAG01 -DatacenterActivationMode DagOnly
[PS] C:\>Get-DatabaseAvailabilityGroup | Select Name,Servers,Datace*
Name Servers DatacenterActivationMode
---- ------- ------------------------
EX2016DAG01 {EX2016SRV2, EX2016SRV1} DagOnly
EX2010DAG {EX2010SRV1} Off
DAC mode can be enabled for DAGs that exist within a single datacenter location. A multi-site DAG is not required. Although less likely, a split brain can still occur for DAGs within a single datacenter location under the right conditions.
Hello Paul,
What is the difference between
Set-DatabaseAvailabilityGroup EXCHDAG01 -DatacenterActivationMode DagOnly
and
Set-DatabaseAvailabilityGroup EXCHDAG01 -DatacenterActivationMode On
Yes, I didn’t displayed it’s full name. This is a Win2008 R2 server in our domain running some apps. It’s not part of the Exch 2013 environment.
Sorry, didn’t display…. 🙁
Paul, we were trying to setup our DAC..
“Set-DatabaseAvailabilityGroup -Identity EX-DAG01 -DatacenterActivationMode DagOnly”
and got this warning…
File share already exists, however directory is different from witness directory. Until this problem is corrected, the database availability group may have increased vulnerability to failures. To correct this problem, delete existing the file share and use the Set-DatabaseAvailabilityGroup cmdlet to retry the operation.
Our file witness is at “\\mail\dagfsw\” not an Exch. server.
Do we need to delete the contens of the dir above and re-run the cmdlet?
Is that what they mean?
Please let us know.
Thx.
NRC
What is “mail”, is that a server name in your org?
Would like to know if there is a chance for fallout to enabling DAC mode on a production system?
All change carries risk. Plan accordingly. Test if necessary. Understand what change you’re making and how to roll it back if needed.
Other than that, I consider it a safe change.
Thank you for the input.
Hi Paul,
We’ve been struggling with a DAG configuration, the members are in a primary datacenter with 2 servers in 1 network and a DR site in a different network.
The FSW was configured in a 3rd location, however if I’m right, it should not be doing anything in this configuration, since we have quorum.
We have the DAC configured as Dagonly in order to avoid splitbrain.
We performed a DR Test and the problem was that when we cut over the network of the primary site (with 2 servers), the DR site would not come up and the databases went to unknown state which did not allow me to run any command on them.
I ran this same test in a lab and it worked, the only difference is that I have only 1 network and I just shutdown the servers to replicate the issue.
We are thinking it might be a clustering issue at OS level.
Any advice on this?
Thanks a lot.
How many members does this DAG have?
Hello Paul,
we have 2 exchange 2013 server in production site and also we are planing for 2 exchange server in DR site.
Both production and DR exchange servers will be member of same DAG.
in this scenario how many witness server required and how it will be configured . please note that we dont have any tertiary data center.
we are looking for automatic failover . is there any workaround for achieving this solution without public cloud or third party data center . what about exchange 2016 ? if we use dynamic DAG, with single witness server will the automatic fail over work? .
we are looking for fully automated process without any manual intervention.
The simple answer is, without a third datacenter you can’t have automatic site failover.
Thanks Paul,
just wanted to know, we are planing to implement GEO load balance for the same purpose and keeping witness server in third location also. see in this scenario how the mx record pointing will be configured. currently we have 2 records mail.xxxx.com(pointing to primary site exch.) and mail2.xxxx.com(pointing to DR site exch.) once we implement GEO do we need to keep same record or single name will do the job?. if you could tell me the required DNS records and public IP for this scenario that would be great.
Simplest option is just to have two separate MX records, one for each datacenter.
Good work Paul appreciated, please let me know only one thing, what’s about datacenter failover? DAC will also work there? we should enable the DAC in case of switch datacenter or not?
DAC prevents split brain in scenarios where the two datacenters lose connectivity.
If you want to do a datacenter switchover, search for the TechNet documentation that explains the steps. It is very well documented there.
Hi Paul, great post. You stated that, although unlikely, a split brain can still occur for DAGs within a single datacenter location under the right conditions. Can you elaborate /share an example out of curiosity?
All you need to do is imagine a scenario where a networking problem within a datacenter creates the same connectivity problem as a WAN failure would.
Pingback: Alternative Witness Server for Exchange 2013 DAG – A random blog from a sysadmin
Paul,
When it comes to DAC mode and an even number of members, is the FSW required to be up and running in order to mount the databases assuming all DAG members go offline ?
Studying your pluralsight videos.
If all DAG members go offline there is nothing left to mount the databases. I suspect I am misunderstanding your question.
Sorry Paul let me be more clear. I am saying this:
If you have a 4 member DAG and all servers go offline then once you bring them backup – is the FSW required to be online as well for Quorum to be attained ?
Meaning in your 4 member dag, do you need all 4 dag members online and the FSW online at the same time for the DAG to get quorum and exchange to mount the databases?
Robert
Paul,
I’m confused now in two datacenters on different geolocations. We should Enable DAC or not and why?
And what about Alternate File Share Witness?
I appreciate your reply.
Thanks..
Paul, I appreciate your reply.
What did he tell you? i have a very similar situation.
2 datacenters, 2 Ex2013’s in both, all in 1 dag.
What is the best way to take care if one site was to go down.
Paul,
This is great info. Thanks very much.
I think it’d be very helpful if you included a link to this post in your 2013 DAG setup/info posts. I did not know about DAC until today and would’ve caught it earlier through your DAG posts.
Thanks again!
So recommendation is use dac when going dag ? not by the fact if using two datacenter on different geolocations?
We are going to build a 10 node dag over 2 high speed low latency datacenter with FSW in azure.
Following your article we will configure dac