Database availability groups are reasonably smart and robust systems, but they do need monitoring, or else you might find one day they are not providing the HA that you need them to. One of the useful PowerShell cmdlets for keeping an eye on things is Test-ReplicationHealth. It can be used to help troubleshoot database availability groups by running it locally or against a remote server.

[PS] C:\>Test-ReplicationHealth

Server          Check                      Result     Error
------          -----                      ------     -----
E15MB2          ClusterService             Passed
E15MB2          ReplayService              Passed
E15MB2          ActiveManager              Passed
E15MB2          TasksRpcListener           Passed
E15MB2          TcpListener                Passed
E15MB2          ServerLocatorService       Passed
E15MB2          DagMembersUp               Passed
E15MB2          ClusterNetwork             Passed
E15MB2          QuorumGroup                Passed
E15MB2          DatabaseRedundancy         Passed
E15MB2          DatabaseAvailability       Passed
E15MB2          DBCopySuspended            Passed
E15MB2          DBCopyFailed               Passed
E15MB2          DBInitializing             Passed
E15MB2          DBDisconnected             Passed
E15MB2          DBLogCopyKeepingUp         Passed
E15MB2          DBLogReplayKeepingUp       Passed

To test a remote server, use the -Identity parameter.

[PS] C:\>Test-ReplicationHealth -Identity E15MB2

Server          Check                      Result     Error
------          -----                      ------     -----
E15MB2          ClusterService             Passed
E15MB2          ReplayService              Passed
E15MB2          ActiveManager              Passed
E15MB2          TasksRpcListener           Passed
E15MB2          TcpListener                Passed
E15MB2          ServerLocatorService       Passed
E15MB2          DagMembersUp               Passed
E15MB2          ClusterNetwork             Passed
E15MB2          QuorumGroup                Passed
E15MB2          DatabaseRedundancy         Passed
E15MB2          DatabaseAvailability       Passed

The cmdlet also accepts pipeline input, however if you were to simply pipe Get-MailboxServer into it and you have Mailbox servers in the organization that are not DAG members then you risk seeing errors in your results that just get in the way. Instead you can pipe only the members of a database availability group into Test-ReplicationHealth using the following method:

[PS] C:\>Get-DatabaseAvailabilityGroup | select -ExpandProperty:Servers | Test-ReplicationHealth
The number of tests shown in the output of Test-ReplicationHealth will vary depending on which version of Exchange you’re running, and whether the DAG member has only active or passive databases on it at the time. Not all tests are relevant if the server has only active, or only passive databases on it.

There’s a lot of output to look at, so it’s often easier to filter the results to only those that have not passed.

[PS] C:\>Get-DatabaseAvailabilityGroup | Select -ExpandProperty:Servers | Test-ReplicationHealth | Where {$_.Result.Valu
e -ne "Passed"}

Server          Check                      Result     Error
------          -----                      ------     -----
EX2016SRV2      DatabaseAvailability       *FAILED*   Failures:...
EX2016SRV1      DatabaseAvailability       *FAILED*   Failures:...

The error details are usually truncated, so outputting the results as a list will let you see more information.

[PS] C:\>Get-DatabaseAvailabilityGroup | Select -ExpandProperty:Servers | Test-ReplicationHealth | Where {$_.Result.Val
e -ne "Passed"} | Format-List

By the way, if you’re curious about what each of the tests do, there’s a “CheckDescription” provided for each of the checks that Test-ReplicationHealth performs. Some of them are still a bit vague, but there’s some useful info there that can help point you in the right direction to investigate further.

[PS] C:\>Test-ReplicationHealth | Format-List Check*


Check            : ClusterService
CheckDescription : Checks if the cluster service is healthy.

Check            : ReplayService
CheckDescription : Checks if the Microsoft Exchange Replication service is running.

Check            : ActiveManager
CheckDescription : Checks that Active Manager is running and has a valid role.

Check            : TasksRpcListener
CheckDescription : Checks that the Tasks RPC Listener is running and is responding to remote requests.

Check            : TcpListener
CheckDescription : Checks that the TCP Listener is running and is responding to requests.

Check            : ServerLocatorService
CheckDescription : Checks that the Server Locator Service is running and is responding to requests.

Check            : DagMembersUp
CheckDescription : Verifies that the members of a database availability group are up and running.

Check            : MonitoringService
CheckDescription : Checks that the Monitoring WCF Service is running and is responding to requests. This endpoint is
                   hosted inside the DAG Management service.

Check            : ClusterNetwork
CheckDescription : Checks that the networks are healthy.

Check            : QuorumGroup
CheckDescription : Checks that the quorum and witness for the database availability group is healthy.

Check            : FileShareQuorum
CheckDescription : Verifies that the path used for the file share witness can be reached.

Check            : DatabaseRedundancy
CheckDescription : Verifies that databases have sufficient redundancy. If this check fails, it means that some
                   databases are at risk of losing data.

Check            : DatabaseAvailability
CheckDescription : Verifies that databases have sufficient availability. If this check fails, it means that some
                   databases are at risk of losing service.

[adrotate banner=”48″]

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. Salim

    HI Paul,

    I hope you’ll find time to response for my request.
    I’ve a reccurent error under exchange 2013 : “OUTLOOK.PROTOCOL UNHEALTHY”.
    I made many research with no success to find an issue with.

    I hope you’ll have ones

    Brgds

  2. Nien

    Hi Paul, Thank you for the time you take to write these articles. Not only are they clear, but also help with getting a better understanding of the PowerShell cmdlets.
    Nien.

  3. Simon Craner

    Hi Paul,

    thanks once again for excellent work. Just like the test-exchangehealth script, I would like to have this output to an email that my team could get say twice a day, i.e. at the typical working start time of the day and before we go home for the day.
    Any assistance to help generate it will be greatly appreciated.

      1. Angga

        How about the “Database Availability Group Member Health” column like:
        Cluster Service
        Replay Service
        Active Manager
        Tasks RPC Listener
        TCP Listener
        Server Locator Service
        DAG Members Up
        Cluster Network Quorum Group
        File Share Quorum
        Database Redundancy
        Database Availability
        DB Copy Suspended
        DB Copy Filed
        DB Initializing
        DB Disconnected
        DB Log Copy Keeping Up
        DB Log Replay Keeping Up
        the result is n/a, even though when I check manually passed

  4. Miko Tupaz

    Hi Paul,
    Would you happen to have the firewall ports required for this to work? We have an exchange site behind a firewall and not getting any results when I run this test.

  5. Ryan

    Hi Paul, what are the next steps if you get a failure from Test-ReplicationHealth? We are running Exchange 2010 and despite seeing no ill effects all 12 members of our three DAG’s are now failing with:

    The health test of the Microsoft Exchange Server Locator Service on server ‘DAG-MEMBER’ failed.
    Error: Server Locator Service call had a communication error.

    Searching the for this error it does’t seem to appear in any MS documentation so we’re at a bit of a loss of how to proceed before contacting MS.

    Many thanks, Ryan.

    1. Avatar photo
      Paul Cunningham

      According to TechNet that…

      “Verifies the Active Manager client/server processes on DAG members and on the Client Access Server that perform lookups in Active Directory and Active Manager to determine where a user’s mailbox database is active.”

      I don’t see any info out there on what to do about it. Are there any other event log entries that might be relevant? Particularly events that may be logged when the server restarts?

      1. Ryan

        Thanks for the advice Paul, I tried rebooting a DAG member but couldn’t see anything useful so investigated further…

        This issue only came to light because we run Test-ReplicationHealth for each node daily and email an alert on failure. The Server Locator Service error first appeared last month and then disappeared before returning last week. It would always affect all servers or none.

        We are in the process of updating from SP2 to SP3 and it seems the Test-ReplicationHealth results vary depending on where you run the cmdlet. If run on SP3 we don’t see ClusterNetwork, QuorumGroup and FileShareQuorum checks but gain the ServerLocatorService failure.

        The server running the daily health checks would connect to a different Exchange server each day thus we got differing results. So far we’ve only upgraded our CAS and HT servers so hopefully once we complete the mailbox servers the ServiceLocatorService will report healthy.

        Thanks again for initial advice though Paul 🙂

        1. Avatar photo
          Paul Cunningham

          I suppose that would make sense that running from a down level server might throw some odd results for higher SP level servers. Though I’ve never run into that issue myself, because I generally run monitoring scripts from the highest version/SP level in the org.

Leave a Reply