Learn Active Directory - Active Directory Disaster Recovery

#Introduction

In episode 25 we moved an organization's identities to a new structure. After the migration finishes and everything runs, the most uncomfortable question arises: what happens if your DCs die — or worse, if the entire forest collapses?

Disaster recovery (DR) isn't about great backup tools. DR is the honest answer to two numbers: how fast you can get back to operations, and how much data you're willing to sacrifice. This episode dissects how to answer both, then build procedures that can be executed under pressure.

#DR Planning: RTO and RPO

Before thinking about tools, set the targets. These two metrics become the compass for the entire DR design:

• RTO (Recovery Time Objective) — how long the service may be down before it's restored.
• RPO (Recovery Point Objective) — how much data loss is still acceptable.

RTO and RPO numbers aren't purely technical decisions — they're determined by business needs. A helpdesk down for 8 hours might cause complaints; a production system down for 8 hours could mean major losses.

#System State Backup

System state on a DC contains everything needed to restore its role: registry, SYSVOL, the AD database (ntds.dit) with its logs, boot files, and AD CS data if installed on the same DC. This isn't a full server backup — it's the backup of a DC's "soul".

Install Windows Server Backup then schedule a daily system state backup:

Install-WindowsFeature Windows-Server-Backup -IncludeManagementTools

wbadmin start systemstatebackup -backupTarget:E: -quiet

Apply the 3-2-1 rule: three copies of the data, on two different media, with one stored offsite. A DC backup stored on the same disk as the DC itself isn't a backup — it's destroyed along with the disk when it dies.

#Restoring a Single DC

First priority for any problematic DC: are other DCs in the domain still healthy? If so, the best approach is rebuild and re-promote — discard the broken DC, run metadata cleanup, then promote a new DC. This is faster and safer than a restore.

A restore is only needed when all DCs are down or the damaged DC is the last one. The process:

bcdedit /set safeboot dsrepair
restart
wbadmin start systemstaterecovery -version:07/20/2026-08:00 -quiet
bcdedit /deletevalue safeboot
restart

Get the -version string from wbadmin get versions. After the restore, the DC comes back online and absorbs changes from other DCs — this is called a non-authoritative restore.

#Non-Authoritative vs Authoritative

• Non-authoritative (normal): the DC accepts restored data then gets updated by newer DCs. Suitable for recovering one lost DC.
• Authoritative: restored objects win over other DCs — used when objects were deleted and you want them back. Run via ntdsutil after a non-authoritative restore:

ntdsutil
activate instance ntds
authoritative restore
restore subtree "OU=Karyawan,DC=corp,DC=local"
quit
quit

For deleted objects, the Active Directory Recycle Bin (episode 17) is a far simpler modern alternative — provided the feature was enabled before the disaster.

#The Danger of USN Rollback

This is the most dangerous DR mistake. Imagine: a DC keeps running and receiving replication, then someone restores an old backup on that DC. That backup carries Update Sequence Numbers (USNs) long left behind. When the DC comes back online, it offers "new" changes to other DCs — when they're actually old, already-deleted data. Other DCs accept it and up-to-date data disappears from the forest.

The prevention is simple: never restore a backup of a DC that recently came online and replicated. If the DC was already connected and replicating, it's safer to discard it and promote a new DC. After any restore, verify with repadmin /showrepl and dcdiag before trusting that DC.

#Forest Recovery: When Everything Falls

Forest recovery handles the worst-case scenario: many DCs dying at once, or the available backups not being in sync. Microsoft's official procedure follows this pattern:

1. Identify the scope. How many DCs fell, how much data was lost, and whether any DC survived and can be trusted.
2. Isolate. Don't let other DCs come online and start replicating with stale data.
3. Choose the recovery DC. Pick one DC with the newest and most complete backup, restore it non-authoritatively, then bring it up as the forest "seed".
4. Clean up metadata. Remove metadata of DCs that won't be restored via ntdsutil metadata cleanup.
5. Rebuild. Promote the other new DCs and let them replicate from the recovery DC.
6. Verify. Make sure SYSVOL (DFSR), FSMO roles, DNS, and replication are back to normal.

The whole procedure must be written down and already exercised in the lab — not read from an article while the fire is burning.

#Building a DR Site and Testing the Plan

#DR Site

A second location with its own DC is the best way to handle the "one site lost" scenario. Some considerations:

• Place a writable DC at the DR site, not just an RODC — an RODC can't write if the WAN is down.
• Design site links and costs so authentication stays fast when traffic is rerouted (episodes 3 and 27).
• Consider virtual DCs in the cloud (Azure, AWS) as a DR option with lower TCO.

#Restore Test

DR's golden rule: if a backup was never restored, it can't be considered functional. Schedule periodic restore tests — e.g. every quarter — in an isolated lab environment: restore system state, boot it, run dcdiag, and record the results. Measure the duration against the RTO and the volume of restored data against the RPO. A failure in the lab is far cheaper than a failure in production.

#Backup Data Security

An AD backup is the source of truth for identity — and therefore also an attacker's target. An attacker who controls a DC backup can extract hashes of all accounts without ever touching the network. Therefore:

• Encrypt backup media and restrict access to backup systems.
• Store an offsite copy disconnected from the main network.
• Protect the DSRM password with a strict secret management process (the secret management series).
• Document the DR runbook: tested restore steps, contact list, and the location of all recovery passwords.

#Conclusion

In episode 26 you built an honest DR foundation: setting RTO and RPO from business needs, daily system state backup with the 3-2-1 rule, single DC recovery via rebuild or restore, understanding non-authoritative vs authoritative, avoiding USN rollback, the forest recovery procedure, a DR site, periodic restore tests, and backup data security.

Key points:

• RTO and RPO are set by business, not technicians.
• Rebuild is faster and safer than restore when other DCs are still healthy.
• Never restore a backup of a DC that was already replicating.
• A backup that was never tested is just hope.

Once you're confident you can survive a disaster, the time comes to answer: how do you design healthy AD for a giant organization with millions of objects? In episode 27 we dissect Active Directory at Scale: multi-domain and multi-forest, Global Catalog, site design, RODCs for branches, and best practices for many DCs. See you there!