Continuity and Recovery: The Restore Nobody Tested

Every backup dashboard shows green checkmarks. Jobs completed, retention policy satisfied, storage utilized as expected. None of that tells you whether you can bring a domain controller back from bare metal in under four hours during an actual incident. The gap between "backup succeeded" and "restore succeeded" is where most continuity plans quietly fail.

Why the checkmark lies

Backup software reports success when it finishes writing bytes to a destination. It doesn't know if those bytes are usable. A SQL Server backup can complete cleanly and still be unrestorable because the transaction log chain broke three days earlier and nobody noticed. VM snapshots can look fine in the console while the underlying VSS writer silently failed inside the guest OS, producing a crash-consistent (not application-consistent) image.

Ransomware operators know this. Groups running Conti-style playbooks in past incidents deliberately targeted backup infrastructure — deleting shadow copies with vssadmin delete shadows /all /quiet, disabling Veeam repositories, encrypting NAS-based backup targets that were reachable over SMB. If your backups live on the same network segment as production with domain credentials that can touch them, they are a target, not a safety net.

Build a restore runbook, not a backup policy

A continuity plan needs step-by-step restore instructions written for someone who isn't the person who normally does it. Write down:

• Exact recovery order (domain controllers and DNS first, then core apps, then everything else)
• Where credentials for the backup console live if your password vault is also down
• The specific restore command or console path, not "use Veeam to restore the VM"
• Expected duration per system, based on actual measured tests, not vendor marketing numbers

For Veeam Backup & Replication, that means documenting the actual steps: open the console, navigate to Backups > Disk, right-click the restore point, choose Instant VM Recovery or Full VM Restore depending on scenario, and select the target host with enough free capacity. If your primary host is also compromised, you need a second, isolated host already identified and licensed.

Test restores on a schedule, not a whim

Pick a rotation. Every month, restore one critical system to an isolated VLAN and validate it boots, authenticates, and serves data correctly. Every quarter, run a full-scope test: restore your domain controller, your file server, and your primary database to isolated infrastructure, then have someone outside the backup team try to log in and pull a report.

For databases, don't just restore the .bak file — verify it:

RESTORE VERIFYONLY FROM DISK = 'D:\Backups\prod_2024.bak'

Then actually restore it to a test instance and run DBCC CHECKDB against it. A backup that passes VERIFYONLY can still contain logical corruption that only shows up when you query it.

For Linux systems using something like Bacula or restic, test the actual restore path:

restic restore latest --target /tmp/restore-test --repo /mnt/backup-repo

Then diff the restored config files against production to confirm nothing silently dropped.

Immutable copies and the 3-2-1-1 rule

The classic 3-2-1 rule (three copies, two media types, one offsite) needs an update for the ransomware era: 3-2-1-1, where the extra "1" is an immutable or air-gapped copy. Object lock on S3-compatible storage (Wasabi, Backblaze B2, or AWS S3 with Object Lock enabled) prevents deletion or modification for a defined retention window, even by an account with admin credentials. Configure it with:

aws s3api put-object-lock-configuration \
  --bucket backup-vault \
  --object-lock-configuration '{"ObjectLockEnabled":"Enabled","Rule":{"DefaultRetention":{"Mode":"COMPLIANCE","Days":30}}}'

COMPLIANCE mode means nobody, including the root account, can shorten retention or delete objects early. That matters when the attacker has domain admin.

Measure RTO and RPO with real numbers, not guesses

Recovery Time Objective and Recovery Point Objective sound like paperwork exercises until an executive asks "how much data do we lose and how long are we down." Time your last three test restores. If your RPO target is one hour but your backup job only runs every six, you have a documented gap, and it's better to find that gap in a tabletop exercise than during an actual encryption event at 2 a.m. on a Saturday.

Run the test, write down the actual clock time, and compare it to what you promised in the disaster recovery document. The difference between those two numbers is the real state of your continuity plan.

For more on hardening the systems you're protecting and building out incident response workflows, check the related Blue Team and Digital Forensics segments on Korra Studio.