Backups exist and a restore has been tested
The data store is backed up automatically, and the team has restored from a backup recently enough to trust it.
HighChecked by hand
- Applies to:
- Services that own a data store
- Rule id:
- data.backups-restore-tested
Why it matters
Nobody needs a backup; they need a restore. Backups fail quietly: the job stopped months ago, the files are encrypted with a lost key, the restore takes two days when the business can stand two hours. The only proof is a restore that worked, and the only time to find out is before the incident.
How to apply it
- Back up automatically, and keep copies out of reach of the service's own credentials and region.
- State how much data you may lose (RPO) and how long recovery may take (RTO), and check the backup schedule meets them.
- Restore into a scratch environment on a schedule, and check the data is usable.
- Time the restore and write the steps in the runbook.
- Use point-in-time recovery where the store offers it: most data loss is a bad deploy or a bad query, not a dead disk.
- Alert when a backup fails or is older than expected.
Example
Postgres takes continuous WAL archives and a daily snapshot, kept 35 days in another region. RPO 5 minutes, RTO 1 hour. A monthly job restores the latest snapshot to a scratch instance and runs row-count checks; the last run took 22 minutes.
References
Checklists with this rule
Related agent skill: database-per-service