You have backups. Nobody has ever restored one.
A backup job that reports success is not the same as a file you can restore from. The difference only becomes visible on the day it matters.
Does this sound familiar?
Almost every business we take over has backups of some kind. Far fewer have ever restored one, and a meaningful number turn out not to have what they think they have.
- Nobody can say when a backup was last restored, only when one last ran
- The backup reports success, and nobody reads the report
- Retention is whatever the provider set by default, and nobody chose it
- The backup lives on the same server, or the same account, as the thing it protects
What we do, in order
Roughly the first week. The sequence matters more than the speed.
We find out what is actually being captured
Which databases, which files, on what schedule, to where. It is common to find something excluded — a database created after the job was written, or one the backup user cannot read. A run that skips a database and reports success is the normal failure, not a dramatic one.
We restore one, for real
Into a scratch environment, from the newest backup, and we count the rows. This is the only test that means anything: everything else is a report about a file, not a check of what is inside it.
We fix what the restore exposes
Missing databases, permissions the dump role never had, retention too short to survive a problem discovered on Monday, backups stored where a compromise of the server would take them too.
We make failure loud
Nightly dumps to offsite object storage, kept 30 days, with an alert when a database stops being captured. A backup that stops working should interrupt somebody, not wait to be noticed.
How this fails quietly
Backup jobs fail one database at a time. The run still finishes, the summary still says most things worked, and the one that stopped is buried in a status file nobody opens. We found exactly this on our own fleet: a production database whose nightly dump had been refused for five consecutive nights because the backup role had lost permission to read it. Everything else in the run succeeded, so nothing ever raised its voice.
Common questions
How often should database backups be tested?
A full restore at least quarterly, and immediately after any change to the schema, the backup user or the storage target. Between those, an automated check that each expected database produced a file of a plausible size catches the common silent failure.
Why would a backup job report success but miss a database?
Because most jobs report on the run rather than on each database. A permission the dump user lacks, a database created after the job was written, or a new security policy on a table will fail that one database while the others succeed — and the overall run still finishes.
How long should backups be kept?
Long enough to survive a problem you discover late. Thirty days is a reasonable default: it covers corruption noticed weeks after it started, which a seven-day window does not.
Do backups need to be stored somewhere else?
Yes. A backup on the same server protects you from deleting a table and from nothing else — not from the disk failing, the account being compromised, or the provider suspending it. Offsite means a different account, not only a different folder.