Backups, Disaster Recovery, and High Availability

Build the mental model

Production systems fail in ways local dev never has to consider: a server dies, a database corrupts, a region goes down, or credentials leak. Preparing for that is the job, not pessimism.

High availability removes single points of failure before they fail: multiple instances, redundant databases with failover, traffic routed away from anything unhealthy.

Rolling back a bad deploy is the fastest recovery — reach for it immediately rather than forward-fixing under pressure. See CI/CD's deployment-strategies lessons for blue-green/canary mechanics.

RPO

Recovery Point Objective — the maximum acceptable amount of data loss, measured in time.

RTO

Recovery Time Objective — the maximum acceptable time to restore the system after a failure.

High Availability

System design that removes single points of failure so the service keeps running through individual component failures.

BACKUP-RESTORE CYCLE AND HIGH AVAILABILITY
------------------------------------------
BACKUP-RESTORE CYCLE AND HIGH AVAILABILITY
----------------------------------------------

BACKUP CYCLE                    HIGH AVAILABILITY
--------------                  -------------------
[ LIVE DATA ]                     [ LOAD BALANCER ]
      |                             /      |     \
      | backup (every X min)       v       v      v
      v                       [inst-1] [inst-2] [inst-3]
 [ BACKUP STORE ]                  X  (one fails)
      |
      | disaster happens           traffic still flows
      v                            through inst-2, inst-3
 [ RESTORE ]  <-- test this          (no single point
      |            regularly!         of failure)
      v
[ DATA IS BACK ]

Connect it to a real scenario

Meeting RPO is one comparison: is the backup interval less than or equal to the RPO target?

Real RPO planning builds in slack rather than targeting the boundary precisely, because a backup job that runs a few minutes late is common.

• RPO sets how much data loss is tolerable (drives backup frequency); RTO sets how fast the system must recover (drives outage response).
• An untested backup is an unverified guess — restoring it regularly is part of a real backup plan, not optional.
• High availability removes single points of failure before they fail; rollback is the fastest recovery from a bad deploy.

Try the working example

function meetsRpoTarget(backupIntervalMinutes, rpoTargetMinutes) {
  const meetsTarget = backupIntervalMinutes <= rpoTargetMinutes;
  return {
    backupIntervalMinutes,
    rpoTargetMinutes,
    worstCaseDataLossMinutes: backupIntervalMinutes,
    meetsTarget,
  };
}

const scenarios = [
  { label: "daily backups, RPO 1 hour", interval: 24 * 60, rpo: 60 },
  { label: "every 15 min, RPO 1 hour", interval: 15, rpo: 60 },
  { label: "hourly backups, RPO 1 hour", interval: 60, rpo: 60 },
];

for (const s of scenarios) {
  const result = meetsRpoTarget(s.interval, s.rpo);
  console.log(`${s.label}: ${JSON.stringify(result)}`);
}

daily backups, RPO 1 hour: {"backupIntervalMinutes":1440,"rpoTargetMinutes":60,"worstCaseDataLossMinutes":1440,"meetsTarget":false}
every 15 min, RPO 1 hour: {"backupIntervalMinutes":15,"rpoTargetMinutes":60,"worstCaseDataLossMinutes":15,"meetsTarget":true}
hourly backups, RPO 1 hour: {"backupIntervalMinutes":60,"rpoTargetMinutes":60,"worstCaseDataLossMinutes":60,"meetsTarget":true}

AWS Well-Architected — Reliability pillar (RPO/RTO) — Cloud & Deployment