GxP / Guide 8

Backup, disaster recovery and business continuity

Building a GxP disaster recovery program: criticality and recovery targets, immutable backups, restore testing as evidence, DR drills, business continuity and archiving for long retention.

For a GxP system, a backup that has never been restored isn't evidence of anything. Annex 11 expects backups to be checked during validation and monitored periodically, business continuity arrangements to be tested, and archived data to stay readable for its full retention period. HIPAA's contingency planning rule asks for much the same for health data. This guide describes a disaster recovery program that satisfies both, on Azure and AWS.

Start with criticality

Recovery targets come from the business, not from IT. For each GxP system, a short business impact analysis sets:

Write the targets into the system's user requirements. They become testable, and the evidence scripts can check them.

Backups that survive an attack

The threat model has changed: ransomware operators and malicious insiders delete backups first. For GxP data, backups should be protected even from your own administrators:

• Azure: Recovery Services vault with immutability enabled and locked (recovery points can't be deleted before they expire, and the setting can't be undone), enhanced soft delete always on, and multi-user authorization with a Resource Guard in a separate subscription, so reducing protection needs a second person's approval.
• AWS: AWS Backup Vault Lock in compliance mode, copies to a vault in a separate backup account (or a logically air-gapped vault), and SCPs denying deletion of recovery points outside a break-glass role.
• Both: backups in a second region for tier 1 and 2 systems, encryption with keys whose deletion is also protected, and alerts on failed jobs and on any change to backup policy.

The platform backup guides for Azure and AWS cover the setup in detail.

Restore testing is the evidence

Plan restore tests like validation tests: a documented procedure, expected results, actual results, and sign-off.

• What to check after a restore: the application starts, records from before the backup point are present and correct (compare record counts and a checksum or sample of records), audit trail entries are intact, and users can sign in with the right roles.
• How often: set by tier, for example quarterly for tier 1, twice a year for tier 2, yearly for tier 3, and after any major change to backup design.
• Where: into an isolated network, so a restored system can't talk to production interfaces or send duplicate data downstream.
• Record: how long the restore took (your real RTO) and how much data was lost (your real RPO).

On AWS, restore testing in AWS Backup restores recovery points on a schedule, runs your validation function and deletes the copy afterwards. On Azure, schedule restores to an isolated resource group with Azure Automation or a pipeline. Either way, collect the evidence with the scripts:

# Azure: vault protection, last backup per GxP item, restore jobs in the last 180 days
.\Get-AzGxPBackupEvidence.ps1 -Days 180

# AWS: Vault Lock, restore testing plans and restore jobs
.\Get-AwsGxPRestoreEvidence.ps1 -ProfileName notes-gxp -Days 180

Both flag vaults with no completed restore in the window: the first thing to fix before an inspection.

Disaster recovery drills

A regional failover is a much bigger event than a restore, and it's where plans fail: DNS, certificates, firewall rules, interfaces, licences, people who don't know the runbook. For tier 1 and 2 systems:

• Keep a DR runbook per system, change-controlled like any GxP document.
• Drill at least yearly, rotating which systems fail over. Azure Site Recovery test failover and AWS Elastic Disaster Recovery drills run into isolated networks without touching production.
• Treat the drill as a protocol: objectives, steps, acceptance criteria (met RTO and RPO, data verified, audit trail intact), deviations, and a report with actions.
• Feed findings back into the runbook, and re-test anything that failed.

Business continuity

Annex 11 §16 is about the process, not only the system: what happens on the production floor or in the lab while a system is down? For each critical system, document the manual or alternative process (paper records, a standalone instrument mode), how those records are controlled, and how data is entered into the system afterwards without losing attribution or creating duplicates. Test it, even as a tabletop exercise.

Archiving for long retention

Retention periods for GxP records are long: in the EU, batch documentation for at least one year after batch expiry or five years after certification, whichever is longer; clinical trial master files for at least 25 years under the EU Clinical Trials Regulation. Check your predicate rules and your own retention schedule. Backups are not archives:

• Archive in a readable, open format where you can (PDF/A for documents, CSV or XML with a data dictionary for structured data), together with the audit trail and metadata that give the data its meaning.
• Protect the archive: immutable storage (Azure blob immutability, S3 Object Lock in compliance mode), with checksums recorded at the time of archiving.
• Verify periodically that archived data can be found, opened and read, and that checksums still match, especially after any platform or format change (Annex 11 §17).
• Plan for system retirement: when a system is decommissioned, its records and audit trails must stay available and readable for the rest of their retention period.

If you also handle health data

The HIPAA Security Rule's contingency plan standard (45 CFR 164.308(a)(7)) requires a data backup plan, a disaster recovery plan and an emergency mode operation plan, and lists testing and revision procedures and an applications and data criticality analysis as addressable specifications. The program above covers all of them: the criticality tiers are your criticality analysis, and the restore tests and DR drills are your testing. One program, two sets of evidence.