Resilience as a Claim: Restore Tests, Game Days and Evidence Age

All courses / AI Operations / Module 5 / Chapter 19

Learning objectives

After the lesson the participant can:

Write a recoverability claim with its six parts for one service and scenario
Derive the status of a claim from its latest record, its clock and listed changes
Plan a restore test that measures recovery time and recovery point with an independent probe in a segregated environment
Explain why AI may plan, execute bounded steps and analyse a test, and may not attest

Key concepts

Recoverability claim: A statement with six parts: the subject, the scenario, the objective, the measured result, the date and the expiry. A statement that lacks one is an assertion, and the claim states which backup copy was restored.
Derived status: The status of a claim is computed from its record, never typed by hand, into verified, expired, implemented but unverified, failed, asserted or untested. Only a verified claim counts as recoverable, and the denominator is every claim the organisation makes.
Expiry by clock and by condition: A result stops counting after its clock, 180 days by default for a Tier 1 restore or failover test, whatever has changed, and earlier on a named change written into the record, such as a new major database version or a change of backup storage or provider.
Game day record: A record for an authorised experiment with a hypothesis of steady state that can fail, a bounded blast radius, an abort rule and a stop path outside the system under test. The experiment is refused if a field is missing.
Rebuild from declared state: A recovery path with a claim of its own in an estate run by state engines: it restores the declaration and not the data, secrets, external state or manual drift, and records the commit, engine version, state and key copies, manual steps and time to a passing probe.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Sort

Deriving the status of a claim

Sort each situation by the status that the chapter's rule derives for the recoverability claim.

Interactive exercise 2 · Match

What each kind of test proves

Match each kind of restore test with what, in the chapter's table, it proves.

Teaching plan

15 min	The teacher rewrites the sentence the database can be restored as a six-part claim and introduces the status rule with the register view of both organisations in September 2028.
20 min	Groups read the GitLab 2017 and Atlassian 2022 cases as summarised in the chapter and name which part of the claim was missing or which scope was never tested.
15 min	Presentation of what a restore test measures, the kinds of test with what each does not prove, and the boundary between game days and threat-led penetration testing.
10 min	Input on the four places where AI takes part in the register and the rule that ties autonomy to evidence.
35 min	Exercise: participants write a claim and plan the restore test that would support it.
10 min	Debrief on calendar triggers such as Nordvik Region's tests before each election and tax season.

Discussion

A dashboard shows that every Tier 1 service has a documented recovery plan. What does that figure report, and what would the register show for the same services?
The Atlassian case held for one customer and not for hundreds. Which of your recovery claims is true only at a scope smaller than the event you fear?
Why does the chapter require the attesting person to be someone who did not run the test, and the measuring probe to be one the runner does not control?

Written exercise

Exercise · 35 minutes

A recoverability claim and the test behind it

Select one recovery claim, your own or Nordvik Region's failover of the portal SVC-NR-002 to the recovery site. Write it with its six parts. Derive its present status from the last record. Then plan the test: who declares the start, the segregated environment and the copy restored from, the runbook followed as written with deviations recorded, the independent probe and integrity checks that stop the clock, how the recovery point is measured, who attests, the abort rule and who can stop the test from outside the system under test, the clock for expiry and the listed changes that end the result early.

Deliverable: A register row with a six-part claim and derived status, and a one-page authorised test plan with roles, probe, abort rule, attestation and expiry conditions.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

Hansa Mutual's failover of payments, RST-HM-2027-007, passed on 2027-10-05 with 1 hour 35 minutes against 2 hours. What is its status in the register in September 2028?
Show the answer
D. Expired, because the 180-day clock ended on 2028-04-02
A passing test stops counting when its clock ends. The 180-day default clock ended on 2028-04-02 with no re-test, so the status is expired and the claim does not count as recoverable.
What does a restore from backup into a segregated environment not prove, according to the table of test kinds?
Show the answer
B. That the production target, at production scale and concurrency, can be restored over
The test proves that data can be recovered within the RPO and the service starts on it. It leaves unproven that the production target at production scale and concurrency can be restored over.
Which role does the chapter allow AI in a restore test at the highest degree of delegation it names?
Show the answer
A. A runbook runner executing a catalogued restore into a segregated environment at Level 3 Bounded
AI plans, executes bounded steps under a mandate in a segregated environment, analyses and maintains records. It does not attest, failover and restore-overwrite in production stay at Level 1, and status is derived by rule.