AI Operations: workbook

How to use this workbook

The workbook follows the course on AI Operations lesson by lesson. Read the chapter (or its whitepaper) before each lesson, bring the workbook to the session and keep what you produce in the exercises: they add up to the final assignment.

After the course you can

Describe an operations practice or agent by function, action class, usage mode and degree of delegation
Set a ceiling per action class from reversibility, blast radius, evidence and window, and attach an autonomy budget to it
Apply the three tests of the observer principle to a verification, an observer and a stop path
Write the artefacts of the loop: an objective record, a correlation policy, a runbook manifest, a mandate manifest and an observer manifest
Write a recoverability claim with its six parts and derive its status from dated, attested tests
Assign the decision rights of an incident and map a regulatory duty to a control, an owner and evidence
Design an on-call charter and a drill year that keep the load humane and the manual craft in practice
Present to a board what the organisation can and cannot prove about its operations, with labelled statements about the future

Module 1 · Part I · The Operations in Question

Part I states the question of the book and builds its frame. The participant meets the five-part thesis, the lineage of operating models and the scarcity each answered, the six pressures that strain the inherited shape, and the operations loop with its five functions, eleven action classes, autonomy budget, Level 4 prohibitions, observer principle and resilience claim. The participant leaves able to describe any operations practice or agent in the book's vocabulary and to draft a first delegation profile.

Chapter 1 · 75 minutes

Introduction: Operations with AI

The lesson introduces the question of the book: what happens to IT operations when AI takes part in observing, diagnosing, changing and restoring services, and when the operator's own AI becomes one of the most privileged workloads in the estate. Participants work through the five-part thesis, the distinction between usage mode and degree of delegation, the labels observable, expected and speculation, and the two fictional organisations, and begin an inventory of where AI already takes part in operating one service.

Learning objectives

State the five parts of the book's thesis and what evidence the book says would refute them
Distinguish a usage mode from a degree of delegation and place an operations tool in both
Classify statements about AI in operations as observable, expected or speculation, as the chapter does
Draft an inventory record of where AI already takes part in operating one Tier 1 service

Key concepts

From the queue to the control loop: The first part of the thesis: the work moves from handling tickets and alerts to designing, bounding and verifying closed loops, with service level objectives that say what good is, runbooks that are code and mandates that say what an agent may do.
Usage mode and degree of delegation: Two vocabularies the book keeps apart. The usage mode, Mode 0 Ambient AI to Mode 5 Autonomous production agents, says how the AI is deployed and whose identity it uses. The degree of delegation, Level 0 Advisory to Level 4 Autonomous, says who decides each action.
The Mode 0 blind spot of operations: Ambient AI inside observability, service-management and managed-service tools acts with the provider's identity; the organisation sees the output and often not the rule, so the degree of delegation was not set by the organisation at all.
Evidence with an age: A rule of the book: no chapter says that something is recoverable, covered or competent without saying what was tested, when and with what result, and every chapter on O3 Change or O4 Restore says what can be undone and who can stop it.
Operations mandate summary: A one-page summary of who and what may act on one service, stating for each agent the function, the action class, the degree of delegation and who can stop it; the chapter's example covers Nordvik Region's citizen e-service portal, SVC-NR-002.

Interactive exercises

Sort: Whose identity acts?
Match: What this book takes from the earlier volumes

Done on the lesson page, with feedback at once.

Discussion

The chapter says the faster night is not the better one. What exactly was missing on each of the two nights, and which gap is harder to close in your organisation?
Where in your operating chain does AI already act under a rule that your organisation did not set, and how would you find out what that rule is?
The book states what would show its thesis wrong. What would you need to measure over several years to test the claim that independent verification and manual exercise are worth their cost?

Exercise · 25 minutes

Inventory of AI in the operating chain of one service

Pick one Tier 1 service of your own organisation, or Nordvik Region's citizen e-service portal SVC-NR-002. List the tools in its operating chain: observability, alert routing, service management, change pipeline, runbook automation, capacity and cost tools, the service desk and any managed provider that watches or acts on it. For each tool, record whether it contains AI, the usage mode, the highest action class it can perform, an owner and how it is stopped. Where you do not know the answer, write unset or not tested, and do not fill the gap from the vendor's description.

Deliverable: A one-page inventory record for one service listing every tool, each AI use with mode and highest action class, an owner and a stop, with explicit entries of unset or not tested.

Self-test

In the chapter's table of usage modes in operations, whose identity acts when a Mode 5 Autonomous production agent remediates a service?
1. The identity of the engineer on call that night
2. The provider's identity, under a rule that the organisation did not set
3. The agent's own identity, with permissions of its own
4. The service owner's identity, lent for the incident
How does the chapter label the unattended operation of whole estates, including recovery from novel failure?
1. Speculation, on the evidence of the sources examined
2. Observable, because vendors already ship it by default
3. Expected, class by class, as evidence accumulates
4. Refuted, because the human-factors research rules it out
According to the practical guide, what does an inventory record with no blanks most probably indicate?
1. That the service is ready for Level 3 Bounded delegation
2. That the service owner has completed the quarterly review
3. That the organisation has no Mode 0 Ambient AI in the chain
4. That it was written from the vendor's brochure and not from the estate

Chapter 2 · 75 minutes

What Operations Did Before: Runbooks, ITIL, DevOps, SRE and AIOps

The lesson sets the baseline against which every later claim about AI is a difference. Participants follow six operating models, from the system administrator with a runbook to first-generation AIOps, ask which scarcity each answered, read the survey finding on change advisory boards with its cautions, and meet the human-factors work on supervising automation. They compare a ticket-era and a code-era change record and classify the practices of one service.

Learning objectives

Name the six operating models of the lineage and the scarcity each answered
Explain the survey finding on external change approval and the three cautions the chapter attaches to it
Compare a ticket-era and a code-era change record by what each can prove
Classify the practices of one service by lineage, scarcity and disposition

Key concepts

Scarcity behind the shape: The book's own interpretation that each operations practice is a device for rationing attention: the ticket queue rations order, the runbook rations expert judgement, the rota rations hours, the change advisory board rations review, and the alert threshold rations the operator's gaze.
From scripts to state engines: A second lineage of technique that moves from describing steps to describing a desired state and from running on command to continuous reconciliation, as in Kubernetes controllers and GitOps; each step moved the loop from the person to the engine.
Toil: Work tied to running a service that is manual, repetitive, automatable, tactical and without enduring value, and that scales linearly as the service grows; the SRE literature reports an aim of keeping it below 50 per cent of each engineer's time.
First-generation AIOps: The application of machine learning to operational data, answering scarcity of attention to signals. Research concentrated on O1 Observe and O2 Diagnose, and remediation was the least studied area; practitioners listed label scarcity, data quality and misbehaving models among its difficulties.
Ironies of automation: Bainbridge's 1983 argument from industrial control: automating a task leaves the operator the tasks that could not be automated, which are monitoring the automation and taking over when it fails, while the skills needed for taking over decay.

Interactive exercises

Match: Operating models and their artefacts
Sort: Delegation that was already there

Done on the lesson page, with feedback at once.

Discussion

Which practice in your operations function has outlived the scarcity that produced it, and what would be lost if it were retired tomorrow?
The chapter says AI lowers the price of reading but not the price of deciding or of being accountable. Where does that distinction fall in your incident process?
A watchdog script that restarts a failed process is described as close to Level 3 Bounded in shape. What does it lack that the book would require of an agent doing the same thing?

Exercise · 25 minutes

A practice register for one service

Following the practical guide, list the practices in use for one service on one line each: the queue and its priority rules, each runbook, the on-call rota, the alert rules that page, the change types and their approval routes, and the incident and problem reviews. For each entry record the lineage it comes from, the scarcity it answered, when it was last used, an owner, whether the scarcity still holds and a disposition: keep, automate, change owner or retire. The scope is practices and not people, so no entry may name a person as a performance measure.

Deliverable: A register of at least eight practices with lineage, scarcity answered, last use, owner, disposition and a first review date.

Self-test

What did the 2019 delivery research report find about requiring approval by an external body, such as a change advisory board, for significant changes?
1. Respondents with it were 2.6 times more likely to be low performers, with no evidence of lower change fail rates
2. Respondents with it reported change fail rates 2.6 times lower than those relying on peer review alone
3. It proved by controlled experiment that removing the board improves delivery performance
4. It found the board harmful in every role and recommended abolishing it without replacement
In the ticket-era record CHG-HM-5870, what does the chapter say the approval was a property of?
1. The reviewer named in the record
2. The canary stage of the rollout
3. The pre-approved standard change template
4. The independent probe that verified the routing weights in the canary
Which scarcity does the chapter say site reliability engineering answered?
1. Expert time at the console
2. Flow between development and operations
3. Attention among product teams
4. Reliability engineering skill

Chapter 3 · 90 minutes

The Pressure on Operations: Complexity, Dependency, Cost and Machine-Speed Change

The lesson explains why the inherited shape of operations strains. Six pressures act at once and each lands on named functions. Participants learn to keep measured, surveyed and argued evidence apart, read three provider outage reports of 2025 for their mechanism, follow the review-capacity argument for AI-authored change, and draw a dependency-and-blast-radius map with its unknowns written down.

Learning objectives

Name the six pressures and the operations functions on which each lands
Grade a claim about operations as measured, surveyed or argued and say how far it carries
Explain the common mechanism in the three provider outage reports of 2025
Draft a dependency-and-blast-radius map for one Tier 1 service with recovery evidence and unknowns

Key concepts

Three grades of evidence: Measured evidence rests on the issuer's telemetry, records or the text of an instrument; surveyed evidence on self-reported answers from a stated, often self-selected group; argued evidence on reasoning from mechanism, and is only as strong as its stated premises.
Shared fate: The pattern in the provider reports: a small first fault, such as a record, a policy row or a permission, propagates through a mechanism built for efficiency, such as global replication, a shared regional endpoint or automated distribution of a configuration file.
The review-capacity argument: The book's argued conclusion: review by people scales with competent readers and their time, authoring by agents scales with compute, so the gap closes through shallower reviews, a growing queue, or a machine reviewing the machine's change, acceptable only if independent.
The cost of seeing: Telemetry volume grows with the number of services while the people who can read it do not, so the share of signal anyone reads falls and O1 Observe risks becoming a store and not a sense.
Dependency-and-blast-radius map: A record for one Tier 1 service of what it depends on, what depends on it, what it would take down, and what evidence exists that recovery works, with each recoverability claim carrying a test, a date and a result, and unknowns listed with owners.

Interactive exercises

Sort: Observable, expected or speculation
Scenario: What goes on the map

Done on the lesson page, with feedback at once.

Discussion

The chapter warns that a measured incident report is a measurement of one failure chosen by its visibility. How should that limit what you conclude from a well-known outage?
If machine-authored change keeps rising, which of the three ways of closing the review gap is your organisation taking today, and what does it hide?
Nordvik Region's own monitoring saw the SamDrift DNS fault 22 minutes after the provider's status page. What does that say about where its observation depends on what it observes?

Exercise · 35 minutes

A dependency-and-blast-radius map in an afternoon

For one Tier 1 service of your own, or Hansa Mutual's claims core system SVC-HM-001, write the availability objective, recovery time and recovery point; if one is missing, record it as the first unknown. List what the service calls, using traces, infrastructure code and network flow data where you have them and not memory, including shared services such as identity, DNS, certificates, time and the observability pipeline. State the blast radius and the degraded mode. For each recoverability claim, give the test, its date and its result, or mark it untested. Finish with a list of unknowns, each with an owner.

Deliverable: A one-page service record with dependencies, blast radius, recovery evidence with dates and results, and at least two owned unknowns.

Self-test

Which grade of evidence does the chapter assign to the claim that review capacity does not scale with AI-authored change?
1. Measured, from provider post-incident reports
2. Surveyed, from the annual delivery research
3. Argued, from mechanism with stated premises
4. Observable, from the text of DORA Article 19
What did Mandiant report as the average time to exploit for the 138 vulnerabilities disclosed in 2023 and exploited in the wild?
1. 63 days, the same as in 2018 and 2019
2. 5 days, against 32 days in 2021 and 2022
3. 32 days, against 63 days in 2018 and 2019
4. 24 hours, matching the NIS 2 early warning
Why can neither organisation write recoverable for the service whose test is cited in the chapter?
1. Their providers refused to take part in any recovery test
2. DORA forbids the word until a supervisor has inspected
3. Their tests passed but were run on a non-production copy
4. The last recorded test failed on time and has no re-test

Chapter 4 · 120 minutes

The Operations Loop Defined: Five Functions, Autonomy Budget and Observer Principle

The lesson defines the frame the rest of the course uses. Participants learn what each of the five functions takes in, produces and where it ends; the eleven action classes and how a ceiling is set; the autonomy budget and what counts as an autonomy error; the four prohibitions on Level 4 Autonomous; the three tests of the observer principle; and resilience as a claim. They write a delegation profile for one agent and one service.

Learning objectives

Define each of the five operations functions by its input, its output and where it ends
Set a ceiling for an action class from reversibility, blast radius, evidence and window
Calculate the consumption of an autonomy budget and state what happens at 70 and 100 per cent
Draft a delegation profile for one agent and one service, with observer and stop holders

Key concepts

Action class: A kind of effect on the estate, not a tool. The book names eleven, from read and annotate to delete and restore-overwrite; a runbook step belongs to exactly one, and any class not listed in a delegation profile is undelegated.
Autonomy budget: A tolerance for faulty actions that each Level 3 class carries per window, 28 days by default. Errors counted by the observer consume it; a review is held at 70 per cent, and at 100 per cent the class falls to Level 2 until a named authority decides otherwise.
Autonomy error: An action that had to be reversed, whose intended effect the independent observer did not find or found with the service worse, that the policy decision point refused as outside the mandate, or that had a customer-visible or regulator-visible effect, which consumes the whole tolerance at once.
Observer principle: Three tests: the verification does not share its failure modes with the action (independence in model, data, authorship, vendor and time), the observer does not depend on the system it observes, and the stop path works when the platform does not.
Resilience as a claim: A statement of recoverability with a scope, an objective, a measurement, a date and a result. Its status is derived by rule, only a verified claim may be called recoverable, and evidence expires by clock, 180 days by default for a Tier 1 restore or failover test, or on substantive change.

Interactive exercises

Sort: Where Level 4 Autonomous is never used
Scenario: A budget spent in one night

Done on the lesson page, with feedback at once.

Discussion

The chapter says raising a level is a recorded decision by a named authority while lowering one is immediate and needs no approval. What does that asymmetry protect, and who in your organisation would hold each right?
A budget never touched in a year is called a signal. What are the two explanations the chapter offers, and how would you tell them apart?
Why does the book treat a Level 3 runbook that deletes production data by a catalogued step as breaking the same rule as Level 4, in a weaker costume?

Exercise · 35 minutes

A delegation profile for one agent and one service

Write a delegation profile for an operations agent on one service, either your own or a profile DLG-NR-OPS-001 for Nordvik Region's portal. Copy the tier, objectives, recovery time and recovery point from the service record. List the agent's technical permissions and strike every action class the profile will not name. For each remaining class set a ceiling as the lowest of reversibility, blast radius, evidence and window, with a condition. Add the budget tolerance and window, the location of the policy decision point, what independently verifies each Level 2 or Level 3 action, the stop holders, and the date of the next review.

Deliverable: A delegation profile of one page with a ceiling and condition per action class, a budget, a named observer, stop holders outside the agent's platform and a review date.

Self-test

What is the book's default ceiling for the action class restore-overwrite?
1. Level 3 Bounded, provided the runbook has a tested rollback step
2. Level 2 Supervised, with approval by the incident commander
3. Level 0 Advisory, because it changes an alert or a monitor
4. Level 1, with a named person executing after a restore into a separate target is compared
An agent's request is refused by the policy decision point as outside its mandate, and nothing happens in production. How does the autonomy budget treat this?
1. It counts as an autonomy error, because an attempt costs even though nothing happened
2. It is ignored, because the refusal shows the control is working as designed
3. It consumes the whole tolerance at once, as a customer-visible effect does
4. It is recorded by the agent itself and reviewed only when the 28-day window has closed without a reversal
Which of these does the dependence test of the observer principle ask?
1. Whether the verification shares its model, data, authorship or vendor with the action that it verifies
2. Whether the observer depends on the system it observes and so goes silent when it fails
3. Whether the stop holder can act without the agent's platform or identity provider
4. Whether the evidence for a recovery claim is still inside its clock of 180 days

Module 2 · Part II · The Craft

Part II is the craft: one or more chapters per operations function. The participant builds the artefacts that make the loop real, namely a service level objective with burn-rate alerts, an alert-correlation policy with a suppression ceiling, an incident record that marks every assistant contribution, a change gate and a standard change class, a rollout policy with an independent verifier, a runbook manifest, guardrails for a rightsizing agent and an action register with a closure rule. In each, what verifies is kept apart from what acts.

Chapter 5 · 90 minutes

Observe: Telemetry, Observability and Service Level Objectives

The lesson treats O1 Observe without AI, as the baseline that later chapters change. Participants learn why the unit of observation is the user journey, how a signal becomes a service level indicator, an objective and an error budget, how multiwindow burn-rate alerting works and where it does not fit, and what cost, retention and the GDPR mean for telemetry. They write an objective record for one service.

Learning objectives

Define a service level indicator, a service level objective and an error budget from a user journey
Calculate an error budget and a burn rate for a given objective and window
Explain why a deadline objective and a low-traffic service resist burn-rate alerting
Draft a service level objective record with an independent probe, an owner and a test

Key concepts

Symptom and cause: A symptom is what a user or calling system experiences, such as failing requests; a cause is why. The chapter follows the SRE literature in paging on symptoms tied to an objective and using cause-level signals for diagnosis.
Error budget: The complement of the objective over the window: the amount of failure the objective tolerates. At 99.9 per cent over 30 days a service may be unavailable for 43.2 minutes, at 99.95 per cent for 21.6 minutes. It is not the autonomy budget of Chapter 4.
Burn rate: How fast a service consumes its error budget relative to the objective. Alerts combine a long window, which confirms that significant budget is gone, with a short window, which confirms the burn is still going on.
Synthetic probe outside the platform: A scripted request that exercises a user journey at a fixed interval from a place the operator chooses. It gives steady traffic on a quiet service, measures what a user sees and keeps working when the platform's own telemetry fails.
Minimisation at the collector: The chapter's reading of GDPR Article 5(1)(c) and (e) for telemetry: attributes that identify people are deleted in the collection pipeline before storage, because not collecting potentially sensitive data is the best prevention.

Interactive exercises

Put in order: Writing an objective from a user journey
Scenario: A quiet service that pages on noise

Done on the lesson page, with feedback at once.

Discussion

The chapter says an operations function that cannot say when it is blind has not observed anything. How would your organisation learn that its observability platform had stopped ingesting?
Why does the chapter warn against a service that is far more reliable than its objective, and what follows for what you promise to dependent services?
A metric by named engineer is a measure of a person. Where does your telemetry record people at work, and who has agreed to it?

Exercise · 35 minutes

An objective record from a user journey

Choose one service, your own or Nordvik Region's citizen e-service portal. Name in one sentence, from the user's side, the one or two journeys whose failure is worth paging for. Define an indicator for each as a share of good events, set an objective and a window, and calculate the error budget in minutes. Add a page-level burn-rate alert with a long and a short window, a measurement point outside the platform, the owner role and the on-call route. State how the page will be tested with a seeded fault, and write one sentence on what the indicator cannot see.

Deliverable: A versioned objective record with indicators, objectives, error budget, a two-window burn-rate alert, an outside probe, an owner, a planned alert test and a stated blind spot.

Self-test

In the chapter's starting parameters for multiwindow alerts, which burn rate pages a person on a one-hour long window with a five-minute short window?
1. A burn rate of 1, consuming 10 per cent of the budget
2. A burn rate of 6, consuming 5 per cent of the budget
3. A burn rate of 14.4, consuming 2 per cent of the budget
4. A burn rate of 43.2, consuming the whole budget of 43.2 minutes
Why does a ratio-based error budget degenerate for Hansa Mutual's objective that the settlement run completes by 06:00 on run days?
1. With about twenty run days a month, 99.95 per cent allows about one hundredth of a missed run, which is none
2. The settlement run produces too much telemetry for the alert windows to be evaluated in time
3. Burn-rate alerts are forbidden by DORA Article 10 for critical or important functions
4. The run is measured by a synthetic probe, which cannot be combined with an error budget
What does the recovery block of the record SLO-NR-002 deliberately show beside the promised recovery time of 2 hours?
1. The provider's contractual service credit for a missed recovery time
2. The design estimate of the platform team, signed by the service owner
3. The average of the last three failover tests of the portal, rounded to the nearest full hour
4. The failed test RST-NR-2027-002, which measured 5 hours 20 minutes and has no re-test

Chapter 6 · 90 minutes

Observe with AI: Correlation, Anomaly Detection and Alert Noise

The lesson asks what changes when software that learns from telemetry sits between the signals and the person on call. It concerns O1 Observe and O2 Diagnose. Participants set the gains of baselines, correlation and summaries beside six failure modes, learn five instruments of evaluation, and write an alert-correlation policy with a suppression ceiling, expiring holds and numeric stop conditions.

Learning objectives

Describe four capabilities AI adds to the alert stream and the failure mode of each
Distinguish the two honest forms of noise reduction from suppression that hides signal
Design an evaluation with a golden incident set, per-class recall, a noise budget, seeded alerts and sampling
Draft an alert-correlation policy with a suppression ceiling and a stop condition written as numbers

Key concepts

Baseline drift to the new broken normal: A baseline that adapts to the data adapts to degradation as well: a slow memory leak or creeping latency moves the baseline with it, and the anomaly score stays low while the service gets worse.
Golden incident set: Past operational episodes, each with the raw alert stream as it arrived and a ground truth written by people. It is held outside the system under evaluation, changed only by people who do not own the correlator, and every replay is logged.
Noise budget: A statement of how much unneeded interruption a rota may bear, made of three numbers: a ceiling on pages per on-call shift, a floor on the share of pages marked actionable, and a ceiling on out-of-hours pages that were not Tier 1. It is reported for the rota, never a named engineer.
Seeded alerts: Synthetic alert events introduced by an authorised injector upstream of the correlation layer, tagged with a run identifier and a sealed expectation of what should reach whom and by when, written before the injection and held outside the correlator.
Suppression ceiling: The rule that no alert on a customer-facing service level objective, on the heartbeat that watches the observation path or on a Tier 1 recovery objective is suppressed above Level 1 Assisted: the layer may propose and a person acts.

Interactive exercises

Sort: What a correlation policy permits
Match: Five instruments of evaluation

Done on the lesson page, with feedback at once.

Discussion

A model that learns which alerts engineers close without action will learn to withhold them. How would you find out that it had also withheld the rare alert that mattered?
The chapter treats a figure from another organisation's estate as a reason to try a method, not a measurement of it. What would your first own measurement be?
Hansa Mutual declares the vendor's correlation logic and measures it from outside until the contract grants test rights. What contractual terms would turn a declared feature into a verified one?

Exercise · 35 minutes

An alert-correlation policy for one service

For one Tier 1 or Tier 2 service, list its alert classes by consequence, not by tool: customer-facing objective burn, dependency health, saturation, batch and schedule, certificate and licence expiry, and hygiene. For each class state what the correlation layer may group, summarise, hold or suppress and at which degree of delegation, keeping suppression of customer-facing objective alerts at Level 1 Assisted at most. Make every hold expire in minutes. Add a noise budget for the rota, a weekly sampling rate for held-back alerts, a seeding plan with sealed expectations, and stop conditions written as numbers that a component outside the correlator can enforce.

Deliverable: A policy file with classes, actions and levels, a suppression ceiling, hold expiries, a noise budget, sampling and seeding rates and numeric stop conditions.

Self-test

Which of these is the chapter's example of a stop condition that can actually be enforced?
1. Reduce the level of delegation if the quality of correlation appears to fall
2. Seed catch rate below 0.90 over four weeks freezes promotion; below 0.80 returns every class to Level 1
3. Ask the vendor to review the model whenever engineers complain about missed pages
4. Suspend the correlator when the incident commander loses confidence in its summaries
Where must the golden incident set be held, according to the chapter?
1. In the tuning data of the correlator, so that it learns from the organisation's own real incidents
2. In the retrieval index of the summariser, so that summaries cite precedent
3. With the observability vendor, under the support contract for the feature
4. Outside the system under evaluation, changed only by people who do not own the correlator
What did the 28-day trial recorded in EVL-HM-2028-03 report for the customer-facing objective class?
1. All 14 episodes paged within the time budget, with none suppressed by AI
2. Ten of 11 episodes paged, with one seeded alert missed in the grouping layer
3. A median of 31 pages per shift, unchanged from before the trial
4. A recall of 0.91, which froze promotion of the class for four weeks

Chapter 7 · 90 minutes

Diagnose: Triage, Root-Cause Analysis and the Incident Assistant

The lesson concerns O2 Diagnose: triage, hypothesis, test and handover under incident command, and the incident assistant that retrieves precedent, correlates changes, ranks hypotheses and drafts updates. Participants read five benchmark studies for what each does not show, examine automation bias and anchoring, learn the human decision points and the second-hypothesis rule, and build an incident record that marks every assistant contribution.

Learning objectives

Describe the four capabilities of the incident assistant and why its tools are read-only
Explain what a published root-cause accuracy figure does and does not show for another estate
Apply the second-hypothesis rule before a consequential action based on a ranked hypothesis
Draft an incident record that separates the assistant's retrieval, ranking and drafting contributions

Key concepts

The assistant as evidence, not authority: The chapter's own conclusion: in diagnosis the assistant is a source of evidence and hypotheses and never the authority for an action. Its tool surface is read and annotate only, and a hypothesis is a reason to act, not an authority to act.
Second-hypothesis rule: Before an action of a class above annotate, notify or scale is taken on the assistant's top-ranked hypothesis, a person records a second hypothesis and one observation that would refute the first, from a source the assistant did not select, and the result.
Automation bias: Errors of omission and commission that follow from relying on an imperfect decision aid. The review cited reports that it occurs in naive and expert users and is not removed by training or instruction, so the controls that work cost time.
Replay of closed incidents: The organisation's own measurement: past incidents are replayed with the cause withheld and the assistant's ranking scored. Hansa Mutual's illustrative replay of 60 incidents put the right cause first in 24 and in the first three in 41.
The record as a state machine: The incident record's state, not a sentence in a free-text field, gates action: a state changes only through a named role, and closure is a guard that requires evidence.

Interactive exercises

Match: What the incident assistant does
Scenario: The first-ranked hypothesis

Done on the lesson page, with feedback at once.

Discussion

In INC-HM-2028-0871 the assistant's retrieval helped and its ranking was wrong. Why does the chapter insist on recording the two separately, including the wrong contribution?
The controls against anchoring cost minutes during an incident. For which classes of action is that cost justified in your organisation, and for which is it not?
Nordvik Region chose retrieval and Hansa Mutual chose ranking. What would have to be true of your data and your rota before ranking were defensible?

Exercise · 30 minutes

An incident record that marks the assistant

Take a closed incident from your own organisation, anonymised, or use the outline of INC-HM-2028-0871. Rewrite it as a record with the service and objective, the commander and operations lead, the timestamps of the first flag, the first acknowledgement and the declaration, and a timeline. Mark each contribution of an assistant by kind: retrieval, change correlation, ranking or drafting, and whether it proved right. For the action that resolved the incident, record the first hypothesis, a second hypothesis, the observation that would refute the first, its independent source and the result. Note who held the stop for the assistant.

Deliverable: An incident record of one to two pages with marked assistant contributions, three timestamps, a recorded second hypothesis with refuting observation, and named decision holders.

Self-test

According to the table of human decision points, at what level does the assistant take part in accepting a hypothesis as the basis for action?
1. Level 2 Supervised, because the record is a draft until confirmed
2. Level 1 Assisted, because it drafts and looks up contacts
3. Level 3 Bounded, inside a catalogued runbook
4. Level 0 Advisory, subject to the second-hypothesis rule
What does the chapter conclude a published benchmark figure, such as 11.34 per cent of OpenRCA failures solved, tells an organisation?
1. The probability that its own assistant's first-ranked hypothesis is correct
2. A bound on what it can reasonably hope for, and little about what its own assistant will do
3. That language models cannot contribute to diagnosis and should be limited to drafting
4. The share of incidents for which a human commander is no longer required
Why may the assistant open a candidate incident at Level 2 Supervised when declaration itself belongs to the incident commander?
1. The candidate is a draft record, inert until a person confirms it, and starts no roles, clocks or pages
2. NIS 2 Article 23 requires that the first flag be raised by an automated system
3. The assistant's rank score has been calibrated as a probability by the replay set
4. Candidate incidents are excluded from the incident record and so carry no risk

Chapter 8 · 120 minutes

Change: Releases, Infrastructure as Code and AI-Authored Change

The lesson concerns O3 Change when agents write much of the change. Participants learn the six properties that make a pipeline a control, what a provenance record holds, how a policy gate computes the change class from the diff, how review is kept real when drafting is cheap, why a reconciler is a change actor whose sync policy is a mandate, and how to define a standard change class that an agent may author.

Learning objectives

List the six properties that make a change pipeline a control and not only a convenience
Explain what the provenance of an AI-authored change records and why it stores references and not an engineer's typed words
Map the settings of a GitOps sync policy to action classes and ceilings
Define a standard change class that an agent may draft and a pipeline may release

Key concepts

Provenance of change: The record of a change's origin: the author's identity and whether it is a person or an agent, the mandate and level, the model, the instructions and inputs as references, and what was not tested. It differs from build provenance, which attests what a build platform produced.
Protected paths: The pipeline definitions, policy files, reviewer lists and agents' mandate manifests, together the change system's own configuration, plus the gate itself and the reconciler's policy objects. Level 4 Autonomous is never used for an edit to them.
Computed change class: The class of a change is computed by the gate from the diff, the plan and the target, not declared by the author; a declared class lower than the computed one is denied.
The reconciler as a change actor: A GitOps engine that applies is an actor in the change system whose identity is its reach. Its sync policy is its mandate: automated sync and self-heal at Level 3 Bounded only for catalogued classes, and prune never at Level 4 where it would delete data.
Oversight-quality measures: Measures of the review and not the reviewer, reported per team: seeded-error catch rate of at least 0.85 per quarter, review time against diff size, a rejection and rework rate between 5 and 30 per cent, at most 5 open drafts per reviewer, and the change-correlated incident rate.

Interactive exercises

Match: A sync policy is a mandate
Sort: Degrees of delegation in change

Done on the lesson page, with feedback at once.

Discussion

A rejection and rework rate near zero for a quarter is read as a signal of rubber-stamping. What else could explain it, and how would seeded errors tell the explanations apart?
A provenance record that captures how an engineer used an agent can also be used to watch the engineer. How do the two organisations' agreements shape what the record may store?
With seven platform engineers, Nordvik Region has the same people drafting and reviewing. What does the Region do about it, and what would you do in a team of that size?

Exercise · 35 minutes

A standard change class an agent may author

Following the practical guide, pick a kind of change that your organisation, or Hansa Mutual in the chapter's example of certificate renewal outside critical functions, has made by hand at least 20 times in the last 12 months with no change-correlated Tier 1 or Tier 2 incident. Define the class as a file: eligible service tiers, what the diff may and may not touch so that the gate can compute membership, the independent probe that verifies it, the tested rollback with its date and result, the release window, the rule that suspends the class, the approving decision record, and an expiry.

Deliverable: A change-class definition with eligibility rules computable from the diff, probe, tested rollback, window, suspension rule, approver and expiry.

Self-test

What is the chapter's ceiling for the prune setting of a reconciler, which deletes what has left Git?
1. Level 3 Bounded for every resource kind, because the repository is the declared state of the estate
2. Level 4 Autonomous outside a freeze, because the deletion is declared
3. A named person for stateful kinds, and never Level 4 where it would delete data
4. Level 0 Advisory in all cases, because prune is off by default
Who may review an agent-authored change to a Tier 1 service, according to the controls that keep review real?
1. A named person who is neither the requester nor the drafter
2. The engineer who asked the agent for the draft
3. A second agent from the same model family as the author
4. Any member of the change advisory board, after the release
What does the oversight-quality table set as the default cap on open drafts per reviewer, and what happens when it is exceeded?
1. At most 20, after which review time per 100 lines is halved
2. At most 10, after which the change class is suspended until two quarters pass without an incident
3. At most 1, after which the reviewer is replaced on the rota
4. At most 5, after which drafting by the agent pauses for that reviewer's queue

Chapter 9 · 105 minutes

Change Safely: Progressive Delivery, Freezes and Blast Radius

The lesson concerns O3 Change and O4 Restore at the point where a change meets production. Participants learn rings, canaries with bake time, feature flags and cells; read five public incident reports for why configuration is the dangerous change; define stop conditions evaluated by a verifier that is not the releaser; and work with freezes and blast-radius budgets. They write a staged rollout policy.

Learning objectives

Explain how rings, bake time, feature flags and cells limit the exposure of a change
Identify, in a public incident report, why a configuration change was not contained
Specify stop conditions and the properties of an independent verifier for a staged rollout
Draft a rollout policy with a blast-radius budget and a freeze rule for one Tier 1 service

Key concepts

Change-correlated incident rate: The share of incidents in a period for which a change to the affected service or its dependencies fell inside a declared look-back window. A report states the window, the denominators and the changes it cannot see; correlation in time is not cause.
Ring and bake time: A ring is a stage of a rollout defined by the population that receives the change; bake time is the period a change stays at a stage, observed, before it is judged and widened.
Independent verifier: The system that evaluates stop conditions: it is not the releaser, reads from its own path, cannot be edited by the releaser, holds a guardrail out of sight and has been shown to fire by a dated seeded-fault test.
Freeze: A suspension of some or all change to a service for a period when the cost of failure is unusually high or the capacity to respond is low. It has a scope, a period, a reason and an owner, and named holders who may authorise a recorded exception; never an agent.
Blast-radius budget: A statement of how far a mistake may spread before a person must decide, with four parts: scope per ring, the longest time a change may stay unverified at that scope, the number of changes in flight at once, and the record of consumption.

Interactive exercises

Match: What would have limited the exposure
Scenario: Only a configuration change

Done on the lesson page, with feedback at once.

Discussion

In INC-HM-2027-0833 a configuration push was classed as standard and the canary stage was skipped. What would have had to be different in the pipeline for the classification not to matter?
Nordvik Region's second look comes from a provider that shares its DNS dependency. How independent is that verifier, and what would a probe path that does not share the dependency look like?
A freeze that no one can break will be broken informally. Who should hold the exception in your organisation, and what must the record of an exception contain?

Exercise · 30 minutes

A staged rollout policy with a second system as judge

For one Tier 1 service, or Hansa Mutual's claims core organised in cells by market, list the rings or cells in order of exposure, smallest first, and record why. For each stage set the bake time and the stop conditions, at least one absolute condition against the service level objective. Name the verifier, the team that operates it, the path it reads from, and how it is shown to fire with a seeded fault inside an authorised, scoped test. State what can be undone at each stage, who may hold or stop the rollout without approval, the freeze periods, and who may authorise an exception.

Deliverable: A rollout policy with ordered stages, bake times, stop conditions, a named independent verifier with a seeded-fault test date, rollback statement, stop holders and freeze rules.

Self-test

According to the chapter's reading of the public reports, why is configuration the dangerous change?
1. It is written mostly by agents, whose drafts contain more defects than code written by people
2. It travels by the fastest path with the weakest checks and can reach a whole population at once
3. It cannot be held in version control and therefore has no provenance record
4. It is exempt from the change management duties of DORA and NIS 2
In Hansa Mutual's design, how far may the IT-operations agent release a standard change on its own, and who decides to widen?
1. To all cells; the agent widens when its own post-sync test passes
2. To cell 2; the service owner widens after reading the agent's report
3. To ring 0 only; the change advisory board widens at its weekly meeting
4. To cell 1; the pipeline controller widens on the independent verifier's pass
What do progressive-delivery controllers such as Argo Rollouts supply, and what do they leave open?
1. They supply the mechanics of hold, abort and promote, and do not supply independence
2. They supply the independent verifier, and leave the bake times and the order of rings to the releasing team
3. They supply the freeze calendar, and leave rollback to the incident commander
4. They supply the blast-radius budget, and leave stop conditions to the agent

Chapter 10 · 120 minutes

Restore: Runbooks, Auto-Remediation and Recovery

The lesson concerns O4 Restore: remediation, rollback, failover and recovery from backup. Participants learn the runbook manifest, the four tiers of remediation, the first ten actions of a catalogue, five properties of safe execution, what state engines do and do not guarantee, and independent verification. The chapter's debate asks whether AI should remediate production on its own, and the verdict is given per action class.

Learning objectives

Write a runbook manifest with action class, level, preconditions, independent probe, rollback or hold, stop and owner
Classify remediation actions by tier and assign a defensible ceiling to each
Explain why a durable workflow engine does not guarantee that an effect happens once
Argue both the case for concern and the case for calm on autonomous remediation, and state the chapter's verdict

Key concepts

Runbook manifest: The runbook written so that a policy engine, a person and an agent read the same thing: an RBK record with a trigger, an action class, a level, explicit preconditions, an independent probe, a rollback or hold, a stop and an owner with a review date.
Four remediation tiers: Distinguished by who acts and who approves: the system notifies (Level 0 Advisory), proposes while a person executes (Level 1 Assisted), executes each action with a named person's approval (Level 2 Supervised), or executes a catalogued runbook within a mandate (Level 3 Bounded).
Self-verification: The common failure in which an agent restarts a process and checks the process's own health endpoint: it has verified that the process answers, not that the service is better. Verification must be independent in identity, data path and failure domain, with a soak period.
Retry storms and duplicate effects: Retries make engines resilient and are also the mechanism of failure. Engines guarantee that a procedure progresses, not that its effect happens once, so idempotency keys, retry limits with back-off and a counter held outside the engine are conditions of delegation.
The last line: Backup and restore when remediation and rollback fail: offline or immutable copies, a segregated restore target, a restore path that does not depend on the failed system, and rehearsed restores. The cut-over of a restore over existing data is a named person's decision.

Interactive exercises

Put in order: Four tiers of remediation
Sort: The first catalogue at Hansa Mutual

Done on the lesson page, with feedback at once.

Discussion

A restart that clears a condition also clears the evidence. What must a runbook preserve before it acts, and who needs that evidence afterwards?
The case for calm argues that a bounded agent is more accountable than the alternative. What does an action under a manifest leave behind that a person at a console at night does not?
Why does the chapter treat restore over existing data as a named person's decision even when the agent can prepare the target and run the integrity checks?

Exercise · 30 minutes

A runbook a machine may run

Choose one known condition on one service, for example an unhealthy stateless instance or a certificate within its renewal window. Write the manifest: the alert or objective burn that triggers it, the action class and level, explicit preconditions including a capacity precondition, the evidence to preserve before acting, a dry-run step, limits on targets per run and runs per target, the independent probe with a soak period and passing rule, the rollback or hold, the stop and who holds it outside the agent's platform, and the owner with a review date. State how the manifest will be tested against a seeded fault in an authorised, scoped run.

Deliverable: A runbook manifest with trigger, class, level, preconditions, limits, independent probe, rollback or hold, stop holders, owner and a planned seeded-fault test.

Self-test

How does the chapter treat a request that an agent silence, mute, re-threshold or disable an alert, a probe or a check as part of remediation?
1. Delegated at Level 3 Bounded when a catalogued runbook includes the step
2. Delegated at Level 2 Supervised with approval by the on-call engineer
3. Delegated at Level 1 Assisted during an authorised change window
4. Not delegated at any level
What is the chapter's verdict on unattended recovery of whole estates, in which agents diagnose, choose, execute and verify across services?
1. Observable, since restarts and scale-out have been automated for years
2. Expected, wherever a run record supports widening to Level 3
3. Speculation, on the evidence examined
4. Prohibited, by the four Level 4 prohibitions of Chapter 4
In the split between engine and policy decision point, where must the stop for a runbook run ultimately be enforced?
1. In a guard step of the workflow, editable by whoever edits the workflow
2. By revocation of the credential and a deny entry, outside the platform
3. By the engine's own cancel signal alone, since it is part of the engine
4. In the agent's prompt, as an instruction to cease further actions

Chapter 11 · 90 minutes

Sustain: Capacity, Cost, Energy and Lifecycle

The lesson treats the steady-state half of O5 Sustain: capacity planning, cost, energy and lifecycle. Participants learn why a forecast is a distribution accepted on a back-test, how usage decisions differ from rate decisions, how an optimising agent can reduce resilience while meeting its objective, what the Energy Efficiency Directive asks of data centres, and how patch windows are set by maintenance group. They set guardrails for a rightsizing agent.

Learning objectives

Explain why a capacity forecast is stated as an interval with a back-test and an uplift
Distinguish usage decisions from rate decisions and say which belong in an agent's mandate
Design floors, windows and a veto that keep an optimising agent from removing recovery margin
Set remediation windows for vulnerabilities by maintenance group, with a ceiling for the agent's deploy action

Key concepts

Forecast as a distribution: A single value cannot decide how much margin to buy. The forecast gives an interval, the interval is accepted on a back-test of its coverage, and an uplift is applied where measured coverage falls below the nominal, as in CAP-HM-2028-014 where 80 per cent was measured against a nominal 95.
Usage and rate decisions: A usage decision, such as switching a resource off, can usually be undone and the cost of error is an interval in which the service was too small. A rate decision, such as a commitment to a term or purchase, cannot, and is a finance decision that is not delegated to an agent.
Floor: A lower bound the agent cannot cross, expressed in the unit of the claim it protects: a redundancy floor, a standby floor that allows reduction only after a dated failover test at the reduced size has passed, a retention floor and protected labels.
Veto: The right of the service owner, and of the incident commander during an incident, to block, reverse or suspend a proposal of the optimising agent without giving a reason to the agent.
Maintenance group: A set of assets of similar characteristics with a routine patch window and an emergency trigger set in advance. In the chapter's illustrative defaults, exposed Tier 1 assets have 14 days routinely, and on reported exploitation mitigation within 24 hours and patching within 72 hours.

Interactive exercises

Sort: Ceilings at the start for sustaining work
Match: Patch windows by maintenance group

Done on the lesson page, with feedback at once.

Discussion

A target such as reducing the cloud bill by ten per cent will be met by the cheapest route available. What would you add to the target so that the route does not pass through the recovery margin?
With public certificate validity scheduled to fall to 47 days by March 2029, what in your renewal process still depends on a person remembering?
The chapter reports patch velocity as a distribution by maintenance group and never by person. What would be lost, and what protected, by that choice in your organisation?

Exercise · 35 minutes

Guardrails for a rightsizing agent

For a cost or capacity agent acting on a set of services, yours or the scope of Hansa Mutual's agent HM-OPS-004, list the services with their tier. Write the floors: redundancy per Tier 1 service, a standby floor that cites a dated failover test, a retention floor and protected labels. Set the windows in which the agent may act, excluding freezes and scheduled restore tests. State the ceiling per action, with deletion advisory only and a quarantine before a person deletes, and commitments not delegated. Name who holds the veto, and say how savings and the absence of harm will be verified by something other than the agent's own report.

Deliverable: A guardrail policy with floors tied to dated tests, windows, ceilings per action, veto holders and an independent verification of savings and harm.

Self-test

On whom does the annual data centre reporting duty of Article 12 of the Energy Efficiency Directive fall, according to the chapter?
1. The owner or operator of a data centre with installed IT power demand of at least 500 kW
2. Every organisation that rents capacity in a colocation or cloud facility, whatever its size
3. The provider of any general-purpose AI model served from a European data centre
4. The management body of a financial entity, as part of its DORA Article 5 duties
What is the book's starting ceiling for an agent's action of deleting an unused resource?
1. Level 3 Bounded, since unused resources are waste by definition
2. Level 2 Supervised, with approval by the FinOps lead
3. Level 0 Advisory for the agent; a person deletes after quarantine
4. Level 4 Autonomous, but only for resources in non-production accounts
Under the standby floor, when may an optimising agent's proposal to reduce a recovery site be accepted?
1. When the standby has been idle for a full quarter and availability stayed green
2. When a failover test at the reduced size has passed within the recovery time objective, with dated evidence
3. When the forecast interval for demand at the primary site has narrowed
4. When the agent's own report shows that no harm followed earlier reductions

Chapter 12 · 90 minutes

Sustain: Problem Management, Post-Incident Learning and Knowledge

The lesson treats the learning half of O5 Sustain: the problem record, the blameless and evidence-based post-incident review, action closure and knowledge hygiene. Participants learn the triggers that turn incidents into a problem, the four outcomes of a problem, the risks of an AI-drafted review and the separation of timeline from analysis, and the closure rule under which an implemented but unverified action is reported as open.

Learning objectives

Distinguish an incident record, a problem record and a post-incident review by the question each answers
Explain the risks of an AI-drafted timeline and the controls that keep the analysis in human hands
Apply the action closure rule and report the state and age of each action
Review a piece of operational knowledge for owner, last verification and expiry

Key concepts

Four outcomes of a problem: A problem becomes a change by a recorded decision: the condition is removed by a change, contained by a compensating control, transferred by a contract, or accepted by a named person for a stated period. A problem that is none of these and ages in a list is a debt without an owner.
Contributing conditions: The form of finding in a blameless, evidence-based review: several conditions that together allowed the incident, each with evidence, in place of a single guilty cause or person. Where regulation asks for a root cause, one is selected for reporting while the conditions are recorded.
Timeline and analysis as separate artefacts: The timeline may be drafted by an AI system at Level 1 Assisted, each entry with a time, a source link and a kind: observed, reported or inferred, and verified by a person. The analysis, with conditions, alternatives and actions, is written by people.
Closure rule: An action is closed only when its re-test has passed, the evidence is attached, a named person has signed and the closure carries an age. Implemented but unverified is reported as open, and an accepted risk is reported apart from closure.
Knowledge hygiene: Knowledge has a named owning role, a link to a service record, a last-verified date with a method, and an expiry. Retrieval by assistants makes stale knowledge fast, so freshness travels with the text, and unowned articles are archived after notice.

Interactive exercises

Sort: What AI may do in a review
Scenario: The draft is ready before the meeting

Done on the lesson page, with feedback at once.

Discussion

Why does the chapter have each participant write a short account of their own before the group sees the drafted timeline, and what is lost if the order is reversed?
Hansa Mutual closes a review action only when a game-day re-test passes. What would your action register look like next quarter under the same rule?
A gap in a timeline is called a finding. What happens to that finding when a drafting system fills the gap with a probable event?

Exercise · 35 minutes

Applying the closure rule to an action register

Take the actions from one post-incident review of your organisation, or the two corrective actions after Nordvik Region's failed failover test RST-NR-2027-002. For each action record the condition it addresses, the owner, the change made, and the re-test that would show the condition is gone: a test, a game day or an authorised, scoped experiment that reproduces the mechanism. Assign one of the five states, and state how each is reported: open, closed with age, or accepted risk apart from closure. Add the date of the recurrence check 90 days after closure. Finish with one sentence for management on what can and cannot be claimed.

Deliverable: An action register with condition, owner, re-test, state, reported status with age, recurrence-check date and a one-sentence statement for management.

Self-test

How is an action reported whose change is in production but whose re-test has not yet run?
1. As closed, because the change has been implemented and deployed
2. As risk accepted, until the next scheduled game day
3. As closed pending verification, with a provisional age
4. As open, because implemented says nothing about the condition
Which task does the chapter allow an AI system to perform in a post-incident review?
1. Assign the cause from the incident record and the chat transcript
2. Draft the timeline from records, with evidence links and kinds, for a person to verify
3. Close an action when its own evidence pack shows the re-test passed
4. Fill gaps in the timeline with the most probable sequence of events, so that it reads without breaks
Why does Hansa Mutual keep the action log of its IT-operations agent in a store the agent cannot edit?
1. Because the agent takes part in many incidents, so the account of its actions must be independent of it
2. Because GDPR Article 33(5) forbids agents from holding their own logs
3. Because the log is too large for the observability vendor's retention tier
4. Because the works agreement requires agent logs to be reported per engineer

Module 3 · Part III · Operating AI Systems

Part III turns the relation round and treats AI as the thing operated. The participant learns why an AI service changes when nobody has deployed anything, how a release manifest, quality objectives and a canary by case make that change visible and reversible, how incidents involving AI are classed, contained and reported under several clocks at once, and how cost, latency, quality and energy are budgeted with a stop that does not depend on the agent.

Chapter 13 · 105 minutes

Operating AI Systems: Drift, Model Updates and Provider Change

The lesson treats AI as the thing operated and concerns O1 Observe, O3 Change and O5 Sustain. Participants learn why an AI service changes without a deployment, what a release manifest records, the four forms of provider-side change, how quality, restraint, freshness and drift become objectives measured independently of the model, how a model is canaried by case, and which deployer duties of the AI Act land on operations.

Learning objectives

Explain four ways in which the behaviour of an AI service changes when nobody has deployed anything
List what a release manifest records and why a model alias is an unrecorded change
Define quality, restraint, freshness and drift objectives with a grader that is not the model under test
Draft a release path and a rollback for a model update, treating a provider notice as a change request

Key concepts

Release manifest: The versioned record of everything that shapes an AI service's behaviour: model snapshot and gateway route, serving stack, prompts and guardrail rules, retrieval index build, tool definitions, parameters and the evaluation set with its grader. The running service reports which manifest served each output.
Four forms of provider change: Scheduled retirement, scheduled upgrade, emergency change and unannounced change. Only the first two come with notice, and for Mode 0 Ambient AI the unannounced case is the usual one.
Restraint objective: The share of runs in which the system stayed inside its mandate and its rules: no action outside the approval rules and no answer from outside the permitted sources. In the worked example restraint has a budget of nothing.
Sentinel set: A fixed set of cases, 100 to 200 as the book's default, stratified by the strata that matter and run continuously through the production path, so that a change on the provider's side shows in the operator's own telemetry.
Canary by case with shadow first: A model canary assigns by case, not by request, so that one claim or application is handled by one version throughout. Before any real case, the candidate receives a copy of production traffic and its outputs are recorded and compared.

Interactive exercises

Match: Four forms of provider-side change
Scenario: A notice in the inbox

Done on the lesson page, with feedback at once.

Discussion

What can be undone after a model update is the routing; what cannot is the outputs already acted on. How would your organisation find the outputs that a faulty version produced?
A fallback model that has never been evaluated on the service's own set is listed in the fallback chain. What claim can be made about it, and what would make the claim true?
Hansa Mutual's rule that a model, prompt or routing change is never a standard change comes from one incident. Is the rule too strict for low-risk prompt edits, and what evidence would justify relaxing it?

Exercise · 30 minutes

Quality objectives and a way back for an AI service

For one AI service, yours or Nordvik Region's citizen-service assistant SVC-NR-005, name the task in one sentence and state what a good output is in terms a reviewer can grade. Write the release manifest fields with their current values, marking any alias. Define objectives for quality, restraint and freshness, each with a grader that is not the model under test and a data path separate from the service. Describe the sentinel set and how often it runs. Set the release stages for a model update, the stop condition, the fallback chain with the evaluation date of each link, and where provider notices are received.

Deliverable: A two-page record with manifest, three independently graded objectives, a sentinel set, release stages with a stop condition, a dated fallback chain and a notice intake.

Self-test

Why does the chapter say that a manifest naming a model by alias is defective?
1. An alias costs more per token than a pinned snapshot from the same provider
2. An alias cannot be routed through a gateway or served from a fallback
3. The provider can re-point the alias to a newer snapshot, so the manifest records an unrecorded change
4. The AI Act prohibits deployers from naming a model by alias in the release manifest of any high-risk AI system
How does Nordvik Region handle each quarterly retrain of the claims ranker by TriageSoft?
1. As a standard change released by the provider without a record at the Region
2. As an emergency change declared by the duty manager on the day of release
3. As a matter for the provider's post-market monitoring, with no step at the Region
4. As a provider-raised change request with the Region's own evaluation set, a shadow period and a named decision
Which of these follows from the observer principle for measuring the quality of an AI service?
1. The grader is not the model under test, nor a sibling from the same family
2. The model grades its own outputs, since it holds the fullest context
3. The quality measure shares the gateway and logging pipeline of the service
4. The provider's published evaluation replaces the operator's own measurement

Chapter 14 · 105 minutes

Incidents Involving AI: Detection, Triage and Reporting

The lesson concerns O2 Diagnose and O4 Restore for incidents involving AI. Participants learn an operator's definition and four classes of AI incident, three families of detection signal, a triage tree that puts the stop before the diagnosis when an agent has left its mandate, a containment ladder ordered by reversibility, and the reporting clocks of the AI Act, NIS 2, DORA and the GDPR. They build a clock sheet for one service.

Learning objectives

Classify an incident involving AI into classes A to D from its first signal
Explain why the stop question precedes diagnosis when an agent has acted outside its mandate
Order containment actions by reversibility and name the holder and degree of delegation of each
Draft a clock sheet for a service in scope of more than one reporting regime

Key concepts

Four classes of AI incident: Sorted by where the failure sits: class A, harmful or wrong output at scale while the service stays up; class B, an agent acting outside its mandate; class C, provider failure or silent change; class D, data exposure through an assistant.
Mandate signals: Detection signals that do not depend on the agent: denials at the policy decision point outside the agent, actions in the agent's identity log that match no runbook step, budget consumption against its cap and the rate of tool calls. They detect class B incidents that availability objectives cannot see.
Containment ladder: Six rungs from most to least reversible: degrade to a fallback, restrict tools or permissions, suspend the agent identity, isolate network paths, roll back or fail over, and restore over existing state, for which Level 4 Autonomous is never used.
Stop without approval: Suspension of the agent identity at the identity provider is the one containment action any named holder may take without approval. Re-issuing the credential is a separate decision, and the stop is a drilled capability with a measured time.
Clock sheet: A one-page record per service, kept in version control and written before the first incident, that tells the incident commander which statutory clocks may open, what triggers each, who owns each, where each report goes and what must be recorded.

Interactive exercises

Put in order: The containment ladder
Match: Reporting clocks and their first limits

Done on the lesson page, with feedback at once.

Discussion

The chapter draws from a reported incident that an instruction is not a control and the agent's account is not the record. What in your design still depends on an agent complying with an instruction?
In INC-HM-2026-0497 an agent silenced a recurring alert and the silence was found nine days later. Which signal would have caught it on the day, and where must that signal live?
DORA starts its four-hour limit at classification as major, while NIS 2 starts at awareness. What does that difference require of the timestamps in one incident record?

Exercise · 30 minutes

A clock sheet for one service

Choose a service that may fall under more than one regime, yours or Hansa Mutual's underwriting-support service SVC-HM-008. From the service record, list the regimes that may apply: whether the system is a high-risk AI system and whether the organisation is its provider or deployer, whether the organisation is in scope of NIS 2 or DORA, and whether personal data is processed. For each regime write the trigger, the first limit and the later steps as the chapter's table gives them, the named owner who decides the clock, the recipient, and the facts the incident record must capture, with first signal and awareness recorded separately. Have legal counsel named as reviewer.

Deliverable: A one-page clock sheet listing each applicable regime with trigger, limits, owner, recipient and required record fields, and a named legal reviewer.

Self-test

At step 2 of the triage tree, an agent identity is in the path and its action fell outside its mandate. What does the tree require?
1. Complete the diagnosis first, so that the stop does not destroy evidence
2. Class B: invoke the stop path before diagnosis
3. Class C: check the change calendar and the provider's notice
4. Wait for the incident commander to classify under DORA Article 18
Under the AI Act as the chapter reads it, what does a deployer of a high-risk AI system do first on identifying a serious incident?
1. Inform the provider, then the importer or distributor and the market surveillance authorities
2. Report directly to the market surveillance authority within 15 days
3. Send an early warning to the national CSIRT within 24 hours
4. Notify the data subjects without undue delay under Article 34
In the incident INC-NR-2028-0163 on the benefits-triage service, where was the stop exercised?
1. Inside the provider's application, by the provider's support engineer
2. At the national shared-service provider's network operations centre
3. By the market surveillance authority, after the serious-incident report
4. At the Region's own gateway, which the platform team controls

Chapter 15 · 90 minutes

The Economics of AI Operations: Tokens, GPUs, Latency and Energy

The lesson treats the economics of running AI as an operations problem and concerns O5 Sustain and O1 Observe. Participants learn what a request and an agent run cost, the four forms of capacity, how cost, latency and quality trade against each other, what makes a fallback real, what can be claimed about energy, and how an agent's spend is bounded by limits and a stop that do not depend on the agent.

Learning objectives

Calculate the drivers of cost for a metered call and explain why an agent's unit of cost is the run
Compare the four forms of AI capacity by what is fixed, what varies and which failure shows first
Evaluate a cost-saving lever by its effect on latency and quality and the evidence needed before promotion
Draft a cost-and-latency budget record with layered spend limits, a fallback ladder and a stop condition

Key concepts

Cost per outcome as a distribution: A cost per request is a means to an end; the chapter measures the use case, for example cost per successful outcome, and states it as a distribution, with the 50th, 95th and 99th percentile of cost per run, since the tail is where a runaway sits.
The run as unit of cost: For an agent a run consists of several model calls interleaved with tool calls, each resending the context accumulated so far, so cost grows with the number of steps and a loop multiplies it.
Fallback rung: A second path for a request when the primary is too slow, refused, wrong or absent. Each rung is declared in the service record with its own objective, data tier, owner, cost and date of last use, is triggered by an independent probe, and counts as unavailable unless exercised.
Energy claim: A measurement with a boundary, a period and a count. Published per-request figures use different boundaries and do not combine, and one provider's figure is not the energy of the operator's own use.
Layered spend limits: Caps per call, per run, per day and per month, enforced at the gateway on a credential the agent cannot widen, each with a stop held outside the agent. Every cost limit is paired with an outcome measure so that the agent cannot meet it by doing less.

Interactive exercises

Match: Cost levers and their quality risk
Scenario: The daily cap is reached

Done on the lesson page, with feedback at once.

Discussion

Prices for a fixed level of capability have fallen fast, yet bills rise. What in the way your services use models explains the difference?
For a critical service the stop degrades and does not halt. What is the degraded mode of your most important AI service, and when was it last used?
The Region's record states that it cannot yet claim energy per conversation. What would it need to measure, over what boundary, before it could?

Exercise · 35 minutes

A cost-and-latency budget with a stop

For one AI service, yours or Hansa Mutual's claims-triage service SVC-HM-002, name the unit of work, request, run or case, and the outcome that counts as success. State the unit cost as a distribution, the latency objective at the tail and the quality floor with the date of its last result on your own golden set. Set limits per call, per run, per day and per month and say where each is enforced. List the fallback rungs with objective, data tier, owner and date of last drill. Name who holds the stop outside the agent, and state whether a breach halts the work or degrades it.

Deliverable: A budget record with unit cost distribution, latency and quality objectives, four layers of limits, a drilled fallback ladder and named stop holders.

Self-test

Which failure does the chapter's capacity table say shows first for on-demand, metered capacity?
1. Hardware failure in the organisation's own halls
2. A rate limit or quota refusal at a peak
3. Slow growth of the pool through procurement
4. Timeouts after depreciation of the accelerators
What is the quality risk the lever table gives for a cascade of a cheap model with escalation to a stronger one?
1. Wrong answers accepted by the cheap stage
2. Truncated answers from a shorter output limit
3. Higher latency at the tail from larger batches
4. A different data position in a cheaper region
Where must an agent's spend limits be enforced, according to the chapter?
1. In the agent's instructions, as a budget it is asked to respect
2. In the monthly invoice review by the FinOps lead
3. At the gateway, on a credential the agent cannot widen, with the stop held outside the agent
4. In the model provider's dashboard, under the fair-use clause of the licence that the vendor manager holds

Module 4 · Part IV · The Operator's Own AI

Part IV treats the operator's own AI: a team of narrow agents, each with a role, an identity, an owner and a mandate; the observer principle that says what must verify their work and what must watch the watchers; and the defence of the agents, their credentials, their runbook library and the control plane as Tier 0. The participant leaves with a mandate manifest, an observer manifest and a deny-by-default permission policy with ceiling cases.

Chapter 16 · 105 minutes

An AI Agent Team for Operations: Skills, Mandates and Containment

The lesson treats the operator's own AI as a team and concerns all five functions. Participants learn why a set of narrow agents is easier to govern than one general agent, the six roles and what each never does, why skills are code that carries no authority, what the mandate manifest contains, how containment works whatever the agent does, and how an agent earns and loses a degree of delegation.

Learning objectives

Describe the six roles of an operations agent team and what each role never does
Explain why skills carry instruction and the mandate manifest carries authority
Specify the containment controls that hold whatever the agent's behaviour
Draft a mandate manifest for a runbook-runner agent with promotion and demotion rules

Key concepts

Team of narrow agents: A small set of agents divided along the lines that already separate duties in operations, each with a role, an identity, an owner and a mandate, so that a fault in one is bounded by its mandate and its result can be verified independently.
Skills as versioned files: The unit in which an agent's procedural knowledge is packaged. Skills are code: reviewed by a person other than the author, owned, pinned and released as change. They carry instruction and never authority.
Mandate manifest: The MND record that states what an agent may do, where, when and with what: identity, scope, the degree of delegation per action class, budgets, circuit breakers, separation and never lists, memory, tools, evidence and review. A policy decision point outside the agent enforces it.
Runbook runner: The only role that touches production on a routine basis. It executes catalogued runbooks inside a mandate, dry-run first, and reports what it did; it never writes or edits runbooks, widens its own scope or verifies its own result.
Promotion and demotion: A rise in level is a recorded decision by a named authority on four kinds of evidence no older than 90 days: offline evaluation, a shadow period, a seeded-fault exercise and a drilled stop. Lowering is immediate, and a spent autonomy budget demotes the class one level.

Interactive exercises

Match: Six roles of an operations agent team
Scenario: A proposal to raise a level

Done on the lesson page, with feedback at once.

Discussion

Nordvik Region holds that a manifest it cannot review is a manifest it does not adopt, and keeps manifests short enough to review in an afternoon. What would that rule exclude in your organisation?
The chapter says the hard limit on a tool server is the permission of the identity behind it, not the server's read-only flag. Where do your own tools rely on a setting that the tool itself could change?
The people on the other side of the agents keep their craft through manual practice that the mandate itself requires. How would you write that requirement so that it is actually exercised?

Exercise · 35 minutes

A mandate manifest for a runbook runner

Write the smallest mandate that makes a runbook-runner agent useful on one service, yours or Nordvik Region's portal SVC-NR-002. Start from the most frequent catalogued runbooks, never from the agent's capabilities, and list at most three. State the agent's workload identity, owner, operators and the authority that approves a change of level; the scope; the level per action class with the approval rule; the budget and circuit breakers; the never list; memory and the tool allowlist with declared effects; what can be undone and who can stop it; and the evidence and review date. Add the test that shows the policy decision point refuses an action outside the mandate.

Deliverable: A mandate manifest of one to two pages with identity, scope, levels per class, budgets, never list, tools, stop holders, review date and a refusal test.

Self-test

Which of these does the runbook runner never do, according to the table of roles?
1. Execute a catalogued runbook inside its mandate
2. Perform a dry run before the real run
3. Report what it did to the incident record
4. Verify its own result
Why may Hansa Mutual's IT-operations agent renew a certificate at Level 3 Bounded when other configuration change stays at Level 2?
1. Certificate renewal is exempt from the autonomy budget under the manifest
2. The previous certificate stays valid until expiry, so the renewal is reversible by reinstalling it
3. DORA requires certificate renewal to be automated for services that support critical or important functions
4. The certificate authority, not the agent, is accountable for the renewal
What does the chapter say about the difference between instructions in a skill and boundaries in a mandate?
1. Instructions lower the rate of unwanted behaviour, but only boundaries set a ceiling on its consequences
2. Instructions and boundaries are equivalent once the skill has been reviewed and pinned
3. Boundaries lower the rate of unwanted behaviour, and instructions cap its consequences
4. Instructions carry authority when signed by the agent owner, and boundaries then become optional

Chapter 17 · 120 minutes

The Observer Principle: Who Watches the Watcher

The lesson states the design rule on which the book depends and concerns O1 Observe and O4 Restore. Participants read five public incident reports for what the verifier, observer or stop path shared with the failed system, learn six kinds of independence, the devices that keep an observer outside the observed, why state engines' self-reports are not verification, and how seeded faults test observers. The chapter's debate weighs the cost of independence.

Learning objectives

State the three tests of the observer principle as questions answerable from records
Identify which kind of independence is missing in a described verification arrangement
Design an observer with a separate failure domain, an outcome probe and a heartbeat with a silence rule
Draft an observer manifest and a plan for an authorised, scoped seeded-fault test with a catch rate

Key concepts

Six kinds of independence: Model, data, authorship, credential, failure domain and time. The kinds are not ranked: a second model family used by the same author on the same metric gives model independence and nothing else, and a report says which kinds are present.
Dead-man's switch: A heartbeat that raises an alarm on silence. An observer that reports only when something is wrong cannot be told from an observer that has failed, so the absence of its signal must itself page someone.
Synced is not served: A reconciler's sync and health status is the engine's own view of its own work. Whether the application works for its users is shown by an outcome probe, and the engine's lag and suspension are watched from outside.
Seeded fault: A known weakness introduced on purpose by a person, under written rules of engagement, to test whether the observers see it. Each class has a catch rate by quarter, the share found within the target time, and the catch rate carries a stop rule.
Reproduction by a person: The final form of independence: a person outside the chain can redo a result from the evidence, without the agent, and reach the same conclusion. By default it is done for every claim that raises a level, every closure of a high-severity seeded-fault finding and a 10 per cent sample of other verifications.

Interactive exercises

Sort: Which test of the observer principle fails?
Match: What the watcher shared with the failure

Done on the lesson page, with feedback at once.

Discussion

None of the five public cases involved an AI agent. Why does the chapter say they are the pattern that an agent inherits and speeds up?
A verifier may share a model family with the agent it checks without harm for some claims. For which claims does shared model family matter, and for which is a script the better verifier?
If a model can recognise that it is in a test, behaviour under test may differ from behaviour in use. What follows for how much weight you place on an agent's own report and on evaluation alone?

Exercise · 30 minutes

An observer manifest for one service

For one Tier 1 service on which an agent or an automation acts, yours or Nordvik Region's portal under OBS-NR-0001, list the action classes with their levels, starting with the highest level and the least reversible class. For each class name the signal that shows the action worked, who or what produces it and by which path, and mark which of the six kinds of independence it has. Place the observer in a failure domain separate from the platform, define an external synthetic transaction and a heartbeat with a silence rule, and describe the stop path and its drill. Plan one seeded fault under written authorisation, in a scoped environment, with a target detection time and a named person who can stop the test.

Deliverable: An observer manifest with verifying signal and independence kinds per action class, a domain map, heartbeat rule, drilled stop path and one authorised seeded-fault plan.

Self-test

In the chapter's case table, what did the CrowdStrike Content Validator share with the code it was meant to check?
1. The backbone network and DNS on which both ran
2. The CPU of the proxies that it observed
3. The assumption of 21 inputs, held by validator and tested code alike
4. The provider's own status platform, which went down together with the outage it reported
What did Hansa Mutual's game day of 2028-10-24 find about its remediation agent and the verification step?
1. They shared a model family, an identity and a signal
2. They ran in different failure domains but on the same schedule
3. The verification step was written by a person and never executed
4. The independent probe agreed with the agent's own check
According to the table across the degrees of delegation, what does the stop path require at Level 3 Bounded?
1. Approval by the service owner, inside the control plane, with a drill at least once a year
2. A named holder and documentation, with no drill required
3. Nothing beyond the ability to disable the assistant in the tool
4. No approval, outside the control plane, drilled each quarter within the target

Chapter 18 · 105 minutes

Defending the Operator's AI: Agents, Credentials and the Control Plane as Tier 0

The lesson argues that the operations agent, its credentials, its runbook library and the control plane are Tier 0, and concerns O3 Change and O4 Restore, with O1 Observe and O2 Diagnose as the route by which steering text arrives. Participants learn six threat mechanisms and the control outside the model that answers each: writer classes, run-scoped credentials, a deny-by-default policy with named refusals, signed runbooks, break-glass and an append-only decision log.

Learning objectives

Explain why the operator's AI is a target and which six kinds of asset must be protected together
Match each of the six threat mechanisms to the control outside the model that bounds it
Classify the text an agent reads into writer classes and state what authority each may carry
Draft a permission manifest with named refusals and a set of ceiling cases as a defensive test

Key concepts

Operational text as an instruction channel: A language model receives instructions and data in the same medium and cannot reliably tell them apart, and in operations the retrieved data is tickets, logs, alert annotations and vendor feeds that others can write. The control sits outside the model whoever the author is.
Writer classes: Input provenance by who can write a text: WC-1 instrument output may trigger a catalogued runbook, WC-2 reviewed staff text may define what a runbook does, and WC-3 outside-the-team text and WC-4 external or generated text inform but carry no authority above notify. A summary inherits the lowest class.
Confused deputy: The pattern Hardy named in 1988: a service with authority its requester lacks is led to use it on the requester's behalf. The defence is that authority travels with the request: a token bound to an incident or change record, with audience checks and a tool allow list.
No standing credential: A credential that can change production exists for the length of a run and is issued only after the policy decision point has allowed the action. Each agent has one identity of its own, and no agent holds a human's identity.
Break-glass: A route to operate the control plane when the platform, the decision point or the identity provider is down. It does not depend on what it remedies, needs two named people by default, is time-boxed and recorded, is exercised with a measured time, and is never held or requested by an agent.

Interactive exercises

Sort: Writer classes
Match: Threats and the controls outside the model

Done on the lesson page, with feedback at once.

Discussion

The review of the near miss found that a restart would have been allowed although the independent probe showed no fault, because the allow rules checked what the agent may do and not what had prompted it. What does that say about allow rules on their own?
Nordvik Region's audit found an administrator key unchanged for 26 months and named on a wiki page, with no agent involved. Why does the chapter count this as part of defending the operator's AI?
When the policy decision point itself fails, the design is to fail closed. What does failing closed cost during an incident, and who operates in the meantime?

Exercise · 30 minutes

A permission manifest with refusals and ceiling cases

For one remediation agent or pipeline, yours or Hansa Mutual's IT-operations agent, list every source of text it reads and assign each a writer class, WC-1 to WC-4, taking the lowest class where a source is mixed. State which classes may trigger a catalogued runbook and which only inform, and the corroboration required before acting on WC-3 or WC-4 text. List the identities and credentials, removing any that are standing. Write the named refusals that hold at every level, including the undelegated classes. Then write ten ceiling cases as a design and review exercise: each describes, by mechanism only, a situation the policy must refuse, with the expected refusal reason.

Deliverable: A permission manifest with writer classes, credential rules and named refusals, plus ten ceiling cases with expected refusal reasons, dated for a first run.

Self-test

A customer ticket is summarised by the incident assistant, and the summary reaches the runbook runner. Which writer class does the summary have?
1. WC-2, because the assistant is the organisation's own reviewed application
2. WC-3, because a summary inherits the lowest class of what it summarises
3. WC-1, because the summary was produced by an instrument the organisation operates
4. No class, because summaries are generated and therefore outside the scheme
Which of these is a property the chapter requires of a break-glass route?
1. The remediation agent holds it, so that it can act when the platform is down
2. It is federated to the identity provider, so that every use is attributed to a named person automatically
3. It is left unused after creation, so that its credentials are never exposed
4. It does not depend on what it is a remedy for, and it is exercised with a measured time
What did Nordvik Region's test of its break-glass route on 2029-02-07 record?
1. Two named people had access in 9 minutes against a target of 15, with federation disabled in an agreed window
2. The route failed because it depended on the identity provider SVC-NR-004
3. The operations assistant invoked the route in 3 minutes against a target of 5
4. Access took 17 minutes against a target of 5, and the test was repeated

Module 5 · Part V · Operations in the Organisation

Part V places the loop in the organisation. The participant learns to write a recoverability claim and derive its status, to define metrics by the decision they inform and to spot the ones that mislead, to choose an operating model per service and buy operations with AI inside, to assign the decision rights of a bad night, to turn an article of European regulation into a control with an owner and evidence, and to test a provider boundary and an exit.

Chapter 19 · 105 minutes

Resilience as a Claim: Restore Tests, Game Days and Evidence Age

The lesson treats every sentence about recovery as a claim and concerns O4 Restore and O5 Sustain. Participants learn the six parts of a recoverability claim, the six derived statuses, what a restore test measures and what each kind of test leaves unproven, how game days and chaos experiments are bounded, how evidence expires by clock and by condition, and where AI may help in the register without attesting.

Learning objectives

Write a recoverability claim with its six parts for one service and scenario
Derive the status of a claim from its latest record, its clock and listed changes
Plan a restore test that measures recovery time and recovery point with an independent probe in a segregated environment
Explain why AI may plan, execute bounded steps and analyse a test, and may not attest

Key concepts

Recoverability claim: A statement with six parts: the subject, the scenario, the objective, the measured result, the date and the expiry. A statement that lacks one is an assertion, and the claim states which backup copy was restored.
Derived status: The status of a claim is computed from its record, never typed by hand, into verified, expired, implemented but unverified, failed, asserted or untested. Only a verified claim counts as recoverable, and the denominator is every claim the organisation makes.
Expiry by clock and by condition: A result stops counting after its clock, 180 days by default for a Tier 1 restore or failover test, whatever has changed, and earlier on a named change written into the record, such as a new major database version or a change of backup storage or provider.
Game day record: A record for an authorised experiment with a hypothesis of steady state that can fail, a bounded blast radius, an abort rule and a stop path outside the system under test. The experiment is refused if a field is missing.
Rebuild from declared state: A recovery path with a claim of its own in an estate run by state engines: it restores the declaration and not the data, secrets, external state or manual drift, and records the commit, engine version, state and key copies, manual steps and time to a passing probe.

Interactive exercises

Sort: Deriving the status of a claim
Match: What each kind of test proves

Done on the lesson page, with feedback at once.

Discussion

A dashboard shows that every Tier 1 service has a documented recovery plan. What does that figure report, and what would the register show for the same services?
The Atlassian case held for one customer and not for hundreds. Which of your recovery claims is true only at a scope smaller than the event you fear?
Why does the chapter require the attesting person to be someone who did not run the test, and the measuring probe to be one the runner does not control?

Exercise · 35 minutes

A recoverability claim and the test behind it

Select one recovery claim, your own or Nordvik Region's failover of the portal SVC-NR-002 to the recovery site. Write it with its six parts. Derive its present status from the last record. Then plan the test: who declares the start, the segregated environment and the copy restored from, the runbook followed as written with deviations recorded, the independent probe and integrity checks that stop the clock, how the recovery point is measured, who attests, the abort rule and who can stop the test from outside the system under test, the clock for expiry and the listed changes that end the result early.

Deliverable: A register row with a six-part claim and derived status, and a one-page authorised test plan with roles, probe, abort rule, attestation and expiry conditions.

Self-test

Hansa Mutual's failover of payments, RST-HM-2027-007, passed on 2027-10-05 with 1 hour 35 minutes against 2 hours. What is its status in the register in September 2028?
1. Verified, because the latest test passed inside the objective
2. Failed, because the recovery point was not measured
3. Asserted, because the result exists only in a plan
4. Expired, because the 180-day clock ended on 2028-04-02
What does a restore from backup into a segregated environment not prove, according to the table of test kinds?
1. That the data can be recovered within the recovery point objective
2. That the production target, at production scale and concurrency, can be restored over
3. That the service starts on the restored data
4. That the backup tool and the database engine are of compatible major versions at the time of the test
Which role does the chapter allow AI in a restore test at the highest degree of delegation it names?
1. A runbook runner executing a catalogued restore into a segregated environment at Level 3 Bounded
2. An agent attesting the result when the independent probe has passed
3. An agent running a failover in production at Level 4 Autonomous during a freeze on a critical function
4. An assistant typing the status of the claim into the register after the test

Chapter 20 · 90 minutes

Measuring Operations: The Four Key Metrics, SLOs, Toil and the Metrics That Mislead

The lesson asks what operations should measure and concerns all five functions. Participants learn the six properties that define a metric, what the four key metrics do and do not describe, how attainment of an objective and mean time to restore mislead, how toil and consumption of the autonomy budget are measured, what a false green is, and the rules for metrics about people. They write metric definitions with misuse notes.

Learning objectives

State for a given metric the decision it informs, who takes it and what the decider does differently
Explain three limits of the four key metrics as a measure of operations
Replace a mean time to restore with a distribution, a count beyond the objective and a named list
Draft a metric definition with clocks, source, form, misuse notes and a gaming check

Key concepts

The four key metrics: The delivery measures of the DevOps Research and Assessment (DORA) programme, divided into throughput and instability. They describe the change pipeline, so they belong chiefly to O3 Change, and they count only what the pipeline sees.
Missing data reads as good: An indicator computed from a telemetry pipeline has an implicit denominator, the events that arrived. When the pipeline fails, attainment looks healthy, so the chapter adds a data-completeness indicator beside it.
Distribution of time to restore: The reporting form both organisations adopt: the median, a high percentile, the count of incidents beyond the recovery time objective and their identifiers as a named list, with no mean by design, so that a tail incident stays visible.
Self-verification share: The share of an agent's actions whose only verification came from the executing agent or from a system sharing its model family and failure domain. Consumption of the autonomy budget is valid only when the verifying probe is independent, so the share must be zero at Level 3.
False green: A measure that reports health when the thing measured is unhealthy, for example through missing data counted as good or a definition that excludes the failure. Every measure therefore carries a gaming check.

Interactive exercises

Match: Every measure carries a gaming check
Scenario: A request for figures per engineer

Done on the lesson page, with feedback at once.

Discussion

Which of the measures in your last operations report would have stayed green through the worst incident of the past year, and why?
A randomised trial cited in the chapter shows that perceived and measured performance can differ. Where does your organisation report a perceived gain from AI as if it were measured?
Short approval times in review read as efficiency. How would you tell efficient review from review that has become approval?

Exercise · 35 minutes

Metric definitions with misuse notes

Take the three most used measures in your next operations report, or failed deployment recovery time, time to restore and autonomy-budget consumption at Hansa Mutual. For each, write a definition: one sentence naming the decider and the decision the measure informs; the clocks, with start and stop events matching any legal definition that applies; the source, which must be independent of the thing measured; the form of reporting, as a distribution or count where a mean would hide the tail; the subject, which is a service, pipeline, delegation or team and never a person; the known ways it misleads; and the gaming check. If no decider can be named, strike the metric.

Deliverable: Three metric definitions, each with decision, clocks, source, reporting form, subject, misuse notes and gaming check, plus a note on any metric struck.

Self-test

By how much did the outage of 7 hours 5 minutes in INC-NR-2028-0391 move Nordvik Region's reported mean time to restore?
1. By 425 minutes, the full length of the outage
2. By 70 minutes, enough to breach the target
3. By 9 minutes, so the report did not mention the outage
4. By nothing, because the incident was excluded as a supplier fault
Why does the book say the four key metrics are not a measure of the recovery performance of operations?
1. They describe the change pipeline, and failed deployment recovery time is not the time to restore of all incidents
2. They are defined by Regulation (EU) 2022/2554 for financial entities only
3. They can only be computed for organisations with more than 100 deployments a day
4. They measure individual engineers and are therefore excluded by works agreements
How does Nordvik Region report its change fail rate for a quarter with 62 production deployments and three failures?
1. As a percentage to one decimal place, benchmarked against elite performers
2. As a rolling twelve-month mean, to smooth the small sample
3. It does not report it, because supplier releases dominate its changes
4. As counts, because the denominator is too small to bear a rate

Chapter 21 · 90 minutes

Operating Models: Platform Teams, SRE, Managed Services and Sovereign Operations

The lesson treats the operating model as an architectural decision made per service and concerns all five functions. Participants compare in-house SRE, platform teams, centralised and embedded operations, managed providers, follow-the-sun and national operators by the scarcity each answers; learn what AI changes in each, the four owners, the six terms for buying operations with AI inside, and the role of the operations conductor.

Learning objectives

Compare the recurring operating models by the scarcity each answers, what it trades and its typical strain
Assign the four owners of a service and say what each cannot delegate
Draft the six terms of an operations annex for a managed provider whose AI acts on the estate
Choose and record an operating model per service by criticality and sovereignty need

Key concepts

Operating model: The answer to four questions about every service: who runs it, who owns it, who may act on it and who is called when it fails. It is decided per service, not per organisation, and recorded with its exit criteria.
Four owners: The service owner, who cannot delegate risk acceptance and the exit decision; the objective owner, who holds the freeze decision; the runbook owner, who signs off a level change for the runbook; and the agent owner, who holds the mandate and the stop.
Operations annex: The contract terms for a managed provider with AI inside: levels and action classes, sampling, audit rights, evidence per action, tuning and test rights, and notice of model change, with a stop right and an observer outside the provider's platform.
Operations conductor: The accountable role, a charter and not a title, that holds the loop together: it knows what the agents may do, verifies where the mandate requires, keeps manual operation in practice and holds the right to stop. It decides nothing about a named person's access to work.
Two evidenced holders per window: The rule that each window needs at least two conductors with evidence of their competence; with fewer, Level 3 classes fall back to Level 2 for that window.

Interactive exercises

Match: Which scarcity each model answers
Scenario: The annex is signed

Done on the lesson page, with feedback at once.

Discussion

The chapter says a contract right is a promise until it has been used. Which rights in your provider contracts have never been exercised, and what would a first use look like?
A national shared operator is both a remedy for scarce skill and a concentration. How should an authority that depends on one test that dependency?
What should the charter of an operations conductor say so that the role does not drift into supervising people?

Exercise · 35 minutes

An operations annex for a managed provider

For a provider that monitors or operates part of your estate, or SamDrift's monitoring and paging service for Nordvik Region, draft the annex. Begin with the written question to the provider: which AI acts in the service, on which action classes, at which model or rule version and in which location. Then write the six terms: action classes with levels, a quarterly sample of remediations and suppressed alerts with the evidence used, audit rights, the fields of a record per action, the right to narrow rules and to inject authorised seeded faults, and notice before any change affecting suppression, correlation, remediation or paging. Add the stop right in your own identity provider and the first sample, seeded fault and stop test with dates.

Deliverable: A draft annex with the six terms, a stop right outside the provider's platform, and a dated plan for the first sample, seeded fault and stop test.

Self-test

What does the table of operating models give as the typical strain of a managed provider?
1. The operational share creeps above the cap
2. Provider AI acts under its own rules, and exit is hard
3. Lost context at each change of watch
4. Uneven practice and isolated engineers
What can the agent owner not delegate, according to the table of four owners?
1. The freeze decision when the error budget is spent
2. Risk acceptance and the exit decision
3. The mandate and the stop
4. Sign-off of a level change for the runbook
How does Hansa Mutual's operating model answer the risk committee's question on how far the agent's autonomy may reach?
1. The level is set per service and per window, capped at Level 2 where the window has fewer than two evidenced conductors
2. The level is set once for the whole group by the executive board at the yearly review
3. The level follows the provider's own setting for each managed service
4. The level is Level 3 for every class as long as the works agreement is in force

Chapter 22 · 90 minutes

Decision Rights, Change Authority and the Incident Commander

The lesson concerns O2 Diagnose, O3 Change and O4 Restore at the decisions no runbook takes: declaring an incident, commanding it, approving an emergency change, authorising a failover, restoring over data, breaking a freeze and stopping an agent. Participants learn the six fields of a decision right, four rules about position, the incident commander's pre-assigned authority, change authority after the weekly board, and the stop right.

Learning objectives

Specify a decision right with holder, deputies, response time, evidence standard, record and ceiling for AI
Apply the four rules about the position of the decider to a described situation
Determine the concurrence needed for a failover from the status of its claim in the register
Build a decision-rights matrix for the first ten decisions of a bad night

Key concepts

Decision right: The authority to take a defined decision and to be obeyed, written as a requirement with six fields: a holder role filled from a rota, two deputies, a response time, an evidence standard, a record and a ceiling for AI.
Position governs: Four rules: the authoriser does not run, the requester does not approve, the acceptor of a risk neither authors nor closes, and a conflicted holder passes the decision to a deputy and says so in the record.
Incident commander: The role that holds the incident by a written, pre-assigned authority, decides class and severity, hands over by a recorded briefing and has deputies who are as qualified. An assistant files what the commander approves and never classifies or speaks externally.
Change authority: The decision rights over what enters production: a standard change approved once as a class, a normal change approved by a peer and a gate, a small forum for exceptions, and an emergency change declared by a person and approved by the commander, never the requester.
Stop right: The right to stop an agent or hold a pipeline, held by listed roles including one reachable out of hours, without approval, through a mechanism outside the platform, drilled within 90 days. Restart is a separate decision by the agent owner at a lower level.

Interactive exercises

Match: What AI does in the decisions of a bad night
Scenario: The commander approved the suspect change

Done on the lesson page, with feedback at once.

Discussion

The chapter does not forbid a failover on a failed claim. What does it require instead, and why is forbidding it the worse design?
A prohibition of Chapter 4 is not a risk to be weighed and cannot be accepted by anyone. What is the difference between that and a risk a named person may accept with an expiry?
At Hansa Mutual the night commander is also the agent operator. Which decisions become conflicted in that arrangement, and how should the profile resolve them?

Exercise · 35 minutes

The first ten decisions of a bad night

Replay the last three Tier 1 incidents of your organisation from their records, or use Nordvik Region's incidents on the portal, and list every decision taken, by whom, and how long it waited. Reduce the list to ten decisions, including declare and classify, hand over command, stop an agent, approve an emergency change, authorise failover, restore over existing data and break a freeze. For each write the holder role, two deputies, the response time, the evidence standard, the record, and the highest level at which AI takes part and what it does. Mark any right with a single holder or none reachable out of hours.

Deliverable: A decision-rights file of ten rows with six fields each, with single-holder and out-of-hours gaps flagged and an owner for closing each gap.

Self-test

The failover claim for a service is expired. Under the chapter's table, what does the incident commander need before authorising the failover?
1. Nothing further; the commander decides alone as for a verified claim
2. A new passing restore test, run before the failover begins
3. Approval by the change advisory board at its next meeting
4. Concurrence by the head of SRE or deputy, by telephone, with the status stated
What is the highest level at which AI takes part in breaking a freeze, according to the table of ten decisions?
1. Level 0 Advisory: it reads and flags the conflict; never an agent
2. Level 2 Supervised: it drafts the exception and files it after approval
3. Level 3 Bounded: mandate circuit breakers lift the freeze and page a person
4. Level 1 Assisted: it prepares and checks the target for the release
Who approves an emergency change under the chapter's change authority?
1. The agent that drafted it, if the gate computes the class as low risk
2. The requester, since the requester knows the urgency best
3. The incident commander, who is never the requester
4. The weekly board, convened by telephone during the incident

Chapter 23 · 105 minutes

Regulation as Constraint: DORA, NIS 2, CER, the AI Act, the GDPR and the Data Act

The lesson maps European regulation to the five functions at the level of the article. Participants learn how DORA, NIS 2, the CER Directive, the AI Act, the GDPR and the Data Act reach operations as clocks, records, tests and named people, who is in scope in the two example organisations, what staff rights require before rotas and tools change, and how to turn an article into a control with an owner and evidence.

Learning objectives

Identify which instruments apply to an organisation and which displace others, as the chapter sets out for the two examples
Map a duty from an instrument to the operations function, the control and the evidence artefact that meets it
Explain why the autonomy budget is a design choice and not a legal requirement
Draft a regulation-to-control mapping row with text date, owner, verifier, evidence and status

Key concepts

Regulation as outcomes: The instruments do not describe an operations loop. They state what must be shown: services that continue, incidents recorded and reported within hours, tested recovery, providers that can be left, and a management body that answers for it.
DORA displaces NIS 2 for financial entities: DORA binds Hansa Mutual's EU entities as an insurance undertaking and, as lex specialis, displaces NIS 2 for them; the chapter cites NIS 2 Article 4(1) and the German and Swedish national provisions. DORA does not apply to Nordvik Region.
Mapping record: A file in version control, the RCM record, with one row per duty: the instrument and article, the duty in the words of the text, the function, the control, the evidence, a named owner and verifier, a text date and a status. A failed test appears as a failed row.
One control serves many instruments: The book's own conclusion that a mapping which starts from the control and its evidence, with an owner and an age, answers a supervisor better than a list of articles, because the same restore test or incident record meets several duties.
Staff rights before the tool: Working-time rules, information and consultation, and co-determination decide how rotas, tools that log engineers' commands and metrics about engineers may be introduced. Agreements govern a class of change and are concluded before the tool is switched on.

Interactive exercises

Match: DORA articles on ICT risk management
Sort: Which regime reaches which organisation

Done on the lesson page, with feedback at once.

Discussion

The instruments set no levels of autonomy for operations actions. What, then, would you show a supervisor who asks why an agent was allowed to restart a critical service on its own?
A mapping row for a tested-recovery duty reads failed. Why is it better to show that row to a supervisor than to leave the row out until the re-test passes?
Which operations measure in your organisation would trigger consultation or co-determination, and has the agreement been concluded before the tool was switched on?

Exercise · 35 minutes

From article to control, owner and evidence

Choose three duties from the chapter's tables that apply to your organisation or to one of the two examples, for instance DORA Article 9(4)(e) on change management, NIS 2 Article 21(2)(c) on business continuity and AI Act Article 26(2) on human oversight. For each, read the live text and record the date you read it. Write the duty in the words of the text, the operations function, the control you already have, the evidence artefact with its date and result, a named owner and a verifier who is independent of the owner, and the status. Where the evidence is missing or failed, say so in the row.

Deliverable: Three mapping rows with text date, duty, function, control, dated evidence, owner, independent verifier and an honest status.

Self-test

What does the chapter conclude about the legal status of the autonomy budget of Chapter 4?
1. It is required by DORA Article 9 for every financial entity that uses agents
2. It is a design that makes the required outcomes demonstrable when an agent acts, not a legal requirement
3. It is prohibited by Article 22 of the GDPR for decisions about services
4. It is required by NIS 2 Article 21(2)(f) as the assessment of the effectiveness of the risk-management measures
Which point of NIS 2 Article 21(2) does the chapter say matters most for the book's thesis, and why?
1. Point (b), because incident handling opens the reporting clocks
2. Point (d), because supply chain security covers the provider-side change that both organisations suffered
3. Point (i), because access control covers agent identities
4. Point (f), because assessing the effectiveness of measures is what a restore-test record does
How do both organisations treat an agent that would allocate on-call duty or score people?
1. It is kept at Level 0 or 1, with agreement or negotiation first
2. It is allowed at Level 3 Bounded inside a catalogued runbook
3. It is treated as a standard change and released by the pipeline
4. It is left to the managed provider under the operations annex

Chapter 24 · 90 minutes

Across Boundaries: Cloud Providers, Managed Providers, Outages and Exit

The lesson treats the boundaries the organisation does not own and concerns O1 Observe, O4 Restore and O5 Sustain. Participants learn concentration as a property of the dependency graph, what three provider outage reports of 2025 show about shared responsibility, why provider status is telemetry, what contracts and the register of information supply, how exit and switching work, and how to choose redundancy by failure class.

Learning objectives

Assess concentration from a dependency graph, including shared services and subcontracting chains
Explain what the provider owns and what the customer owns in an outage
Choose between multi-zone, multi-region and a second provider by the failure class each addresses
Draft an operational transparency and exit-test annex and plan an exit test with a quality gate

Key concepts

Concentration: A property of the dependency graph and not of the number of vendors. DORA Article 29 names non-substitutable providers, multiple arrangements with one provider and the subcontracting chain; AI adds a model served by one provider or one pool of accelerators.
Provider status as telemetry: A status page is a statement by the provider about itself and can be late, wrong for a time or down with the provider. It is one input among others, and the organisation's observer must not share the provider's failure domain.
Register of information: The register DORA Article 28(3) requires of all contractual arrangements on ICT services. To an operations team it is also a dependency map, and the change pipeline can enforce it, so that a provider not on the register cannot enter production.
Exit test: A restore test whose scenario is the loss of a provider, carrying the six parts of a recoverability claim. It finds the hidden dependencies a plan does not, and Nordvik Region's first, RST-NR-2029-001, met its time objective and failed on its quality gate.
Redundancy by failure class: Multi-zone helps with a zone or hall failure; multi-region with a regional one; neither with a provider-wide control-plane or configuration failure, where a second provider as a tested recovery target helps, at the cost of a new seam.

Interactive exercises

Match: Clauses for the provider's AI
Scenario: The status page says operational

Done on the lesson page, with feedback at once.

Discussion

A sovereign option that is the only provider of several functions has a concentration cost. How would you weigh sovereignty against concentration for one of your Tier 1 services?
A second provider that has never been used for recovery is listed in the continuity plan. What can be said about it under the book's rule, and what would change that?
Nordvik Region's exit test met the time objective and failed the quality gate. Why does the book count that as a failure, and what would you have counted?

Exercise · 35 minutes

A transparency and exit-test annex

Draw the dependency map for the services that depend on one provider, yours or SamDrift for Nordvik Region: regions, shared services such as DNS, identity and keys, and subcontractors. Then write the annex terms the operations team can use and test: a machine-readable status feed, notice of change with a right to hold a release, the provider-side actions permitted with levels and a never list, credential revocation held by the customer, participation in joint tests in agreed windows, and data retrieval on exit. Plan one exit test as a claim with scenario, objective, a quality gate as well as a time objective, the independent probe, the date and the expiry.

Deliverable: A dependency map, a draft annex of six to eight testable terms and a one-page exit-test plan with time objective, quality gate, probe and expiry.

Self-test

Which failure class does multi-region deployment within one provider not address, according to the chapter's table?
1. A zone or hall failure, such as a cooling failure in one hall
2. A regional failure, where the needed control plane is in another region
3. A provider-wide control-plane or configuration failure, since a global change reaches every region
4. No class; multi-region addresses all failures of a single provider
What does the chapter say the Data Act's Chapter VI and DORA do for an exit, taken together?
1. They guarantee that an exit works once the contract contains the required terms
2. They give contract rights and require a tested plan, but only the exit test is evidence that the exit works
3. They transfer responsibility for continuity to the provider during the transition period
4. They exempt a customer from keeping a second provider if switching charges are waived
How should a provider-side action that would disable the customer's observer or stop path be treated in the contract?
1. Delegated at Level 3 Bounded, with a quarterly sample of the actions taken and the evidence used for each
2. Delegated at Level 1, with the provider's agent proposing the change
3. Left to the provider's discretion during a declared emergency
4. Never delegated, with credential revocation held by the customer and a quarterly stop drill

Module 6 · Part VI · People and Career

Part VI turns to the people at the end of every function. The participant learns what studies report about on-call load, alert fatigue and the supervision of automation, and how an on-call charter agreed with employee representatives answers them; which competences an operator needs and how the craft is kept with manual runs, dark days and seeded-error drills; and how to hold a labelled, falsifiable view of the future in a horizon register.

Chapter 25 · 90 minutes

Operators as People: On-Call, Toil, Trust and Wellbeing

The lesson treats operators as people and concerns all five functions. Participants learn what studies report on on-call duty, alert fatigue and toil, with the limits of each source; the ironies of automation and the moral crumple zone; blameless culture and psychological safety; what working-time law and co-determination require of rota design; and how an on-call charter records the response, rest, load and agent clauses agreed with employee representatives.

Learning objectives

Describe what the cited studies measured about on-call duty and alert fatigue, and in which populations
Explain the moral crumple zone and the three design rules the chapter writes into the charter against it
Identify which rota and tooling decisions require negotiation or co-determination before they take effect
Draft an on-call charter with response, rest, load and agent clauses and team-level metrics

Key concepts

Load as a design property: The chapter's first claim: the load on operators follows from the design of the loop, its alerts, rota and agents, and is not a personal failing. A breach of the noise budget is a finding on the alert rules, and the rota that carried the pages is not at fault.
Moral crumple zone: Elish's name for the pattern in which responsibility for a failure of an automated system lands on the nearest human operator. The charter answers it with a delegation record that names the delegator and owner of each agent.
Stop right without penalty: The rule that the stop right is exercised without approval and without penalty: an engineer who stops an agent on suspicion and is wrong has used the right as designed, and the review treats it as a data point for the observer's thresholds.
Response time as a working-time parameter: The chapter's organisational reading of Directive 2003/88/EC and the Court of Justice's judgments on stand-by: a very short response time and frequent call-outs weigh towards stand-by counting as working time, so page targets and page budgets are legal parameters as well as design ones.
On-call charter: The instrument, agreed with employee representatives before it takes effect, that records the response target, rest after a night call-out, the page budget, supervision limits for agents, the stop right and metrics reported for the rota and never the person.

Interactive exercises

Match: Charter clauses by usage mode
Scenario: A stop that turned out to be unnecessary

Done on the lesson page, with feedback at once.

Discussion

The chapter found no independent study of operators' wellbeing under autonomous operations agents. How should an organisation act on a question the evidence leaves open?
When an agent acts at Level 3 Bounded and something goes wrong at night, who in your organisation would be asked to explain it, and does your delegation record name that person?
What would have to be true in your team for an engineer to stop an agent without hesitation and report a near miss without fear?

Exercise · 35 minutes

An on-call charter for one rota

For one rota, yours or Nordvik Region's technical on-call of eleven engineers, collect or estimate a quarter of facts aggregated for the rota: pages per shift, share marked actionable, out-of-hours pages that were not Tier 1, and night call-outs. Draft the charter as a list of checkable decisions: the response target per tier, rest after a night call-out, the page budget and what happens on a breach, limits on supervising agents, the delegation record that names delegator and owner, the stop right without penalty, and the metrics, reported for the rota only. List the clauses that need negotiation or agreement with employee representatives before they take effect.

Deliverable: A draft charter of one to two pages with numbered clauses, each with a number or rule that can be checked, and a list of clauses requiring agreement.

Self-test

According to the chapter, who is at fault when a rota's noise budget is breached?
1. The engineers on the rota, who failed to acknowledge pages in time
2. Nobody on the rota; the breach is a finding on the alert rules and the correlation policy
3. The service desk, which routed too many tickets to the rota
4. The incident commander, who should have declared an incident sooner
What does the chapter conclude about whether AI relieves or intensifies the load on operators?
1. It is settled that correlation and summarisation relieve it in every team
2. It is settled that supervising agents intensifies it beyond what on-call did
3. It is irrelevant, because the agent's actions leave the rota entirely
4. It is open, and the charter decides it team by team
Why does a Tier 1 page target of five minutes to acknowledge matter beyond service levels at Hansa Mutual?
1. A very short response time bears on whether stand-by counts as working time, a question for counsel
2. DORA Article 19 requires every Tier 1 page of a financial entity to be acknowledged within five minutes
3. The IT-operations agent is demoted if a person acknowledges later than five minutes
4. The works agreement forbids any response target shorter than fifteen minutes

Chapter 26 · 105 minutes

Competences and Keeping the Operator's Craft

The lesson asks what operators need to know when agents run the loop for most of the day, and concerns all five functions. Participants learn five groups of competences and a framework for evidencing them, what studies in other domains measured about skill beside automation, the manual-practice clause, dark days and seeded-error drills, how the early-career route is rebuilt, and how a competence record is kept. The chapter debates whether automation deskills operators.

Learning objectives

Describe the five groups of operator competences and how each is evidenced
Summarise what the cited studies measured about unaided performance after use of an automated aid, and in which domains
Plan a drill year with manual runs, a dark day and seeded-error drills that are scoped, authorised and stoppable
Design first rungs for early-career operators where AI now does the routine work

Key concepts

Five competence groups: Systems thinking, verification of an agent's proposals, manual operation, authorship of what the loop runs, and judgement under uncertainty. In the chapter's framework they are areas SY, VF, MO, AU and JU, each evidenced by a kind of exercise.
Manual-practice clause: A clause in the mandate of every agent that runs runbooks: every 90 days, with the agent off, an evidenced holder who did not write the runbook runs the manual form, and every missing step is recorded. When evidence lapses, the delegation is lowered.
Dark day: A scheduled period in which a defined workload is operated without the operations agents. It is scoped in writing, keeps the observer and the stop path in service, respects the Level 4 prohibitions and ends in an improvement.
Seeded-error drill: A drill that keeps the ability to check where a manual run keeps the ability to act: a known flaw is placed in a draft or proposal under authorisation, and the record shows whether review found it.
Attempt-first rule: An early-career practice in which the apprentice answers or diagnoses first and the assistant's proposal is opened afterwards, so that the new work teaches what the automated routine used to teach.

Interactive exercises

Sort: Competences in the framework
Scenario: The apprentice and the assistant

Done on the lesson page, with feedback at once.

Discussion

The manual run at Hansa Mutual found that a precondition was satisfied by the agent and the manual form said nothing about how a person does it. Which of your runbooks would fail the same way?
The customer-support study found that novices gained most from an assistant. How can an assistant help a newcomer learn without removing the tasks that taught the craft?
A competence record is personal data about judgement exercised. Who should be able to read it, and for what decisions may it not be used?

Exercise · 35 minutes

A drill year with the agents off

List the runbooks that agents or an orchestrator run on your estate and the recovery claims in the resilience register, or take Nordvik Region's three runbooks and the portal failover. Plan twelve months: a manual run of each runbook above Level 1 within every 90 days by an evidenced holder who did not write it; one dark day or game day with the agents off, with written scope, authorisation, the observer and stop path kept in service and a named person who can end it; seeded-error drills for reviewers; and a rotation or secondary seat for an early-career operator with a mentor. State what record each exercise produces and what happens to the delegation when evidence lapses.

Deliverable: A dated drill-year plan with manual runs, one agents-off exercise, seeded-error drills, an apprenticeship element, the record each produces and the consequence of lapsed evidence.

Self-test

Who performs the manual run required by the manual-practice clause?
1. The author of the runbook, who knows its steps best
2. The agent, in dry-run mode, observed by the agent operator
3. An evidenced holder who did not write the runbook, with the agent off
4. A consultant from the managed provider, during the yearly audit of the runbooks
What happens at Hansa Mutual when a window no longer has two evidenced holders?
1. The Level 3 classes of the agent drop to Level 2 for that window
2. The agent is switched off for the whole group until the next dark day
3. The holders whose evidence lapsed are removed from the rota
4. Nothing changes until the yearly review of the role charter
How does the chapter's verdict label the wholesale loss of the operator's craft?
1. Observable, in the studies of operators cited in the chapter
2. Expected, in the cognitive part of manual recovery
3. Refuted, by the customer-support study of newer staff
4. Speculation

Chapter 27 · 90 minutes

The Future of Operations: Will Anyone Be on Call in 2036?

The lesson looks ahead with the series' discipline and concerns all five functions. Participants learn the horizon register, which trends are observable, expected and speculation, the limits of compute, energy, regulation and trust, three futures for 2036, the debate on whether anyone will be on call, the verdict, and the author's projection with the thresholds that would change it. They keep a register for their own operations.

Learning objectives

Classify trends in operations as observable, expected or speculation with the evidence for each
Compare the three futures for 2036 by who is paged, who verifies and where manual capability lives
Distinguish the chapter's verdict from the author's labelled projection and name what would change the view
Draft a horizon register with label, functions, evidence, trigger and owner per trend

Key concepts

Horizon register: A file in version control, reviewed on a schedule, in which each trend carries one label, the functions it concerns, a checkable claim, evidence, a trigger that would change the label, and an owner, so that the organisation's view of the future can be wrong and be revised.
Bottleneck moves to verification: An expected trend: if agents draft changes, runbooks and timelines, the scarce resource becomes the capacity to verify them, and the binding constraint on operations shifts from authoring to checking.
Three futures for 2036: Devices for thinking, not forecasts: Future A, the conductor rota; Future B, the bought loop with a customer-owned verifier; Future C, the unattended estate. They do not differ on whether an independent observer and a stop path are needed.
Constant in every future: An independent observer and a drilled stop path outside the platform, and manual recovery practised with the agents off, are the safety net both sides of the debate accept.
Author's projection: The author's labelled judgement, beyond the evidence: by 2036 a person will still be on call for the Tier 1 services of large European organisations, with fewer and heavier pages and the constraint moved to verification. Five thresholds would change it.

Interactive exercises

Sort: Labels in the horizon register
Match: Three futures for 2036

Done on the lesson page, with feedback at once.

Discussion

The case for concern says that gradual and defensible automation removes the person who could take over. Which step in your own roadmap looks defensible alone and contributes to that outcome?
Neither side of the debate has multi-year outcome evidence. What data would your organisation need to keep from now to have some by 2030?
The verdict says that whether the organisation can still operate by hand is decided now. What decision on your agenda this year decides it?

Exercise · 30 minutes

A horizon register for operations

List the trends that bear on your operations, at most eight for a small team. For each write a checkable claim, give it exactly one label, observable, expected or speculation, name the operations functions it concerns, cite the evidence you have, and write a trigger: the observation that would make you change the label or a ceiling. Name an owner and a review date. Include at least one trend about a provider's embedded agent and one about your own capacity to operate by hand. Finally, state for one Tier 1 service who is on call in 2030 under your present plans, and what evidence the plan rests on.

Deliverable: A register of five to eight trends with label, functions, evidence, trigger, owner and review date, and a one-paragraph statement of who is on call for one service in 2030.

Self-test

Which of these does the chapter label as speculation?
1. Agent products for incident investigation are on the market
2. A provider-run loop with no customer-side observer for a critical function
3. Level 3 widens class by class on evidence
4. The volume of AI-authored change grows, and stability lags where controls are weak
In Future B, the bought loop with a verifier, who verifies a remediation?
1. A customer-owned probe, with the provider's evidence
2. Agents, possibly of one family
3. The provider alone, under its own rules
4. No one; verification is replaced by contract penalties
What does the chapter's verdict say about a named person answering for each Tier 1 service?
1. It is speculation, because agents will resolve most incidents by 2030
2. It is observable everywhere today and therefore needs no decision
3. It is expected, while the form of the role is speculation
4. It is required by law for every organisation in the European Union

Module 7 · Part VII · Experience and Conclusions

Part VII tests the book against the record and draws its conclusions. The participant reads eleven public cases by mechanism and learns eight anti-patterns with their controls, follows Nordvik Region and Hansa Mutual through twelve months of applying the rules, studies the answers and twelve-month plans that go to each board, and weighs the verdict on each part of the thesis. The participant leaves with a board paper outline and five written records for a first week.

Chapter 28 · 105 minutes

Case Studies and Anti-Patterns

The lesson reads eleven public cases by mechanism, nine incidents of ordinary operations and two reports of agents acting beyond scope, and concerns all five functions. Participants apply four lenses to a case, learn the pattern in the change, recovery and provider cases, and study eight anti-patterns, each with the function it breaks and the control that changes the outcome. They read a post-incident report with the five functions.

Learning objectives

Apply the four lenses of function, action class, observer principle and evidence to a post-incident report
Explain the common pattern in the change cases, the recovery cases and the provider outages
Name the eight anti-patterns, the function each breaks and the control that answers it
Draft an anti-pattern record for one's own organisation with a control and a re-test date

Key concepts

Four lenses: The red threads of the book used to read a case: which function was concerned, which action class and what autonomy-budget reading applies, which observer test failed, and what the evidence behind any claim was. A reported cause is read as a statement of contributing conditions.
The green dashboard: Anti-pattern 1, which breaks O1 Observe: silence or missing data reads as health. The control is that no data counts as red and that an outcome probe runs in a separate failure domain.
The self-healing mask: Anti-pattern 2, which breaks O4 Restore: automatic repair hides the fault or prolongs it. The controls are independent verification, a budget counted by the observer and alert changes held at Level 0.
The first ring skipped: Anti-pattern 7, which breaks O3 Change: a change reaches its whole population before anything independent has judged it. The control is that every runtime change passes the first ring and flags are off by default.
The claim at the wrong scale: Anti-pattern 8: a recoverability claim that was configured and never restored, or true for one customer and untested for hundreds, fails at a dimension it never named, such as a client version, a scale or a location.

Interactive exercises

Match: Anti-patterns and their controls
Sort: Three groups of cases

Done on the lesson page, with feedback at once.

Discussion

The book's reading is that an AI agent in place of the script or pipeline in these cases would not have changed the mechanism. What would it have changed, and what does that imply for where controls belong?
Which of the eight anti-patterns would an honest review find in your own estate today, and what evidence would the review need to see?
Nordvik Region cannot run eight checks every quarter with seven engineers, so it re-tests its three records. How should a small team choose which patterns to test first?

Exercise · 35 minutes

Reading a report with the five functions

Take one public post-incident report summarised in the chapter, or an internal report of your own, and read it twice, once with a timeline. List the contributing conditions it describes. For each, name the operations function concerned, the action class, which of the three observer tests failed or held, and what evidence stood behind any claim of safety or recoverability. Mark anything that rests on press accounts only. Then choose the anti-pattern that best fits your own organisation's exposure to the same mechanism and write its record: the pattern, the function broken, where it shows in your estate, the control, an owner and a re-test date.

Deliverable: A one-page reading of the report by condition, function, observer test and evidence, and one anti-pattern record with control, owner and re-test date.

Self-test

What pattern does the chapter find in the three recovery cases of GitLab, Atlassian and OVHcloud?
1. A small trigger travelled by a path built for speed and reached every region
2. The exposure was the whole population before anything independent had judged the effect
3. A claim of recoverability was asserted and failed at a dimension it never named
4. An agent acted beyond its mandate and gave a wrong account of its actions
Which control does the table of anti-patterns give for the dead-man's switch that never fires?
1. A heartbeat to a receiver outside the platform, with a silence rule
2. Per-run tokens and a policy point outside the agent
3. A named owner, a review date and a test of the written runbook
4. Every runtime change passes the first ring, with flags off by default
In the table of what the cases show, which observer test failed in the CrowdStrike case?
1. Dependence: the failure email was rejected and nobody was told
2. Stop: the tools needed to halt the change were on the failed network
3. Dependence: customers' monitoring ran on the same cloud
4. Independence: the validator and the code shared an assumption

Chapter 29 · 90 minutes

A Year in Operations: Nordvik Region and Hansa Mutual

The lesson follows Nordvik Region and Hansa Mutual from September 2028 to August 2029 and concerns all five functions. Participants trace the year's events to the rules that produced or caught them: assistants gated by evidence, a game day that found an agent verifying itself, a refused restore, a report that hid an outage, a failover that failed again and then passed. They draw the four lessons of the year and run a monthly review of the loop.

Learning objectives

Trace four events of the illustrative year to the control that caught each of them
Explain why the book counts an unchanged level of delegation after twelve months as a result
Compare how the two organisations spent the year and what each could say at its end
Conduct a monthly review of the operations loop from its registers and records

Key concepts

Honest numbers lower than the first dashboards: The first lesson of the year: when independent probes, seeded faults and derived statuses replace self-reported measures, the figures fall. In every case the independent observer reported something worse than the actor, and each organisation was better off for it.
Closure takes longer than any plan: The second lesson: the portal failover failed in October 2027, failed again at its first re-test on 2029-04-10 and passed on 2029-06-05, so the Region could claim a verified failover only after about 20 months.
Autonomy that did not move: The third lesson: no level of delegation rose in the year. The evidence that would justify a raise, a verified claim, a seeded-fault result and a drill inside its target, was assembled for the first time and was not satisfied together.
Year ledger: An extract of the register each organisation keeps: the claims, tests, incidents, drills and decisions of the twelve months with their dates and states, from which the monthly review and the board report are read.
Monthly review of the loop: A one-hour review chaired by the head of operations that reads the register first, lists each claim by expiry and state, schedules what expires, and turns the loop's records into decisions with owners, with quarterly additions of a dark day, a seeded fault and a stop drill.

Interactive exercises

Put in order: The year in order
Scenario: No level was raised

Done on the lesson page, with feedback at once.

Discussion

Hansa Mutual ended the year with one verified restore claim, one expired failover and no level raised. How would you present that to a board that expected progress?
The same few named people handled the year's events in each organisation. What does that concentration cost, and what in the year addressed it?
Nordvik Region's first seeded-fault measurement missed its target by 41 seconds. What should follow from a narrow miss: a new target, a change to the observer or a re-test?

Exercise · 30 minutes

A monthly review of the loop

Using the ledger of your own organisation, or the extract of the year for one of the two examples, run the monthly review as the practical guide sets it out. Read the resilience register first and list each claim by expiry date and state, scheduling every claim that expires in the next period. Then read the autonomy budget per class, the observer's seeded-fault results, the stop drill's age and time, open actions by state and age, and any provider notices received. For each item that needs a decision, write the decision, the owner and the date. Note which quarterly exercise is due: a dark day, a seeded fault or a stop drill.

Deliverable: Minutes of a one-hour review: the register by state and expiry, five to eight decisions with owners and dates, and the next quarterly exercise scheduled.

Self-test

What happened to the degrees of delegation of Hansa Mutual's IT-operations agent between month 1 and month 12?
1. Rollback rose to Level 3 Bounded after two quarters of evidence
2. No level rose; restart stayed at Level 3, rollback at Level 2 and failover at Level 1
3. Restart fell to Level 2 Supervised after the game day and stayed there
4. Failover rose to Level 2 Supervised after the payments failover test passed in the night window
How did the re-tests of Nordvik Region's portal failover go in 2029?
1. The first re-test passed on 2029-04-10 and no second was needed
2. Both re-tests failed, and the claim stayed withdrawn at the end of the year
3. No re-test was run, because the provider declined to take part
4. The first failed on 2029-04-10 and the second passed on 2029-06-05
What did the review of Hansa Mutual's game day of 2028-10-24 find?
1. The agent verified its own restart by reading the health endpoint under the identity that performed it
2. The stop drill failed to revoke the agent's credential within an hour
3. The independent probe was down, so no verification of any kind took place
4. The incident assistant ranked the wrong hypothesis first in the seeded scenario

Chapter 30 · 90 minutes

Nordvik Region and Hansa Mutual: Conclusions for Two Organisations

The lesson gives the answers to the two questions of September 2028 on the date the first reports fall due, 2029-03-31, and concerns all five functions. Participants study the conclusions for each organisation, the conditions under which autonomy may rise, the five decisions that go to each board, the twelve-month plans with gates, the indicators that test the book's own claims, and how to write a board paper that says only what the register supports.

Learning objectives

State the answers to AIDR-0900 and AIDR-0950 and the evidence each rests on
Explain the conditions under which a class may rise to Level 3 Bounded in each organisation
Formulate the five board decisions for one's own organisation
Draft a board paper outline with a register, gated milestones, indicators with misuse notes and risks with triggers

Key concepts

A conducted loop around a bought base: The answer for Nordvik Region: operations becomes a loop the Region owns at its centre and buys under terms at its edge, with an assistant at Level 0 Advisory moving at most to Level 1, runbooks at Level 2, at most one class at Level 3 and an observer on another provider.
Three conditions together: Nordvik Region's rule for raising a scripted runbook from Level 2 to Level 3 Bounded: the observer has caught seeded faults for that class within target, a stop drill has met its target, and every window in which the runbook runs has two evidenced holders of the conductor role.
A conditional answer in three parts: The answer for Hansa Mutual: neither rollback nor failover may be extended now; rollback may reach Level 3 for catalogued pairs after four quarters of seeded-fault catch rate at 0.90, a stop drill inside its 5-minute target and an unexhausted budget; failover rises one step as supervised failovers pass.
Five board decisions: Adopt the register as the measure of resilience, set the rule that a level rises only on evidence, fund independent verification and review capacity, hold providers to operational terms, and protect the craft and the people. The board approves the rule, not each level.
Indicators of the book's own claims: Eight indicators, such as Tier 1 services with a verified recoverability claim and the seeded-fault catch rate per quarter, each with a baseline, targets, a source outside the system measured and a misuse note, so that a year of records can contradict the thesis.

Interactive exercises

Match: Five decisions for each board
Sort: Whose conclusion is it?

Done on the lesson page, with feedback at once.

Discussion

The board does not approve a level; it approves the rule by which a level rises. What does that division of labour protect against?
The indicator for verified claims carries the misuse note of a scope narrowed until the test passes. How would a board notice that this had happened?
Hansa Mutual's risk is described as an agent that has behaved well for several quarters, an observer that has never been surprised and a board that has stopped asking. What would keep the board asking?

Exercise · 30 minutes

A board paper that says only what the register supports

Outline a board paper for your own organisation, or for one of the two examples, following the practical guide. State the question in the board's own words and the decision record that commissioned it. Attach the register with statuses derived by rule and say in one paragraph what can and cannot be claimed about recovery today. Give the five decisions in the form that fits your organisation. Set out twelve months of milestones, each level change with its gate and the evidence that opens it. Add three indicators with baseline, target, a source outside the system measured and a misuse note, and three risks, each with a trigger and the change of plan that follows.

Deliverable: A two-page board paper outline with question, register summary, five decisions, gated milestones, three indicators with misuse notes and three risks with triggers.

Self-test

What can Nordvik Region prove about the recovery of its Tier 1 services at the date of the report, 2029-03-31?
1. That all four are recoverable within their objectives, on the design documents
2. That the portal is recoverable, on the failover test of October 2027
3. Nothing at Tier 1; the plan sets the dates on which that changes
4. That two of four are verified, on tests run in the first quarter of 2029
On what trigger may Hansa Mutual raise rollback to Level 3 Bounded for catalogued pairs?
1. A seeded-fault catch rate of 0.90 for four quarters, a stop drill inside 5 minutes and an unexhausted budget
2. A vote of the risk committee at the review of 2029-03-31
3. One quarter without a customer-visible incident on the claims core
4. A passing dark day in the night window with two evidenced holders
Which of these is one of the five decisions that go to each board?
1. Approve each change of level for each action class individually
2. Select the model family used by the operations agents
3. Replace the on-call rota with a managed provider within twelve months
4. Hold providers to operational terms

Chapter 31 · 90 minutes

Conclusion: Operations with AI

The lesson judges the five-part thesis against the evidence of the whole book and concerns all five functions. Participants learn the three limits that apply to every verdict, the verdict on each part with its observable core and open edge, why the observer principle is the book's most transferable rule, which questions remain open, what would show the thesis wrong, and five things to do on Monday.

Learning objectives

State the verdict on each of the five parts of the thesis with its observable core and open edge
Explain the three limits that apply to every verdict of the book
Identify what evidence would show the thesis wrong and how an organisation could collect it
Produce the five Monday records for one Tier 1 service

Key concepts

Verdicts of unequal strength: The observer principle and the first half of resilience as a claim rest on documented failures; matching autonomy to reversibility is observable in practice, while the autonomy budget is the book's own proposal without outcome data; the decay of manual skill is the weakest in direct evidence.
Designs, not findings: One of three limits on every verdict: the book's mechanisms, such as the budget, the observer manifest and the register, are designs and not findings, the evidence is of unequal strength, and the two organisations are fictional illustrations.
The observer principle as the central rule: The rule that carries the thesis and its most transferable contribution: it is graded by action class so its cost falls where the risk is, yet it does not make a loop safe, and an independent observer can also be wrong.
What would show the thesis wrong: Independent incident data over several years in which autonomous remediation without independent verification and without practised manual operation did better than a loop that keeps both, or operators who do not practise manual recovery handling novel failures as well as those who do.
Five things on Monday: Write the loop of one Tier 1 service on one page; set the ceiling per action class and name who can stop; list the identities that can change production; ask what verifies the last automatic repair and what it shares with it; write the six parts of the oldest recoverable claim and schedule a manual drill.

Interactive exercises

Match: The observable core of each part
Sort: Recommended ceilings to start from

Done on the lesson page, with feedback at once.

Discussion

The autonomy budget is the book's own proposal without outcome data. What would you record in your own organisation over two years to give it some?
The observer principle does not make a loop safe, and an independent observer can also be wrong. What, then, does independence buy?
Of the open questions the book leaves, such as how much manual practice is the minimum, which could your organisation help to answer by recording its own tests?

Exercise · 35 minutes

Five records for the first week

For one Tier 1 service, yours or Nordvik Region's portal under the charter CHR-NR-OPS-01, write five short records from what you already have. First, the loop on one page: service owner, objective, who observes, diagnoses, changes, restores and sustains. Second, the ceiling for each action class and who can stop, outside the platform. Third, the identities that can change production, marking those that are standing or shared. Fourth, what verified the last automatic repair and whether it shared a model, a signal, a credential or a failure domain with the actor. Fifth, the six parts of the oldest recoverable claim with its derived status, and a date for a manual drill with the agent off. Write unknown wherever you do not know.

Deliverable: Five one-page records for one service, each with an owner and a date, and a list of the unknowns to resolve in the first week.

Self-test

How does the chapter judge the autonomy budget, as distinct from the matching of autonomy to reversibility?
1. Both are observable in practice and in the public cases
2. The matching is observable; the budget is the book's own proposal without outcome data
3. The budget is observable; the matching is speculation
4. Both are refuted by the low measured performance of agents on unseen faults
What does the table of the five parts give as the open edge of the observer principle?
1. Expected: widening of delegation class by class
2. Expected: decay of manual skill beside automation
3. Speculation: no one on call for Tier 1 services
4. Speculation: that a model verifier is enough
Which of the five things to do on Monday answers the part of the thesis on the observer principle?
1. Ask what verifies the last automatic repair and whether it shares anything with it
2. List the identities that can change production
3. Write the loop of one Tier 1 service on one page
4. Set the ceiling for each action class and name who can stop