Module 1 · Part I · The Loop in Question
Part I sets the question of the book and the vocabulary for the rest of the course. Participants meet the working thesis in five parts, classify the six traditional testing forms by the stage each leaves unfinished, measure the pressure on an annual rhythm with the test-to-change ratio and the decay estimate, and leave with the definition of the purple loop: five stages, their records, the closure rule, the degrees of delegation and the independence principle.
Chapter 1 · 75 minutes
Introduction: The Purple Loop with AI
The lesson states the book's question: when AI sits on both sides of the table, does the loop between the red team and the blue team still show whether an attack would have been seen and stopped? Participants work through the five-part working thesis, the evidence that would show it wrong, the three groups of usage modes in the loop and the European obligations that frame it, and draft a one-page inventory of AI in testing and in detection.
Learning objectives
- Explain the five parts of the working thesis and what evidence would count against it.
- Compare the contracted catalogue run at Nordvik Region with the pipeline run at Hansa Mutual by what each can later prove.
- Classify uses of AI in the loop into the three groups of usage modes and say where the risk sits in each.
- Draft a one-page inventory of AI in testing and in detection, marking where AI is on both sides of one check.
Key concepts
- The purple loop
- The structure that carries a finding from an adversary's behaviour to a defence that has been shown to work. The purple team in this book is that joint loop between the red team and the blue team, not a team beside them.
- Working thesis
- The author's own five-part claim: purple is a loop, not a team; AI moves the loop from the event to the pipeline and keeps the event; a loop closed by machines alone verifies nothing new; coverage is a claim that needs evidence; safe emulation is a design question.
- Agreement without verification
- The risk that an emulation agent and a detection agent share training data, a model family and blind spots, so that a passing result shows only that two similar systems agree.
- Usage mode groups in the loop
- Modes 1 to 3 assist a person under the person's identity; Mode 4 performs a loop function under a service identity bounded by a mandate; Mode 5 agents such as an emulation runner act with an identity of their own, in effect a privileged account.
- Inventory of AI in the loop
- A one-page list of every testing and detection activity, whether AI takes part, whose it is, its usage mode, its identity and its highest level, with the entries marked where AI is on both sides of one check.
Interactive exercises
- Match: Five risks of a loop that two systems can close
- Sort: Three groups of usage modes in the loop
Done on the lesson page, with feedback at once.
Discussion
- The book commits in advance to evidence that would show its thesis wrong. Which of the two kinds of disconfirming evidence would be hardest for your organisation to collect, and why?
- Neither Thursday night is better in every respect. What would be lost if Nordvik Region copied Hansa Mutual's pipeline, and what would be lost if Hansa Mutual dropped its human-led work?
- Where in your own testing and detection might a provider's AI already be on both sides of one check without the contract saying so?
Exercise · 25 minutes
A one-page inventory of AI in the loop
Following the practical guide of the chapter, list the testing activities (penetration tests, red-team engagements, threat-led tests, tabletops, purple sessions, simulation tools) and the detection and response activities (triage, correlation, detection drafting, playbook execution, out-of-hours monitoring) of your own organisation, or of Nordvik Region as the chapter describes it. For each entry record who runs it, whether AI takes part, whose AI it is, the usage mode, the identity it uses and the highest level at which it acts, writing not stated where the contract is silent. Mark every entry where AI is on both sides of one check, and give each unknown entry a date by which the provider must answer.
Deliverable: A one-page inventory with an owner, a mode and a status for every entry, the both-sides entries marked, and a review date six months ahead.
Self-test
Which statement is the third part of the book's working thesis?
- A purple team should be staffed and budgeted as a third team beside red and blue.
- A loop closed by machines alone verifies nothing new.
- Continuous emulation makes the human-led exercise redundant within a few years.
- Coverage is proved once every catalogued scenario has passed at least once.
In the chapter's grouping of usage modes, where does the risk sit for a Mode 5 emulation runner?
- In over-reliance on a draft, because the person who asked for it remains accountable for the output.
- In a meeting tool that summarises a tabletop and assigns actions without review.
- In a shared service identity that only has read access to the telemetry.
- In an agent with its own identity and permissions, in effect a privileged account.
What must an answer to AIDR-0800 and AIDR-0850 state, according to the worked example?
- What was tested, when and with what result, and not a count of findings or of scenarios.
- The number of findings closed in the quarter and the share of green results.
- The names of the tools used by the tester and by the managed provider.
- The budget needed to move from two catalogue runs a year to a pipeline.
Chapter 2 · 75 minutes
What the Loop Did Before: Penetration Tests, Red Teams, Tabletops and Purple Sessions
The lesson describes the loop as it was before AI: the penetration test, the red team engagement, the threat-led test, the tabletop, the purple session and breach-and-attack simulation. Participants compare the forms by the question each answers, who knows, who acts and what each can and cannot show, identify the stage each usually leaves unfinished, and build a testing-forms register of the kind the chapter gives for Nordvik Region and Hansa Mutual.
Learning objectives
- Classify a testing activity as one of the six baseline forms using the chapter's four questions.
- Identify the stage of the loop that each form usually leaves unfinished.
- Explain why findings stalled, separating what the sources report from the book's own conclusion.
- Draft a testing-forms register with an owner for each unfinished stage and an evidence age for each result.
Key concepts
- Six testing forms
- The penetration test, the red team engagement, the threat-led test, the tabletop, the purple session and breach-and-attack simulation. They differ in the question answered, who knows, who acts, the output and the cadence.
- Stage usually left unfinished
- In the book's classification of the common case, the penetration test, the threat-led test and the purple session leave P5 Verify, the red team engagement and the simulation leave P4 Improve, and the tabletop performs no P2 Emulate and leaves P5 Verify.
- White team or control team
- The group that referees an exercise: it sets rules and metrics, observes and derives lessons. The 2018 TIBER-EU framework says White Team; the current ECB text and the Delegated Regulation say control team. The book uses white team for the control group that governs an exercise.
- Trusted agent
- One named person on the defended side who knows of the exercise, so that real incidents are not mistaken for it and the stop can be reached. No verified source uses the term in this sense; it is the book's usage.
- Testing-forms register
- A register (TFR-NR-01, TFR-HM-01) that states for every testing activity its form, cadence, who it informed, what it proved and the stage it left unfinished, with an owner for that stage and an evidence age.
Interactive exercises
- Match: Six testing forms and the question each answers
- Sort: What a result can and cannot show
Done on the lesson page, with feedback at once.
Discussion
- Hansa Mutual has every testing form and still left P5 Verify to the end. What does that say about adding more forms as a remedy?
- A blue team that is unaware measures real detection and response; a blue team in the room learns faster. How should an organisation choose between the two for a given question?
- The chapter uses one vendor's dataset only as an indication of a mechanism. How should a team treat a statistic of that kind when it presents its own case to a board?
Exercise · 25 minutes
A testing-forms register
Using the seven steps of the chapter's practical guide, list every activity of the last 24 months in your organisation that was called a test, assessment, exercise, simulation or review of the defence. Classify each as one of the six forms by asking who knew, who acted, whether a technique was actually run and whether the result was a document, a changed control or a shared record. Record the last run, the cadence and the scope, trace where the findings of the last run went, and name the stage each form left unfinished. Participants without access to such records extend the registers TFR-NR-01 and TFR-HM-01 from the chapter's description of Nordvik Region and Hansa Mutual.
Deliverable: A register with a form, a date and an owner per activity, a named unfinished stage per form with its owner, and an evidence expiry, by default 180 days.
Self-test
Which stage does a breach-and-attack simulation usually leave unfinished, in the chapter's classification?
- P1 Target, because the catalogue has no scenarios at all.
- P2 Emulate, because no behaviour is ever executed.
- P4 Improve, because a failure is recorded with no one to act on it.
- P3 Observe, because the tool cannot record a pass or a fail.
What can a tabletop show, according to the chapter's evidence table?
- That the people, roles and plan hold together on paper.
- That a named vulnerability was exploitable at a given date.
- That a detection still catches a variant of the technique.
- That a control blocked a catalogued behaviour on a tested path.
Which cause of stalled findings does the chapter put above the others, as its own conclusion?
- Testers wrote reports that the system owners could not read.
- Budgets for penetration tests were cut after the first year.
- Supervisors never asked for a remediation plan.
- Closure of a finding had no owner and no test to prove it.
Chapter 3 · 70 minutes
The Pressure on the Loop: Machine-Speed Adversaries, Detection Decay and the Annual Test
The lesson measures the pressure that makes the traditional rhythm inadequate: a shrinking exploitation window, an estate that changes faster than the test, detection decay, the finding half-life, scarce talent and a calendar that rewards a report. Participants calculate a test-to-change ratio, read a decay estimate in three states, and separate what AI makes cheaper, repetition, variation and drafting, from what it does not: novelty, judgement, independence and safety.
Learning objectives
- Explain the assumptions of the annual rhythm and the evidence the chapter cites against each.
- Calculate a test-to-change ratio for a priority technique from change and evidence records.
- Classify a detection as current, at-risk or expired from its last passing test and its dependency changes.
- Compare what AI makes cheaper in the loop with what stays costly.
Key concepts
- Test-to-change ratio
- For a given technique, the number of production changes that touch what the technique depends on, between two passing tests. A ratio of one means every change was followed by a verification; a ratio of twenty means twenty changes accumulated before anyone looked.
- Detection decay
- The loss of a detection's effect between tests as its dependencies change. It is measured by age since the last passing test, set against the evidence expiry, together with the count of dependency changes since that test.
- Current, at-risk, expired
- The three states of a decay estimate: current if a test passed within expiry and no dependency changed since; at-risk if within expiry but a dependency changed; expired if the last passing test is older than the expiry.
- Finding half-life
- The time a finding spends between P3 Observe and P5 Verify, in two parts: the wait for a change at P4 Improve and the wait for the proof of that change.
Interactive exercises
- Sort: Current, at-risk or expired
- Scenario: A test-to-change ratio above ten
Done on the lesson page, with feedback at once.
Discussion
- Age alone misleads in both directions. Which of your detections would look healthy by age and unhealthy by dependency changes, or the reverse?
- The chapter says a calendar set by regulation or procurement rewards a report. What would have to change in a contract or a testing programme for it to reward a verified finding instead?
- If AI makes variation and repetition cheaper but not novelty, judgement, independence or safety, where does the bottleneck of your own loop move?
Exercise · 25 minutes
Test-to-change ratio and decay estimate
Choose three priority techniques. For each, write down the dependency path: the service or identity it targets, the log source, the detection record and the playbook. From the change log of the last 12 months, count the changes that touched anything on that path, list the dates of passing tests in the same period and divide changes by passing tests. Mark any technique whose ratio exceeds 10 or whose last passing test is older than its evidence expiry, classify its detection as current, at-risk or expired, and choose a remedy per technique: a trigger after a change on the path, a schedule, or an accepted gap recorded as such.
Deliverable: A dated table with one row per technique: changes, passing tests, ratio, last passing test, decay state and the chosen remedy.
Self-test
A priority detection at Hansa Mutual passed its last test 53 days ago, and its log connector was upgraded after that test. Which state does the chapter's decay estimate give it?
- Current, because 53 days is inside the 90-day expiry.
- At-risk, because it is within expiry but a dependency has changed.
- Expired, because any dependency change voids the evidence.
- Retired, because the connector upgrade replaced the detection.
What does a test-to-change ratio of twenty mean for a technique?
- Twenty passing tests were run for each production change on the technique's dependency path.
- Twenty analysts would be needed to verify the technique.
- Twenty changes accumulated on its dependency path before anyone verified it.
- Twenty per cent of the scenario library covers the technique.
According to the chapter, which of the following does AI not make cheaper in the loop?
- Independence between the side that tests and the side that is tested.
- Repetition of a catalogued, reversible scenario on a schedule.
- Variation of a scenario's parameters such as account and hour.
- Drafting of scenario variants and of result summaries.
Chapter 4 · 90 minutes
The Purple Loop Defined: Five Stages, Evidence and the Independence Principle
The lesson defines the purple loop: P1 Target, P2 Emulate, P3 Observe, P4 Improve and P5 Verify, each with its record; the closure rule and its five finding states; ceilings on delegation per stage and action class; the independence principle with five kinds of separation, one incentive rule and three proofs; and the roles that hold the loop. Participants write a delegation profile and learn why the loop is not a team, an event or a dashboard.
Learning objectives
- Explain each of the five stages by the question it answers and the record it produces.
- Classify findings into the five states of the closure rule and say which may be reported as closed.
- Draft a delegation profile with a ceiling, a decider and a stop holder for each action class.
- Review a loop's independence declaration against the five kinds of separation, the incentive rule and the three proofs.
Key concepts
- Stage records
- Each stage is defined by its record: a scenario record (SCN) at P1, an emulation plan and run log (EMU) bounded by rules of engagement (RoE) at P2, a finding (FND) and purple session record (PRP) at P3, a change at P4, and a regression case (REG) with an evidence manifest (EVD) at P5.
- Closure rule
- A finding is closed only when the re-test and its variant pass, the evidence manifest is signed and attached, and a named person signs the closure with its date. The five states are open; implemented, unverified; re-test failed; closed and verified; risk accepted.
- Delegation ceiling
- The highest autonomy level, from Level 0 Advisory to Level 4 Autonomous, allowed for an action class at a stage. The lowest ceiling governs, and Level 4 is never used for an emulation action that is irreversible, leaves the signed scope, touches personal data beyond authorisation or targets a named person.
- Independence principle
- A check counts only where the side that tests and the side that is tested can fail in different ways. It is declared as five kinds of separation (model, data, authorship, vendor, time), one incentive rule between roles and three proofs: held-out scenarios, seeded gaps and reproduction.
- Purple-team coordinator
- The role that runs the loop: it schedules, makes sure findings reach their owners in a usable form and tracks the age of open findings. It is accountable for the flow and legibility of the register and is never a third team.
Interactive exercises
- Put in order: The five stages of the purple loop
- Sort: Ceilings per stage and action class
Done on the lesson page, with feedback at once.
Discussion
- A white-team sample at Hansa Mutual found 6 of 20 closed findings had been closed when the change merged. Why is that state so easily reported as closed, and who has an interest in the difference?
- Nordvik Region signs only P2, P3 and P5 with its counterparties and keeps P1 and P4 internal. What does that choice say about which decisions can be bought and which cannot?
- The book says a kind of separation that is not shown is treated as shared. Which of the five kinds would your organisation find hardest to show, and what would compensate for it?
Exercise · 30 minutes
A delegation profile for a loop
Following the chapter's practical guide, list the action classes for each of the five stages, starting from the chapter's table. For each class ask the four questions: is the action reversible, does it stay inside the signed scope, does it touch personal data beyond what the exercise is authorised to touch, and does it target a named person. Set the ceiling at the lowest level any answer permits and no higher than your organisation has evidenced. Name the decider, the person who delegated and the stop holders with deputies. Write the conditions as numbers, and add an independence declaration for every row where AI sits on both sides of a check. Use Hansa Mutual's agents if you have none of your own.
Deliverable: A one-page delegation profile in which every row has a ceiling, a decider and a stop holder, and the closure row names a person.
Self-test
A change has been merged for a finding but the re-test has not yet run. How is the finding reported under the closure rule?
- As closed, because the change was reviewed and merged by its owner.
- As risk accepted, because no evidence exists either way.
- As re-test failed, because the absence of a pass counts as a failure.
- As open, in the state implemented, unverified.
What is the recommended ceiling for declaring a finding closed and verified at P5 Verify?
- Level 3 Bounded, enforced by the regression case outside the runner.
- No delegation: the runner proposes and a named closure owner signs.
- Level 2 Supervised, with the purple-team coordinator sampling results.
- Level 4 Autonomous, provided the scenario is benign and reversible.
Which set names the three proofs of the independence declaration?
- A signed manifest, a stop path and a white-team approval.
- Model, data and vendor separation between the two sides.
- Held-out scenarios, seeded gaps and seeded errors, and reproduction.
- A re-test, a variant and a dated signature on the closure.
Module 2 · Part II · The Stages of the Loop
Part II takes the loop stage by stage. Participants build a priority list and a scenario record at P1 Target, write rules of engagement that a policy decision point can enforce, plan a human-led and AI-assisted campaign and a continuous pipeline at P2 Emulate, classify outcomes against a sealed expectation at P3 Observe, turn a finding into a reviewed change at P4 Improve, and close it at P5 Verify with a regression case and an evidence manifest that expires.
Chapter 5 · 90 minutes
Target: Threat-Informed Scenarios, Risk and Priority Techniques
The lesson covers P1 Target: how intelligence, the threat register, risk assessment and incidents become scenarios, how ATT&CK and ATLAS serve as a vocabulary of identifiers and not procedures, and how priority techniques are ranked by impact, relevance and uncertainty about the defence. Participants build a first priority list, write a scenario record, and learn the guards against a library that fills with what is easy to emulate.
Learning objectives
- Explain what each of the four inputs to a scenario supplies and why the proposer is not the selector.
- Rank candidate techniques by impact, relevance and uncertainty about the defence into priority tiers.
- Draft a scenario record with source, expected outcome per layer, safety class, evidence rule and exclusions.
- Review a scenario library for selection bias and record exclusions in an exclusion register.
Key concepts
- Priority technique
- A technique chosen for regular testing by the sum of three coarse tiers: impact on critical assets, relevance to this organisation or sector, and uncertainty about whether the defence would stop it. The result is an order and a tier, with reasons.
- Scenario record
- The testable statement built from one or more techniques and held as code (SCN-NR, SCN-HM): identity and owner, source, technique set and parameters, expected outcome per layer, safety class and environment, evidence rule, exclusions and sensitivity.
- Scenario curator
- An agent or drafting assistant that reads intelligence, register entries and incident reviews and proposes scenario drafts at Level 1 Assisted. A named person accepts each draft, sources are treated as untrusted input, and the reviewer's work is measured.
- Selection bias
- The tendency of a library to fill with scenarios that are cheap to run. The guards are gaps reported as gaps, a reserved share for hard techniques, a check of the curator's mix and a protected class of human-authored scenarios.
- Exclusion register
- The record of what is deliberately not tested: out of scope, unsafe, unauthorised or not worth testing, each with the reason, the decider and a review date. An excluded technique is untested, and untested is not covered.
Interactive exercises
- Sort: What not to test, and why
- Scenario: A hard technique on the priority list
Done on the lesson page, with feedback at once.
Discussion
- At Hansa Mutual the detection-engineering team may nominate techniques but cannot remove one. What would the priority list look like after two years if the tested side could remove entries?
- Nordvik Region reported T1190 and T1486 as untested when their mirror evidence had expired. What pressures push an organisation to report such techniques differently?
- The curator's proposals favoured well-documented techniques. How would you design the review so that a reviewer who sees mostly good drafts still reads each one?
Exercise · 35 minutes
A first priority list and one scenario record
Choose one service of your own organisation, or Nordvik Region's benefits-triage service. Following the practical guide, collect candidate techniques from intelligence tags, your own incidents and findings, and the exposure of the service. Give each candidate an impact tier, a relevance tier and an uncertainty tier from 1 to 3, sort by the sum, break ties by uncertainty and cut the list. For the first twenty entries record how the technique can be emulated safely and mark the hard ones. Then write a scenario record for the top entry by ATT&CK identifier only: source, parameters, expected outcome per layer, safety class, evidence rule and exclusions. Include no procedure.
Deliverable: A priority list with tiers, reasons, owners, emulation routes and a review date, plus one scenario record at the level of identifier and parameter.
Self-test
Which three factors does the chapter combine to rank priority techniques?
- Novelty, ease of emulation and the number of public reports.
- Cost of the test, tester availability and the audit calendar.
- Impact, relevance and uncertainty about the defence.
- Severity, exploitability and the age of the vulnerability.
What is the ceiling for AI at P1 Target, and who decides what enters the scenario library?
- Level 1 Assisted; a named owner accepts or rejects each draft.
- Level 2 Supervised; the curator adds drafts and a person samples them.
- Level 3 Bounded; the policy decision point accepts catalogued drafts.
- Level 0 Advisory; the detection-engineering team selects the scenarios.
How does the chapter treat a priority technique that has been placed on the exclusion register?
- As covered by the compensating controls of the wider estate.
- As removed from the priority list until the next six-monthly review.
- As low risk, because the exclusion was approved by a named person.
- As untested, and untested is not covered.
Chapter 6 · 90 minutes
Rules of Engagement and Safe Emulation: Scope, Blast Radius and the Stop Path
The lesson treats rules of engagement as the first design question of the loop. Participants separate the rules of engagement from the mandate manifest and the decision record, express scope as code that a policy decision point enforces, assign safety classes S1 to S4, record third-party consent and the handling of personal data, and design a stop path that runs outside the emulating system and is drilled. The white team owns the manifest.
Learning objectives
- Explain why the rules of engagement, the mandate manifest and the decision record are kept apart and which ceiling governs.
- Classify scenarios into safety classes S1 to S4 and state the environment and level each allows.
- Draft a rules-of-engagement manifest whose every sentence a policy decision point can evaluate.
- Design a stop path with machine-detectable abort conditions, named holders and a drill.
Key concepts
- Rules-of-engagement manifest
- A signed, versioned file (RoE) that says what may be done to which systems, when, with whose consent and under what stop conditions. It belongs to the exercise or programme and is enforced by a policy decision point outside the agent.
- Scope as code
- Scope in a form read at every action: default deny, exclusions checked before inclusions, the manifest outside the agent's reach, and refusal tested by three dry runs against an out-of-scope target, after the window and with a stop flag set.
- Safety classes S1 to S4
- S1 is benign, reversible and tagged; S2 is reversible with bounded side effects; S3 needs realistic state or may disturb service and never runs in production; S4 has impact or irreversible effect, or involves staff or third parties.
- Stop path
- The mechanism, held by named people outside the emulating system, that denies further actions at the policy point, revokes the system's credentials and egress, and invokes the clean-up. It needs no approval to use and is drilled.
- White team
- The control group that governs the exercise: it owns the manifest and its versions, holds or delegates the stop path, runs deconfliction and keeps the record. It lies outside both the red team and the SOC so that it can stop either.
Interactive exercises
- Sort: Safety classes S1 to S4
- Scenario: Real personal data in an authorised exercise
Done on the lesson page, with feedback at once.
Discussion
- The chapter says a control that has never refused anything has not been shown to work. Which controls around your own testing have never been seen to refuse?
- Nordvik Region's risk is a manifest signed once and treated as a formality; Hansa Mutual's is a pipeline running inside rules nobody has re-read. Which is closer to your organisation, and what keeps the rules alive?
- A managed provider is on the blue side of an exercise and also holds a stop path. How should the two roles be reconciled in a contract?
Exercise · 35 minutes
A manifest a policy point can enforce
Write a rules-of-engagement manifest for an authorised exercise of three scenarios against your own organisation's systems or Nordvik Region's benefits-triage service. Follow the ten steps of the practical guide: name the question and the scenario identifiers; list exclusions first, then inclusions with their owners; give permitted techniques by ATT&CK identifier with a safety class each; set the window, the blackouts and who may shorten the window; set the autonomy ceiling per class; require short-lived credentials, tagging and synthetic accounts; record third-party consent; write machine-detectable stop conditions with named holders and deputies; and state data classification, legal basis and the rule for real personal data met by accident. Write every line so that a policy point could evaluate it.
Deliverable: A signed-ready manifest of one to two pages, with the three dry-run refusals and the stop-path test listed as conditions of its definition of done.
Self-test
A manifest allows Level 3 Bounded for class S1 scenarios, but the agent's mandate allows only Level 2 Supervised. At which level does the agent run?
- Level 3 Bounded, because the manifest belongs to the exercise.
- Level 2 Supervised, because the lowest ceiling governs.
- Level 4 Autonomous, once the white team has signed both documents.
- Level 1 Assisted, because any conflict falls back to the minimum.
What does the stop path do, in order, when a halt is called?
- It asks the emulating system to finish its current step and report.
- It notifies the red-team lead, who decides whether the run continues.
- It pauses the schedule, opens a ticket for the red-team lead and waits for the system owner to approve the halt.
- It denies further actions, revokes credentials and egress, and invokes the clean-up.
Which safety class does the chapter assign to a scenario that involves staff, such as a social-engineering test?
- S4, with AI limited to Level 0 Advisory and named people acting.
- S1, provided every action is tagged and reversible.
- S2, with white-team approval for each run in production.
- S3, run on a range or mirror at Level 3 Bounded.
Chapter 7 · 75 minutes
Emulate: Adversary Emulation Plans, Human-Led and AI-Assisted
The lesson turns a scenario into a campaign. Participants learn what the emulation plan record contains, why the red team, the white team, the trusted agent and the blue team that does not know are separate roles, what a person and what AI each contribute to planning and execution, and how a tamper-evident event log and a sealed hand-over give P3 Observe its ground truth without giving the blue team the plan.
Learning objectives
- Draft an emulation plan record that binds scenarios to the signed scope and names measurable abort conditions.
- Explain the four groups of people in a live exercise and what each can and cannot do.
- Compare what a person contributes to a campaign with what AI adds in planning and variation.
- Design an event log that is tagged at the source and that testers cannot rewrite.
Key concepts
- Emulation plan record
- The plan as code (EMU): the manifest it runs under, ordered scenarios with techniques and safety classes, expected outcomes copied from the scenarios, target scope by reference, window, abort conditions, roles, delegation profile, tag scheme and event-log store. It contains no exploit detail.
- Blue team that does not know
- Everyone on the defending side who is not told of the exercise, so that it measures what they would do on an ordinary day. They are measured in aggregate and never by name.
- Event log
- A timestamped, tamper-evident record of every action taken, written as it happens to an append-only store held by the white team. It includes any agent's actions and records the unexpected, such as side effects and stops.
- Ceiling and boundary for execution
- An agent executes only catalogued, benign, reversible scenarios inside the signed scope at Level 3 Bounded, and the agent that executes does not decide what to execute: a person chooses the scenario from a catalogue.
Interactive exercises
- Match: Four groups of people in a live exercise
- Put in order: Writing an emulation plan
Done on the lesson page, with feedback at once.
Discussion
- The plan asks for one scenario written by a person and shown to no assistant. What does that scenario test that a library scenario cannot?
- The white-team lead's signature, not the red team's own approval, opens the window. What goes wrong when the red team authorises itself?
- Nordvik Region demands the event log, the plan and the drilled stop path as contract deliverables. What would you add to a tester's contract after this lesson?
Exercise · 30 minutes
An emulation plan with ground truth
Using the nine steps of the practical guide, write an emulation plan for an authorised two-week window against your own organisation's systems or the benefits-triage service of Nordvik Region. Select three to eight scenarios by identifier from a priority list, copy the expected outcome per layer from each, and add one scenario written by a person and shown to no assistant and one variant of a known scenario. Bind targets by reference to the manifest's scope, write abort conditions that someone other than the testers can measure, name two stop-path holders outside the red team, define the tag scheme and the event-log store, and state who knows and who does not. Give techniques by ATT&CK identifier only.
Deliverable: A plan record ready for the white-team lead's signature, with measurable abort conditions, named stop holders, a tag scheme and a list of who knows.
Self-test
How does the emulation plan state the targets of a campaign?
- As a new list of hosts chosen by the red-team lead for the window.
- As whatever the emulation agent discovers during the first day.
- As a reference to the scope in the signed manifest, never as a wider list.
- As a description agreed verbally with the system owners beforehand.
Which contribution does the chapter attribute to a person in a campaign, as distinct from AI?
- Judgement that depends on consequence and context.
- Breadth across the scenario library at low cost.
- Speed in drafting the plan record from the scenarios.
- Listing the techniques in a report that lack a scenario.
Where is the event log of an exercise written, according to the chapter's design rules?
- To the testers' own workstations, to be cleaned up after the report.
- To an append-only store held by the white team, which testers cannot rewrite.
- To the SIEM of the blue team, alongside the production detections.
- To the emulation agent's memory, exported when the window closes.
Chapter 8 · 90 minutes
Emulate Continuously: Breach-and-Attack Simulation, Atomic Tests and Automated Red Teaming
The lesson treats P2 Emulate as a pipeline: breach-and-attack simulation, atomic tests and automated red teaming, triggered by change, schedule and intelligence. Participants decide what runs where, set triggers and cadence, design variation and a held-out set against overfitting and Goodhart's law, and work through the chapter's debate on whether continuous automated emulation makes the quarterly human-led session unnecessary, ending in its three-level verdict.
Learning objectives
- Compare breach-and-attack simulation, atomic tests and automated red teaming by what each is good for.
- Design triggers and a cadence that keep evidence for priority techniques inside its expiry.
- Explain the four overfitting failures and how a held-out set and the generalisation gap expose them.
- Review the debate on the human-led session and state the verdict with its observable, expected and speculation labels.
Key concepts
- Three kinds of trigger
- Change triggers answer a detection that worked yesterday and not today; schedule triggers answer decay when nothing changed; intelligence triggers answer a world that moved. Hold-backs during a freeze or a major incident are logged with the evidence age.
- What runs where
- Every new scenario and variant runs first on the red-team range; scenarios above S1 run in a pre-production mirror; production carries S1 on the pipeline's authority and S2 only with white-team approval per run; shadow replay tests a detection change against stored telemetry.
- Overfitting failures
- Marker leakage, where a rule fires on the telemetry tag; signature fitting to the public form of a test; catalogue closure, where detections converge on the library; and dashboard comfort, where a high pass rate over a stable library is read as safety.
- Held-out set and generalisation gap
- A share of scenarios and variants hidden from detection engineers and never used for tuning. The generalisation gap is the difference between the pass rate on known forms and on held-out variants; above ten percentage points it is treated as a finding.
- Protected share of human-led emulation
- A share of red-team hours reserved for novel, human-led work, with a default of 30 per cent that is the book's own figure, and with novel-finding yield measured separately from the pipeline's regression results.
Interactive exercises
- Sort: What runs where
- Scenario: A request to see the held-out set
Done on the lesson page, with feedback at once.
Discussion
- The case for calm says spending a red-team operator's scarce hours on re-running last year's techniques is a poor use of them. Where does that argument stop being true?
- A pipeline that adds alerts consumes analyst attention. Who in your organisation should hold the right to cap its volume, and on what evidence?
- If the model that proposes variants also drafts detections, the set is not held out from it. How would you show a board that your held-out set really is held out?
Exercise · 25 minutes
Triggers and cadence for continuous emulation
For ten priority techniques of your own organisation, or of Hansa Mutual as the chapter describes it, write a pipeline definition in outline. Give each technique its evidence expiry and set the schedule trigger to one third of it. List the detections, connectors and mandates that can affect each technique and attach a change trigger with a maximum delay. Define the intelligence trigger as a named intake with deadlines. Estimate the alerts per week the pipeline will generate and compare them with the alert budget of the affected detections. Add hold-back rules and a quarterly stop-path drill with a target time. Participants whose organisation buys the loop instead specify five atomic tests, by technique identifier, for a contracted catalogue run.
Deliverable: A one-page pipeline definition, or contracted test subset, with three trigger types, a volume cap, hold-back rules and a stop-drill target.
Self-test
In Hansa Mutual's first quarter of 2028, detections fired on 37 of 41 known-form scenarios and on 10 of 14 held-out variants. What does the chapter conclude?
- The pipeline is sound, because nine in ten known scenarios were detected.
- The held-out set is too small to say anything and should be dropped.
- The red team's held-out variants were unfair and should be shown to the detection engineers before the next run.
- The 19-point generalisation gap is above the threshold and is a finding on the detections.
Which overfitting failure occurs when a detection fires on the telemetry tag that gives ground truth?
- Catalogue closure.
- Marker leakage.
- Signature fitting.
- Dashboard comfort.
How does the chapter's verdict label the claim that continuous automated emulation replaces the human red team?
- Speculation: no evidence shows it, and a protected share of human-led work remains.
- Observable: it is documented practice in large financial groups.
- Expected: it follows from AI-generated variants at scale becoming routine in large organisations within a few years.
- Refuted: the sources show that pipelines find fewer issues than people.
Chapter 9 · 90 minutes
Observe: Telemetry, Detection and Response Compared with Expectation
The lesson covers P3 Observe, where what the emulation did is compared with what the organisation's coverage claims said should happen. Participants write and seal an expectation sheet before results are shown, use tagged ground truth held outside the system observed, classify each step into one of seven outcomes ordered by layer, run a purple session with a rule for disagreement, and raise findings that record facts and propose an owner.
Learning objectives
- Explain why the expectation is written from the detection catalogue and sealed before the ground truth is unsealed.
- Classify emulated steps into the seven outcomes, recording the earliest failing layer.
- Draft a finding that states the expectation, the outcome, the layer that failed and the evidence, without choosing the fix.
- Review the use of an AI correlator at P3 against its four failure modes and the privacy rule.
Key concepts
- Expectation sheet
- The organisation's coverage claim made explicit for the techniques in a scenario: telemetry, rule, severity, triage and time bound per step. The detection owner writes it from the catalogue, knowing only the list of techniques, and the white team seals it.
- Ground truth
- The record of what the emulation did, written by the emulation side at the time. It is tagged, time-stamped from a source both sides trust and held outside the system observed, and the tag must never reach detection logic.
- Seven outcomes
- Prevented; detected; alerted but not triaged; triaged but not acted on; missed; no telemetry; not tested. They are mutually exclusive and ordered by layer, and detected demands the whole chain through a qualified triage decision and the playbook action.
- Purple session record
- The record (PRP) of the meeting in which red and blue read the result together: both seals, the roles, one outcome per step with a saved query, disagreements recorded with both readings, and results reported by technique, never by analyst.
- Optimistic join
- A correlator's failure in which it finds an alert near in time and on a related host and counts the step as detected. At Hansa Mutual all three corrections to the correlator's proposals were of this kind.
Interactive exercises
- Sort: Classifying emulated steps
- Put in order: A purple session with sealed expectations
Done on the lesson page, with feedback at once.
Discussion
- Step 2 at Nordvik Region looked detected in the SIEM alone. Which of your own detected results would survive the question of whether a qualified person took a triage decision in time?
- Unsettled steps take the more cautious outcome, and disagreement is not settled by seniority. What does that rule cost a session, and what does it protect?
- A Mode 5 agent that runs a scenario does not also grade it. Where else in your tooling does one system both act and judge its own result?
Exercise · 35 minutes
A purple session with sealed expectations
Work in groups of four as red lead, detection owner, SOC representative and white team. The white team gives the detection owner only the list of techniques of a six-step scenario. The detection owner writes the expectation sheet from the catalogue: telemetry, rule, severity, triage and time bound per step. The white team seals it, then unseals the supplied ground truth step by step in time order. For each step the group agrees one of the seven outcomes, working from prevention towards action, and records disagreements with both readings. For one step that is not prevented or detected, draft the finding: expectation, outcome, layer that failed, evidence and proposed owner. Do not discuss the fix. Use the Nordvik Region session of the chapter as the supplied case if you have none.
Deliverable: A purple session record with one outcome per step and one draft finding naming the failed layer and a proposed owner.
Self-test
A detection fired and an alert was closed by an automated process without a qualified triage decision. Which outcome does the chapter record?
- Detected, with a note that automation handled the alert.
- Missed, because no person ever saw the alert.
- Alerted but not triaged, with the reason closed by automation.
- Triaged but not acted on, because closure counts as triage.
What does the detection owner receive before writing the expectation sheet?
- The list of techniques in the scenario, and nothing of the plan, order or timing.
- The full emulation plan, so that the expectations match each step.
- The emulation log, so that expectations can be checked against it.
- The tester's draft report, with the findings removed by the white team.
Why does the chapter record the layer at which telemetry disappeared for a no telemetry outcome?
- Because the layer sets the severity rating given by the tester.
- Because supervisors require the layer in the attestation.
- Because the correlator cannot join steps without a layer field.
- Because the layer decides who owns the fix.
Chapter 10 · 90 minutes
Improve: From Finding to Change in Detections, Controls, Playbooks and Mandates
The lesson covers P4 Improve, the stage in which a finding becomes a change. Participants route a finding by the layer that failed to a detection, control, playbook, mandate or recorded risk acceptance, assign an owner who is not the scenario's author, review AI-drafted changes for overfitting and shared blind spots, count false-positive cost by replay and shadow, order the backlog, and record findings that no one can fix.
Learning objectives
- Classify a finding by the layer that failed and choose the route, giving a reason for each route rejected.
- Explain the ownership rules, including who must not own each kind of change.
- Review an AI-drafted change against the five review questions and the three risks specific to the loop.
- Draft a finding-to-change decision that a second person could reproduce.
Key concepts
- Five routes
- Detection, when activity was visible and no detection fired or it was routed wrongly; control, when a preventive control was absent or bypassable; playbook, when response was slow or wrong; mandate, when an agent, role or provider acted outside or short of its authority; recorded risk acceptance.
- Change record
- The CHG record of P4: the finding, the route, the decision with routes considered and urgency, the owner, the artefact, who drafted and who reviewed, cost, rollout, and the regression case handed to P5 Verify.
- Alert budget and tune-or-retire rule
- Each detection has a budget of alerts per week. A change is replayed over at least 30 days of telemetry, runs in shadow for 14 days, and is reviewed or rolled back if it exceeds its budget or twice its budget for two consecutive weeks.
- Finding half-life
- The number of days after which half of the findings opened in a quarter have a passing re-test; it counts only verified closure. Findings are sorted into urgent, standard and deferred, with default targets of 14, 30 and 90 days, by technique importance and evidence age, what the defence did and what is at stake.
- Recorded risk acceptance
- A decision by a role with authority over the residual risk, for a stated period, with a reason and a review date. The person who would do the work does not accept the risk, and the coverage record keeps showing the technique as failed or declared.
Interactive exercises
- Match: Five routes for a finding
- Scenario: A finding with no telemetry
Done on the lesson page, with feedback at once.
Discussion
- Lowering an agent's level takes effect at once, while raising it needs a named authority and a re-test. Why does the chapter build that asymmetry into review?
- A loop that turns every finding into a rule ends with an alert queue nobody can read. Who should decide that an older rule is retired to make room for a new one?
- Nordvik Region cannot edit its provider's closure behaviour. What can a buyer of the loop do at P4 Improve, and where does its leverage end?
Exercise · 35 minutes
A reproducible finding-to-change decision
Take one finding from your own register or use FND-NR-2027-011 from Chapter 9. Following the nine steps of the practical guide, restate the finding in behaviour terms with the technique identifier and evidence reference; name the layer that failed and list the routes considered with a one-line reason for each not chosen; choose the route and split by route with an owner who is not the scenario's author; set the urgency from the three questions and a target date; state the expected alert cost, budget and shadow period; name the reviewer and record what an AI system drafted; identify the regression case with at least one variant; and set the review date. Then hand steps one to eight to another pair and compare the route they reach.
Deliverable: A finding-to-change decision record with route, owner, reviewer, cost, regression case and expiry, plus a note of the reproducibility test's result.
Self-test
A finding shows that activity was detected but the response was too slow and too wide. Which route does the chapter's table indicate?
- Detection: rewrite the logic of the rule that fired.
- Playbook: change the playbook manifest, runbook or decision right.
- Control: add a preventive control in front of the service.
- Mandate: narrow the mandate of the tester who ran the scenario.
Who must not own the detection change that answers a scenario, according to the ownership table?
- The detection-engineering lead of the organisation.
- The purple-team coordinator who keeps the register.
- The tester who wrote the scenario that it answers.
- The platform team that runs the log pipeline.
What is the status of a finding whose detection change has merged but whose regression case has not yet run?
- Implemented, unverified, with the coverage cell still shown as failed.
- Closed and verified, with the merge date recorded as the date of closure in the register.
- Risk accepted, until the next scheduled run of the pipeline.
- Deferred, with a 90-day target for the re-test.
Chapter 11 · 90 minutes
Verify: Re-test, Regression and Evidence
The lesson covers P5 Verify, the stage at which the loop closes or fails to. Participants apply the four clauses of the closure rule, design a re-test with variants of path, context and rate, a held-out variant and a negative control, keep it as a regression case that runs on change, schedule and intelligence, produce an evidence manifest that expires by clock and by condition, and report the denominator, the reasons and the age.
Learning objectives
- Explain the four clauses of the closure rule and the failure each one answers.
- Design a re-test with a pass criterion, an independent runner, variants and a negative control.
- Draft a regression case and the fields of an evidence manifest for one finding.
- Review a verification report for its denominator, its reasons for unverified techniques and its evidence ages.
Key concepts
- Four clauses of closure
- A finding is closed only when the re-test and its variant pass; the evidence is attached; a named person signs; and closure has an age, so that when the condition lapses the finding is unverified again.
- Variants
- Path variants change how the technique is carried out, context variants change where and when, and rate and shape variants change volume and timing. A held-out variant is written by someone who did not write the fix and is not shown to the fix author.
- Negative control
- A benign look-alike that must not alert, included in each case because a detection can pass its re-test by alerting on everything.
- Regression case as ratchet
- A re-test that has been kept (REG): the exact scenario, its variants and the negative control, run on change, on schedule and on intelligence. A case once added is retired only by a recorded decision when its control is retired.
- Evidence manifest
- The unit of proof (EVD): generated by the run and not written, content-addressed, versioned with its dependencies, signed by a nameable person and minimised. It expires by a clock and by a condition, such as a change to a dependency.
Interactive exercises
- Put in order: From finding to closure, and after
- Sort: Three kinds of variant
Done on the lesson page, with feedback at once.
Discussion
- Under a rule that closed findings on a passing exact re-test, Nordvik Region's finding would have been closed in November 2027. What would the Region have reported to its audit committee, and what would it have missed?
- A green run is itself an automation output that people learn to trust. What keeps a nightly regression suite from being read by nobody?
- Evidence that never expires is not credible, the chapter says. How would you explain to a system owner that a finding closed last spring is now unverified again?
Exercise · 35 minutes
From closed finding to regression case
Choose one finding that your register, or the Nordvik Region or Hansa Mutual material, calls closed. Following the practical guide, restate the hypothesis: technique, environment and pass criterion with its time bound and layer. Record the exact scenario as the base case. Describe at least three variants for a priority technique, one of path, one of context and one of rate or shape, each at the level of technique and telemetry and never of procedure. Name who will author the held-out variant, add one negative control, and confirm for each scenario its safety class, tag, clean-up step and stop-path holders. Set the triggers, the cadence and the evidence expiry, and list the versions the evidence manifest must record.
Deliverable: A regression case record with base case, variants, held-out variant, negative control, triggers, expiry and a named closure signer.
Self-test
Why is re-running the exact scenario necessary but insufficient for closure?
- Because the exact scenario can only be run by the original tester.
- Because supervisors accept only re-tests that were run in a pre-production mirror and signed by the tester.
- Because the exact scenario expires as soon as the change is merged.
- Because a detection written from a finding tends to fit that finding and can miss a variant.
Who signs the closure of a finding at P5 Verify?
- The re-test runner, once the full regression case and its variants have passed at Level 3 Bounded.
- A named person in a named role; never an agent, and never the author of the fix alone.
- The pipeline, by rule, when the evidence manifest has been hashed.
- The contracted tester who reproduced the original finding.
Which two parts make up the expiry of evidence in the chapter?
- A weekly alert budget for the detection and a rule to tune or retire it after two weeks over budget.
- A denominator of priority techniques and a list of reasons.
- A clock since the last full pass and a condition that ends it early on a dependency change.
- A signature by the closure owner and a hash of each artefact.
Module 3 · Part III · The Loop Meets AI Systems
Part III turns the loop on AI itself. Participants apply all five stages to the organisation's own AI systems and agents, choosing scenarios by ATLAS technique against four classes of target and building detections from the agent's manifest, and then to the defenders' AI: SOC agents, playbooks, the stop path and a provider's AI, tested with seeded incidents, benign twins and an injected-test clause in the contract. They leave able to write both artefacts.
Chapter 12 · 90 minutes
Purple for AI Systems and Agents: Attacks on Models and Detections That Catch Them
The lesson applies the purple loop to the organisation's own AI systems and agents. Participants choose scenarios by ATLAS technique against four classes of target, read what each telemetry source can and cannot tell the SOC, write a detection record for an agent that exceeds its permission ceiling, carry an AI red-teaming finding to a detection and a regression case, and distinguish a passing evaluation from a running detection.
Learning objectives
- Classify AI-system scenarios into the four classes of target by ATLAS technique identifier.
- Compare what each telemetry source records with its blind spot, and explain why the agent's narrative is not telemetry.
- Draft a detection record for an agent that exceeds its permission ceiling, with two separate conditions.
- Explain the difference between an evaluation and a detection and how each decays.
Key concepts
- Four classes of target
- The model and its instructions (AML.T0051 LLM Prompt Injection), retrieval and the index (AML.T0070 RAG Poisoning), tools and actions (AML.T0053 AI Agent Tool Invocation), and data and outputs (AML.T0057 LLM Data Leakage).
- Compound system
- The unit of test and of detection: the model, the tools it can call, the data it can read and the identity it acts under. The weak link is often not the model but the control or identity around it.
- Permission-ceiling breach
- A call outside the agent's manifest. A call the policy point denies shows the control worked and is a detection input; an allowed call outside the ceiling is more severe. The two conditions are kept separate in the detection record.
- Evaluation versus detection
- An evaluation asks whether a version resists chosen cases at a gate; a detection asks whether something is happening in production that should not be and who hears of it. A passing evaluation is not a running detection.
- Three families of agent detection
- Tool misuse, anomalous retrieval and permission-ceiling breaches. Expected behaviour is written in the manifest, and a baseline is added only for what the manifest cannot state.
Interactive exercises
- Match: Telemetry sources and their blind spots
- Scenario: An allowed call outside the ceiling
Done on the lesson page, with feedback at once.
Discussion
- Nordvik Region holds a declared, not verified, position for the supplier's scoring component. What would it take, in contract and in test, to move that position to verified?
- The chapter concludes that detections cover an agent acting beyond its mandate but not an agent steered within it. What follows for how narrowly a mandate should be written?
- When a team reports that its AI system has been red-teamed, which follow-up questions separate an evaluation result from a detection that works today?
Exercise · 35 minutes
A detection record for a ceiling breach
Take the permission manifest of an agent in your organisation, or the description of Hansa Mutual's claims-triage agent, whose tool calls are bound to the market of each claim. Following the practical guide, list each permitted tool, action, resource scope and budget and write one adjacent action that must be refused. Confirm which telemetry must be present: the policy log and the tool-side audit log, each with a maximum silence. Write two separate conditions, A for a policy deny under a ceiling rule and B for an allowed call outside the ceiling, with severity and routing for each, including who may revoke the agent's credentials. State the positive and negative tests, the owner, an approver who did not write the scenario, and the coverage expiry.
Deliverable: A detection record with two conditions, data sources with maximum silence, tests, routing, owner, approver and a 90-day expiry for a priority agent.
Self-test
A scenario tests whether a tool can be invoked with parameters the permission manifest does not allow. Which class of target and ATLAS technique does the chapter assign?
- Retrieval and the index; AML.T0070 RAG Poisoning.
- Tools and actions; AML.T0053 AI Agent Tool Invocation.
- Data and outputs; AML.T0057 LLM Data Leakage.
- The model and its instructions; AML.T0051 LLM Prompt Injection.
Why does the chapter say the agent's narrative is not telemetry?
- Because the narrative is stored in a format that the SIEM cannot parse.
- Because the narrative contains personal data that the GDPR forbids the SOC to collect or read.
- Because the narrative is always shorter than the tool-call log.
- Because an agent's summary of its own work is a model output that can be wrong.
What does a pass mean for a detection, in the chapter's comparison of evaluation and detection?
- Nothing fired; the absence of an alert is not evidence of absence.
- The version resisted every case in the suite at the release gate.
- The model has been shown safe against the four classes of target.
- The SOC has confirmed that no adversary is present in the estate.
Chapter 13 · 90 minutes
Testing the Defenders' AI: SOC Agents, Playbooks and Providers' AI under Emulation
The lesson uses the loop to test the blue team's own AI. Participants distinguish six subjects, triage, investigation and drafting agents, the playbook layer, the stop path and a provider's AI, and test them for correctness, grounding, restraint and robustness to hostile input with seeded incidents and benign twins derived from emulation. They write an injected-test clause for a managed-service contract and learn why a level falls at once and rises by decision.
Learning objectives
- Explain the six subjects of testing and where the ground truth for each comes from.
- Design a seeded-incident register with replayed telemetry, live S1 seeds, benign twins and a held-out share.
- Draft an injected-test clause for a managed-service contract with rates, scoring, evidence and a stop right.
- Review a finding against an agent mandate and state what happens to the agent's level.
Key concepts
- Four properties under test
- Correctness, where the output agrees with the ground truth; grounding, where every claim can be traced to the evidence; restraint, where the system does no more than its mandate allows; and robustness to hostile input. Restraint comes first wherever an action is delegated.
- Seeded incident from emulation
- A known case injected into the queue, derived from a scenario record so that the correct disposition is fixed before the run. It takes three forms: replayed telemetry, live emulation observed by the agent, and benign twins.
- Benign twin
- A variant that produces telemetry similar to a malicious scenario from a legitimate change, such as a rollout using a service account from new hosts. It tests whether the agent refrains from acting on a lookalike.
- Injected test
- A seeded incident that the customer causes to arrive in the provider's pipeline. The provider knows that tests exist, their number per class and the scoring, but not the identity, timing or content of any seed.
- Level falls at once, rises by decision
- A failed test lowers the agent's level for the affected action immediately. It is restored only by a named authority after a passing re-test that includes variants and the benign twin.
Interactive exercises
- Match: Six subjects and the question each test asks
- Sort: Four properties under test
Done on the lesson page, with feedback at once.
Discussion
- The chapter's own claim is that a delegated level has to be earned repeatedly. Which of your organisation's delegations to an AI system were shown once and never again?
- At Hansa Mutual the agent, its tests and its tuning are all on the same platform. What makes the seeds independent in that setting, and what would erode it?
- A round of fifteen seeds cannot show an escape rate of one in a thousand. How should such results be worded for an audit committee so that they are not over-read?
Exercise · 35 minutes
An injected-test clause and schedule
Write an injected-test clause for the contract with a managed provider whose AI closes alerts for your organisation, or for Nordvik Region's annex MS-2. Following the eight steps of the practical guide, list the closure classes the provider's AI may act on; set the number of seeds per class per quarter; state what the provider may and may not know; give the scoring and thresholds in numbers, including what one escape and two escapes in a class trigger; require the evidence to be returned for every seeded closure within a stated time; give the customer a stop right with a yearly out-of-hours drill; say what happens after a failure; and protect the seeds as security-sensitive. Then outline the first round: classes, seeds, scenario sources by identifier and who authors the seeds.
Deliverable: A contract clause of about one page and a first-round schedule from which a reader can say, per class, how many seeds were injected and how they are scored.
Self-test
Who authors the seeds used to test an agent, according to the chapter's rules for keeping seeds honest?
- The platform team that operates and tunes the agent under test, since it knows the agent's weak points best.
- The agent itself, from its own record of earlier cases.
- A party who does not tune the agent under test, with a share the operators have not seen.
- The managed provider whose AI is being tested in the round.
An agent holding Level 3 Bounded for an action fails a seeded test on that action. What does the chapter prescribe?
- The level falls to Level 2 at once and is restored by a named authority after a passing re-test.
- The level stays at Level 3 Bounded until the group AI governance board has met for its next quarterly review.
- The agent is retired and its mandate is permanently withdrawn.
- The seed is rewritten so that the agent can be re-tested immediately.
What may the provider know about injected tests under the clause the chapter describes?
- The identity and timing of each seed in advance, so that its analysts on shift are not disturbed.
- Nothing at all, because any knowledge would invalidate the test.
- The content of each seed after the round, for training its AI.
- That tests exist, the number per class and the scoring, but not which alerts or when.
Module 4 · Part IV · The Loop's Own AI
Part IV treats the AI that the loop itself uses. Participants design a team of narrow agents with mandate manifests, permission ceilings, budgets and a drilled stop; apply the independence principle where AI sits on both sides of a check, with held-out scenarios, seeded gaps and reproduction by a person; and protect the findings register, the scenario library and the emulation agents as tier-0 assets. They leave with a mandate manifest, an independence statement and a classification policy.
Chapter 14 · 90 minutes
An AI Agent Team for the Loop: Skills, Mandates and Containment
The lesson designs the loop's own AI as a team of up to eight narrow functions: scenario curator, emulation planner, emulation runner, observation correlator, detection drafter, finding triager, re-test runner and critic. Participants set the level per function with four questions, write a mandate manifest with a permission ceiling, identity, budgets, circuit breakers, a never list and a separation list, and decide what stays with people.
Learning objectives
- Classify the eight candidate functions by stage, side, what they read and what they write or do.
- Explain the four questions that set a ceiling and why only the two runners reach Level 3 Bounded.
- Draft a mandate manifest for an emulation runner that a policy decision point can enforce.
- Compare the delegation profiles of Nordvik Region and Hansa Mutual and the reasons behind each.
Key concepts
- Eight candidate functions
- Scenario curator, emulation planner, emulation runner, observation correlator, detection drafter, finding triager, re-test runner and critic. They are functions and not products; several may run in one system as long as the separations hold.
- Mandate manifest
- The written mandate for one agent (MND), in version control: named owner, operators and authority, the rules-of-engagement reference, the catalogue binding, the level per action class, identity, budgets, circuit breakers, a never list, a separation list, review and expiry.
- Critic
- A function on a second model family at Level 0 Advisory that challenges the planner's choices, the correlator's outcomes and the drafter's detections. It reads outputs, never working stores, and it flags and questions but never gates.
- Budgets and circuit breakers
- Ceilings an agent cannot argue with: actions, targets, time and cost per run or period. Circuit breakers stop the run on conditions such as an action outside the permission ceiling, contact with an out-of-scope asset or real personal data in a result.
- Pause, revoke, undo
- The three forms of stop: the policy decision point stops admitting steps; the identity provider and egress gateway end the agent's credentials and routes from outside the runner; and the clean-up manifests are executed and confirmed.
Interactive exercises
- Match: Loop functions and what they write or do
- Scenario: A step that is not in the catalogue
Done on the lesson page, with feedback at once.
Discussion
- Nordvik Region delegates only the critic and the correlator and buys no runner. What would have to change, in capacity and in contract, before a Mode 5 runner made sense for it?
- The triager ranks and routes but does not close. What damage could a triager do without ever exceeding that mandate, and how would you see it?
- A team of eight functions invites one model gateway, one secret store and one log store. Which shared components in your own tooling would make two controls fail together?
Exercise · 35 minutes
A mandate manifest for an emulation runner
Following the ten steps of the practical guide, write a mandate manifest for an emulation runner in your own organisation or for Hansa Mutual's runner. Start from a signed rules-of-engagement manifest and make the mandate no wider. Name one owner, at least two operators, the authority that grants the level and the stop holders. Bind the catalogue, starting with S1 only, read-only to the agent. Set the level per action class. Give the agent a workload identity, test accounts only and short-lived tokens issued outside its runtime. Set budgets for actions, targets, runs and minutes. Write the circuit breakers, the never list and the separation list. Set review and expiry, and list the three planted conditions the policy decision point must refuse before the first run.
Deliverable: A mandate manifest of one to two pages with named people, ceilings, budgets, circuit breakers, never and separation lists, and three ceiling tests.
Self-test
Which functions reach Level 3 Bounded in Hansa Mutual's delegation profile, and for what?
- The curator and the planner, for proposing scenarios from intelligence.
- The correlator and the triager, for classifying and routing findings.
- The emulation runner and the re-test runner, for catalogued S1 scenarios inside the signed scope.
- The detection drafter and the critic, for merging reviewed detections into the production repository without a person.
What does the chapter say the critic may do?
- Flag and question the outputs of other functions, on a second model family, without gating.
- Approve or reject detections before they are merged to production.
- Read the working stores of the other agents on the same model family in order to find their hidden errors.
- Stop the emulation runner when its outcome looks implausible.
Which of the following is delegated to no agent at any level, according to the chapter?
- Assembling and hashing the evidence manifest after a codified S1 regression case has been run.
- Drafting a detection as a pull request for review.
- Proposing the owner and priority of a new finding.
- Authorisation, the stop, the closure of a finding and the authorship of novelty.
Chapter 15 · 105 minutes
Independence: Verifying the Loop When AI Is on Both Sides
The lesson develops the independence principle: a check counts as verification only when its failure modes are shown not to be those of the thing it checks. Participants analyse correlated failure, separate the five kinds of independence, use held-out scenarios, seeded gaps, layering against test-aware behaviour and reproduction by a person, write a quarterly independence statement, and work through the debate on whether a loop can be closed by machines alone.
Learning objectives
- Explain correlated failure and how it inflates a coverage record, separating what the research measured from the book's inference.
- Classify the dependencies of a chain of tester, tested and checker into the five kinds of independence.
- Draft a one-page independence statement for a coverage claim, with controls, evidence, residual risk and expiry.
- Review the debate between concern and calm and state the chapter's verdict.
Key concepts
- Correlated failure
- The case where tester and tested share a model family, training data, authors, vendor or time, so that their errors coincide and a passing result shows agreement and not verification. The inference from the cited research to the loop is the book's own.
- Five kinds of independence
- Model, data, authorship, vendor and time, each with a shared dependency, what it hides, a control and what is shown. The kinds are not ranked, and a kind that is not shown is treated as shared.
- Held-out scenario
- A scenario written by a person, kept where the detection side and its agents cannot read it, and run only after the detections are built. The gap between open and held-out pass rates measures generalisation, and the set is where human-authored novelty enters.
- Seeded gap
- A known weakness introduced on purpose, by a person, under the signed rules of engagement and in a place where it can be undone, to test whether the loop that is meant to find gaps finds this one, how fast and by what.
- Independence statement
- A one-page, signed record per coverage claim: the claim, the chain, the shared dependencies, the controls and their evidence, the residual risk in words, the evidence age and an expiry, by default 90 days for priority techniques.
Interactive exercises
- Match: Five kinds of independence and what sharing hides
- Scenario: A shared model family across the contract
Done on the lesson page, with feedback at once.
Discussion
- The chapter says Nordvik Region's independence is by contract and cheap to show, while Hansa Mutual's is by design and cheap to lose. What does cheap to lose look like in day-to-day engineering?
- Simulation markers protect against mistaking an incident for an exercise, yet a marker visible to an agent under test weakens the test. How would you resolve that tension in your own design?
- If independence checks catch nothing over a long period, either the loop is sound or the checks are too weak. How could you tell which?
Exercise · 30 minutes
An independence statement for one claim
Choose one coverage claim that your board relies on, or use Hansa Mutual's seven verified priority techniques whose scenario and detection shared a chain. Following the practical guide, name who or what wrote the scenario, wrote the detection and judged the result. For each agent in the chain record the model family, provider, platform and retrieval corpus, or write not declared. Mark each of the five kinds as shared, separated by control or not shown. For each shared dependency name the control you would apply. State which results a person must reproduce, including a 10 per cent sample of other re-test passes. Write the statement on one page and say whether the claim is verified, verified with limited independence or downgraded.
Deliverable: A one-page independence statement with chain, five kinds, controls, residual risk, evidence age, expiry and a named signatory.
Self-test
Which control does the chapter pair with a shared model family between tester and tested?
- A re-test run immediately after the fix, on the fix author's own data and by the fix author's agent.
- A critique agent on a second model family, used for questions and not verdicts.
- A longer retention period for the exercise telemetry.
- A higher autonomy level for the re-test runner.
What is the chapter's verdict on a loop closed by machines alone?
- It is unsafe in every respect and regression should return to people.
- It is sufficient once two different model families are in use.
- It verifies coverage for as long as the pass rate on the open scenario library stays above 90 per cent.
- It can show that the known still works, and cannot by itself show that its coverage is real.
Hansa Mutual's statement IND-HM-2028-Q1 assessed seven of 25 verified cells. How does the chapter say the board figure of 25 should then be read?
- As seven cells failed or at limited independence, and eighteen not yet assessed.
- As 25 verified cells, since the statement found no failures at all.
- As seven verified cells, because the other eighteen are now unverified.
- As 18 verified cells, with the seven assessed cells removed from the board figure until the next review.
Chapter 16 · 90 minutes
Defending the Loop's AI: Scenario Libraries, Findings and Emulation Agents as Tier 0
The lesson treats the loop's own assets as tier 0: the findings register, the scenario library, emulation plans, coverage records and agent credentials. Participants classify findings into handling classes FC-1 to FC-4, segregate a register of record with access policy as code, apply the five controls to the emulation agent as a privileged insider, secure the supply chain of scenarios, decide what may reach an AI tool, and design detections of misuse.
Learning objectives
- Explain why the loop's records are valuable to an adversary and to whom each asset is legitimately needed.
- Classify findings into the handling classes FC-1 to FC-4 by what their content would let a reader do.
- Draft a classification and access policy for a findings register of record, including agents and hosted models.
- Design detections of misuse of the loop's assets and the evidence that the protection works.
Key concepts
- Tier-0 set of the loop
- The findings register, the scenario library, emulation plans and rules of engagement, coverage records, and agent credentials, mandates and connectors. Together they are an adversary's best map of the estate and the records on which coverage claims rest.
- Handling classes FC-1 to FC-4
- FC-1 Aggregate: counts and trends with no asset named. FC-2 Redacted: technique, severity, owner and status. FC-3 Restricted: asset-specific detail of an open finding. FC-4 Exploitable: anything that would shorten an adversary's work, such as unreleased gap lists.
- Register of record
- The authoritative findings store, in its own store with its own identity group: append-only writes with an attributed change log, access policy enforced at a policy decision point, references in place of copies elsewhere, and second-person approval for exports.
- Emulation agent as privileged insider
- An agent that holds working credentials and tooling inside the estate. Five controls sit outside it: identity of its own, task-scoped credentials issued outside the runtime, default-deny egress and a separate range, no standing access to open findings, and an external stop path.
- Deterministic redaction gateway
- A rule-based gateway that replaces assets and accounts before a call to a model. FC-4 never goes to a model, FC-3 only to a model in the organisation's own environment, and a model asked to remove asset names has already seen them.
Interactive exercises
- Sort: Handling classes FC-1 to FC-4
- Scenario: A finding and a hosted assistant
Done on the lesson page, with feedback at once.
Discussion
- At Nordvik Region the working copy of exercise findings lived in the managed provider's ticket system. Where do the findings of your last test live today, and how many people can read them?
- The access review is the control that lapses first when a small team is busy. How do you design a control so that its lapse is visible?
- The chapter handles insider risk by separation of duties and attributable records rather than by covert monitoring. What does that choice protect, for staff and for the loop?
Exercise · 35 minutes
Classifying and segregating a findings register
Following the practical guide, inventory every place a finding lives in your organisation today, or in Nordvik Region as the chapter describes it: tickets, mail folders, tester portals, shared drives, chat channels, slide decks and backups, with a count of people who have access. Choose a register of record and its identity group. Adopt the four handling classes and write the default audience of each. Write the access policy as a short structured text: roles and what each may read and write, each loop agent by name with its reads and writes, what hosted models may receive, the export rule for FC-3 and above, the access review cadence and when a tester's access expires. List three tests that would evidence the policy.
Deliverable: An inventory of finding locations and a one-page classification and access policy with roles, agents, hosted-model rule, review cadence and three evidence tests.
Self-test
Into which handling class does the list of priority techniques with no passing test fall?
- FC-4 Exploitable, for a named handful of people with a second person for any export.
- FC-1 Aggregate, because it is a summary for the board.
- FC-2 Redacted, because it names only technique identifiers and severities and not the affected assets.
- FC-3 Restricted, because it concerns open findings only.
What access does the emulation runner have to the findings register in the chapter's design?
- Read access to the FC-3 findings for the systems in its signed scope, so that it can choose its next scenario.
- Full read access, so that it can avoid re-testing closed findings.
- No read of the register; it appends results and cannot delete, reclassify or export.
- Write access to change a finding's classification after a run.
Why must redaction happen deterministically before a finding is sent to a model?
- Because models cannot process structured finding records.
- Because a model asked to remove asset names has already seen them.
- Because supervisors require redaction to be done by a person.
- Because redaction by rule is cheaper than a model call.
Module 5 · Part V · The Loop in the Organisation
Part V puts the loop into the organisation. Participants replace a technique heat map with coverage records that carry age and expiry, design a board page that activity cannot improve, choose an operating model per stage with role charters and exit criteria, assign ten decision rights and test them in a tabletop, map legal obligations to controls and evidence, and write the consent, notice, sharing and disclosure protocols for a loop that crosses organisational boundaries.
Chapter 17 · 90 minutes
Coverage as a Claim: Technique Maps, Evidence Age and Detection Decay
The lesson examines the word covered. Participants learn that a coverage claim has five parts, a technique, an environment, a result at a named layer, a date and an expiry, and that a heat map with no age is not a coverage statement. They build coverage records whose status is computed by rule into six values, handle decay, environments and variants, keep not tested honest, and report by denominator, reason and age.
Learning objectives
- Explain the five parts of a coverage claim and the three recurring failures of a heat map.
- Classify coverage records into the six statuses and say which counts as covered.
- Draft coverage records with expiry, environment, independence and declaration fields for a priority list.
- Review a coverage page for its denominator, reasons, age distribution and honest not tested lines.
Key concepts
- Coverage claim
- A statement that a specified technique was exercised in a specified environment inside a window of time, with a result at a named layer, a date and an expiry. A claim that lacks one of the five parts is not a coverage statement.
- Six statuses
- Verified, expired, implemented-unverified, failed, asserted and untested. The status is derived by a deterministic rule from the evidence; a person does not edit it, and only verified counts as covered.
- Tested versus asserted
- A test is an exercise the organisation or an independent party ran and whose result it holds. An assertion is a statement by a provider, vendor or owner. An assertion repeated in four documents is still one assertion.
- Decay
- The process by which a detection that worked when tested stops describing the estate, through change in the estate or in the adversary and the variant. The record guards against it with a clock and with conditions that end the evidence early.
- Declaration
- The act of a named person who declares a technique covered, with a date. The field is empty unless the status is verified; AI compiles and proposes, the rule decides the status, and a person declares.
Interactive exercises
- Put in order: The status rule: first match wins
- Scenario: Activity or coverage
Done on the lesson page, with feedback at once.
Discussion
- The most valuable line on a coverage page is the one that says not tested, because it is the hardest to write. What makes it hard in your organisation?
- Nordvik Region's risk is a page that is accurate on the day and not refreshed; Hansa Mutual's is a page refreshed so often that nobody reads its age. Which guard fits which risk?
- A risk-accepted finding keeps its technique at failed. What would a board conclude wrongly if accepted gaps were shown as covered?
Exercise · 35 minutes
From heat map to coverage records
Take a technique heat map from your own organisation, or the nine priority techniques of Nordvik Region's benefits-triage service. Following the practical guide, create one record per technique and environment. For each green cell look for the evidence: the scenario, the last run, the result at a named layer and the manifest; where none can be linked, mark the cell asserted or untested. Set the clock, 90 days for priority techniques and 180 for the rest or your own figures, and list the conditions that end the evidence early. Apply the status rule in order and write the coverage page: verified out of all priority techniques, the others by reason, the age distribution, and the oldest unexpired and oldest expired claims.
Deliverable: A set of coverage records with computed statuses and a one-page coverage page with denominator, reasons, age distribution and a named declarer.
Self-test
A provider states in its runbook that it handles a priority technique, and the organisation has run no test. Which coverage status applies?
- Verified, because the provider is contractually responsible.
- Expired, because the statement has no date of its own.
- Implemented-unverified, because a control exists somewhere.
- Asserted, which does not count as covered.
Who sets the status of a coverage record, according to the chapter?
- The coverage owner, after reviewing the last evidence manifest.
- A deterministic rule applied to the records; nobody sets it by hand.
- The coverage agent, which judges the strength of the evidence.
- The board, when it approves the quarterly coverage page.
What did Hansa Mutual's conversion of its 34-technique heat map show that the colours had hidden?
- Four cells were green although their evidence was past its expiry, because the map had no age.
- Nine cells were green although no scenario had ever been written for those techniques in the library.
- All 34 cells were green because the pipeline ran each technique monthly.
- Two cells were red although the detections had passed every re-test.
Chapter 18 · 90 minutes
Measuring the Loop: Cycle Time, Yield, Re-test Pass Rate and the Metrics That Mislead
The lesson asks what the loop should report about itself. Participants learn six measures that running more scenarios cannot improve, four common measures that mislead when they become targets, the four mechanisms of Goodhart's law as they apply to the loop, AI-specific measures, and the rules that keep every measure at the grain of the system and never of a named person. They design a board page with gaming checks.
Learning objectives
- Compare the six recommended measures with the four that reward activity, giving the replacement for each.
- Explain the four Goodhart mechanisms with an example from the loop for each.
- Draft measure catalogue entries with definition, denominator, grain and gaming check.
- Design a board page of four to six measures that respects the aggregation rules agreed with employee representatives.
Key concepts
- Six measures of the loop
- Finding-to-control cycle time, finding half-life, re-test pass rate at first attempt with held-out variants, verified share within expiry, yield per exercise and detection debt. Each has a denominator and cannot be improved by running more scenarios.
- Measures that mislead
- Scenarios run, findings opened, findings closed and a green percentage. They are useful as operational data on cost and load but reward activity when they become targets or headline figures.
- Goodhart mechanisms
- Regressional, extremal, causal and adversarial, after Manheim and Garrabrant. AI sharpens the adversarial case: a drafter rewarded for passing scenarios produces detections that pass scenarios.
- Yield per exercise
- Findings per exercise that no catalogued run could have produced, and how many reach verified control within 90 days. It is the only one of the six that measures what the people on the red side add.
- Grain and minimum group size
- Every measure is computed per technique, scenario, class, service or team, never per named person. A role held by one person is that person. Hansa Mutual's floor is five people; Nordvik Region reports the unit of six as a whole.
Interactive exercises
- Sort: Measures of the loop and measures of activity
- Scenario: A figure by owner
Done on the lesson page, with feedback at once.
Discussion
- Nordvik Region's first closure after 28 days would have been the headline; the verified closure took 112 days. Which of your own headline figures would change if only verified closure counted?
- The chapter states the price of aggregation openly. How should a manager find out who needs support without turning the loop's records into a ranking?
- Step nine of the guide doubles the scenarios run in a test quarter to see whether any figure moves. What would that test do to your present dashboard?
Exercise · 35 minutes
A board page that activity cannot improve
Design a board page for the loop of your own organisation or of Hansa Mutual. Start from the six measures of the chapter and the decisions a board can take: fund work, extend or hold a delegation, accept a named gap until a date. Choose four to six measures and write a catalogue entry for each: definition, source records, denominator, grain, gaming check and the count shown beside the figure. Attach the age to each coverage figure, including the techniques that expire within 30 days. Put the first-attempt re-test pass rate beside the pass rate on held-out variants. State the grain allowed and forbidden, the minimum group size and that no result enters an appraisal. Limit the decisions block to three, each traced to a measure.
Deliverable: A one-page board page layout with four to six measures, a catalogue entry per measure and a decisions block of at most three.
Self-test
Which measure does the chapter recommend in place of findings closed?
- Scenarios run per quarter, with the share above S1.
- The green percentage over the whole scenario library.
- Finding half-life, with the recurrence rate beside it.
- Findings opened per exercise by the red side.
At what grain may a measure of the loop be computed under the chapter's rules?
- Per technique, scenario, class, service or team, never per named person.
- Per analyst, provided the figures are shown only to the CISO.
- Per shift, so that out-of-hours performance can be compared.
- Per role holder, including roles that are held by one person.
Which Goodhart mechanism is at work when a detection drafter rewarded for passing scenarios produces detections that pass scenarios?
- Regressional.
- Extremal.
- Causal.
- Adversarial.
Chapter 19 · 90 minutes
Operating Models and Team Design: In-House, Contracted, Hybrid and Sovereign Loops
The lesson asks who runs the loop. Participants compare four operating models, in-house, contracted, hybrid and sovereign or sector, decide the model per stage with a decision record and exit criteria, charter the seven roles of the loop, apply independence between roles even in a team of six, and learn what procurement must secure: ownership of scenarios and findings, declared AI, return of data and the right to stop and leave.
Learning objectives
- Compare the four operating models by who runs each stage, what is bought and what is left exposed.
- Draft an operating-model decision per stage with exit criteria stated as numbers.
- Draft a role charter with mandate, decision rights, prohibitions and the agents under the role.
- Review a small team's role assignments against the forbidden pairs and name the compensation for each unavoidable pair.
Key concepts
- Four operating models
- In-house, contracted, hybrid and sovereign or sector. An operating model answers per stage who does the work, who is accountable and where the records live, and an organisation usually runs a mixture.
- What must stay
- Whatever the model, the organisation keeps the coordinator, the closure owner, the white team, the register of record and the right to stop. The model buys capacity and a different view.
- Role charter
- A versioned ROLE artefact with a holder and deputy, the mandate, accountabilities, decision rights, prohibitions and the agents under the role. An AI system does not hold a role; a person holds it and answers for what agents do under it.
- Forbidden pairs
- Pairs of roles or acts that the same person must not hold, such as authoring a scenario and the detection it tests, or authoring a change and closing its finding. Where unavoidable, the compensation is countersignature, sampling, a held-out scenario or a contract clause.
- Viable minimum loop
- A loop in which the five stages exist, each stage has an owner and a deputy, and the forbidden pairs are kept apart by contract, countersignature or sampling. Nordvik Region's uses two own people, two counterparties, two borrowed people and sampling by internal audit.
Interactive exercises
- Match: Roles of the loop and what each never does
- Scenario: A tester's terms
Done on the lesson page, with feedback at once.
Discussion
- The coordinator can escalate a stale finding without approval and cannot close one. Why does the charter grant the first right and refuse the second?
- Nordvik Region's loop rests on five people, none full time, and two counterparties. Where is its single point of failure, and what would you do about it first?
- The chapter says buying the loop is buying a right to learn from a result and not only a service. Which clauses in your current testing contracts secure that right?
Exercise · 35 minutes
Operating-model decision and a role charter
For your own organisation or Nordvik Region, list the five stages and the party that runs each today. Mark what must stay with the organisation. Score your preferred model per stage on whether it can show evidence within expiry, whether its independence can be shown, and whether the organisation could exit. Write the decision as a short record with an owner, a review date within twelve months and exit criteria as numbers, and test the exit on paper: what moves, to whom, in how many working days. Then write a role charter for the purple team coordinator: holder and deputy, mandate, accountabilities, decision rights, what the role may never do, and the agents under it.
Deliverable: A per-stage operating-model decision with numeric exit criteria and a one-page role charter with decision rights and prohibitions.
Self-test
Which exposure does the chapter attach to the in-house operating model?
- A loop that starts only when the contract renews.
- Overfitting to one's own habits, and a loop that nobody outside questions.
- Dependency on a shared sector platform that the organisation neither operates nor controls on its own.
- Two sets of records that disagree at a seam.
A small team cannot avoid one person authoring the P4 change and also closing the finding at P5. What compensation does the chapter give?
- None is accepted; the organisation must hire a second analyst.
- The purple team coordinator closes all findings in that person's place.
- Countersignature by a named second person and a sample by internal audit.
- The contracted tester signs the closure on the organisation's behalf.
What does the chapter say about an AI system and a role?
- An AI system does not hold a role; a person holds it and answers for the agents under it.
- An AI system may hold the coordinator role at Level 1 Assisted.
- An AI system may hold a role if its mandate manifest names an operator, an owner and a stop holder.
- An AI system may hold the closure owner role for S1 regression cases.
Chapter 20 · 105 minutes
Decision Rights, Authorisation and the White Team
The lesson assigns the decisions of the loop. Participants work with ten decisions, D1 to D10, each with a holder, a deputy, a time, an evidence standard and a record; examine what an authorisation covers; position the white team and the trusted agent; separate the right to stop from the decision to restart; route risk acceptance by tier with an expiry; and rehearse a real incident during an exercise in a tabletop.
Learning objectives
- Explain the ten decisions of the loop and the four rules that govern the decision-rights file.
- Draft a decision-rights file with holders, deputies, times and records, checking the forbidden pairs.
- Classify residual findings into the three acceptance tiers and name who may not accept.
- Run a tabletop with injects for an exercise that becomes an incident and a real incident behind an exercise.
Key concepts
- Ten decisions D1 to D10
- Authorise an exercise; change scope during a window; approve a run above S1; stop; restart; declare an event real or pause for deconfliction; raise a delegation level; accept a residual finding; declare a finding closed and verified; declare a technique covered.
- Decision-rights file
- The DRF record: the table of decisions in machine-readable form with the white team, the stop holders and the acceptance tiers. It is versioned, changed by pull request, approved by the management body and tested in a tabletop.
- Stop and restart
- Any named stop holder may stop without approval or justification, through a route outside the exercise platform. Restart is not the mirror of stop: it needs the white-team lead and the system owner, after review, at a lower level.
- Acceptance tiers
- Tier 1, low or medium severity and not a priority technique, accepted by the head of the owning function for up to 180 days; tier 2, high severity or any priority technique, by the CISO for 90 days; tier 3, critical or beyond appetite, by the management body.
- Tag ledger
- The record that links tagged emulation actions to the exercise. No alert is closed as an exercise without a match in the ledger, and it must be readable by the incident commander, the trusted agent and the provider's shift lead without going through the tester.
Interactive exercises
- Sort: Decisions of the loop and the stage each belongs to
- Put in order: When an emulation becomes an incident
Done on the lesson page, with feedback at once.
Discussion
- The right to stop is never routed upwards and needs no justification. What does an organisation risk by granting it so freely, and what does it risk by not doing so?
- An adversary who knows an exercise is running may rely on the defender to dismiss what it sees. How does the tag-ledger rule change the incentives of an analyst at two in the morning?
- Karin Lund keeps Nordvik Region's file but is excluded from authorisation, restart and acceptance. What does that exclusion protect?
Exercise · 35 minutes
Tabletop: an exercise that becomes an incident
Using the decision-rights file your group drafted, run a tabletop with a facilitator who holds none of the decisions. Play four injects in order: an alert with no match in the tag ledger; a tagged emulation with a real effect, such as a locked staff account; a holder who cannot be reached for a scope change; and a request to continue with the next scenario after a stop. For each inject, name the decision that applies, who holds it, the route used and the evidence relied on, and record the target time and the time your group actually needed. Anything over target is written as a finding with an owner, and the file is amended.
Deliverable: A tabletop record with target and actual time per inject, the gaps found, and the amendments proposed to the decision-rights file.
Self-test
Who may accept a high-severity finding or a finding on a priority technique, and for how long at most?
- The CISO, at tier 2, for at most 90 days.
- The head of the owning function, at tier 1, for at most 180 days.
- The closure owner for the class, for at most 30 days.
- The contracted tester, until the next catalogue run.
How does the chapter describe the decision to restart after a stop?
- Any stop holder may restart the exercise once the emulating system reports that its clean-up is complete.
- Restart is automatic when the abort condition is no longer measured.
- The red-team lead restarts at the same level after informing the white team.
- It needs the white-team lead and the system owner, after review, and resumes at a lower level.
What rule catches a real incident hidden behind an exercise?
- Every alert raised during an exercise window is closed as an exercise by default and reviewed after the window.
- No alert is closed as an exercise without a tag-ledger match, and a failed match freezes it.
- The tester confirms by telephone that each alert belongs to the plan.
- The blue team is told the plan in advance so that it can filter alerts.
Chapter 21 · 90 minutes
Regulation as Constraint: DORA, TIBER-EU, NIS 2, the AI Act, the GDPR and Employee Rights
The lesson reads the legal frame of the loop as design constraints, not legal advice: DORA and threat-led penetration testing, TIBER-EU, NIS 2, the AI Act, the Cyber Resilience Act, the GDPR for exercise data, staff co-determination and criminal law. Participants learn why continuous emulation is not a threat-led test, why the owner's authorisation separates an exercise from an offence, and how to map each obligation to a control, an evidence artefact and an owner.
Learning objectives
- Explain which regime applies to Nordvik Region and to Hansa Mutual and why the difference matters.
- Compare a threat-led penetration test under DORA with continuous emulation and explain how the two are kept apart.
- Classify obligations from the instruments by the stage of the loop they constrain.
- Draft obligation register entries that map an article to a control, an evidence artefact with an age, an owner and a review date.
Key concepts
- Threat-led penetration testing
- Under DORA Article 26, entities the competent authorities identify carry out TLPT at least every three years on live production systems, by testers who meet Article 27, under Delegated Regulation (EU) 2025/1190, with a mandatory replay and purple teaming exercise at closure.
- Hold list
- The list (HLD-HM-TLPT-01 at Hansa Mutual) that keeps the continuous pipeline from masking or revealing a covert threat-led test. Continuous emulation is not a TLPT, and its manifest, authority and evidence are kept apart.
- Management body duties
- NIS 2 Article 20(1) has the management body approve the cybersecurity risk-management measures, oversee their implementation and be capable of being held liable for infringements of Article 21; Article 21(2)(f) asks for policies and procedures to assess the effectiveness of the measures.
- Authorisation as the dividing line
- The Directive on attacks against information systems has Member States criminalise access without right. The owner's written authorisation for each asset is what separates an exercise from an offence, so the manifest carries a consent field per boundary.
- Obligation register
- The OBL record that maps each instrument and article to a restated obligation, the stage it constrains, the control, the evidence artefact with an age limit, an owner and a review date. An open entry with a question and a person is a legitimate state.
Interactive exercises
- Match: Instruments and what they ask of the loop
- Sort: Threat-led test or continuous emulation
Done on the lesson page, with feedback at once.
Discussion
- The Delegated Regulation on TLPT contains no provision on AI. What should an organisation do before planning an agent in a supervised test, and why ask first?
- Agreements with staff govern classes of exercise so that a covert test can be notified as a class and not as a date. What trust does that arrangement require on both sides?
- Nordvik Region keeps an open entry on the disclosure of synthetic content in a deepfake scenario under AI Act Article 50(4). When is holding a scenario the right design decision?
Exercise · 35 minutes
An obligation register for the loop
Write six entries of an obligation register for your own organisation, or extend OBL-NR-2028-01 or OBL-HM-2028-01. For each entry follow the practical guide: name the instrument, article, paragraph and point as the chapter gives them and the date on which you read the text; restate the obligation in one sentence in architectural terms, naming the addressee; name the stage of the loop it constrains and the control that meets it, a manifest field, a policy-point rule, a contract clause, a runbook step or a record; name the evidence artefact, its holder and its age limit; and name the owner, a deputy and the review date. Mark at least one entry open, with the question and the person who will answer it. Treat the entries as design implications, not legal advice.
Deliverable: Six register entries, each with instrument and article, restated obligation, stage, control, evidence with age, owner and review date, one of them open.
Self-test
How does the chapter relate continuous emulation to a threat-led penetration test under DORA?
- Continuous emulation replaces the TLPT once the pipeline covers every priority technique.
- A TLPT is one form of the loop and uses the same manifest as the pipeline.
- Continuous emulation is not a TLPT; the two are kept apart, and a hold list protects the test's secrecy.
- The pipeline must be stopped across the whole estate for the active phase of a TLPT and restarted after the attestation.
Which provision does the chapter cite for policies and procedures to assess the effectiveness of cybersecurity risk-management measures?
- DORA Article 27.
- NIS 2 Article 21(2)(f).
- AI Act Article 50(4).
- GDPR Article 28.
What, according to the chapter, separates an authorised emulation from a criminal offence?
- The use of benign, reversible and tagged scenarios of safety class S1 on the organisation's test tenant.
- The presence of a white team during the exercise window.
- A report delivered to the system owner after the exercise.
- The authorisation of the owner or right holder of each asset, recorded in the manifest.
Chapter 22 · 90 minutes
Across Boundaries: Suppliers, Managed Providers, Sector Exercises and Authorities
The lesson takes the loop across organisational boundaries. Participants distinguish three ways a test crosses a provider boundary, write a consent and notice protocol with standing consent, window notice and stop holders on both sides, read cloud providers' published testing rules, share detections with peers without exposing the target mapping, route vulnerability disclosure in both directions, handle cross-border entities, and assemble the evidence package for a supervisor.
Learning objectives
- Classify a test that crosses a provider boundary as pipeline, response chain or provider platform, and state the permission each needs.
- Draft a consent and notice protocol for a test across a provider boundary.
- Explain what may leave the organisation in sector sharing and what stays inside, by handling class.
- Design the disclosure routes for a vulnerability found in a supplier's product and for a report about one's own estate.
Key concepts
- Three boundary crossings
- Through the provider's pipeline, with a seeded incident in its queue; through the provider's response chain, with a scenario on one's own estate; and against the provider's platform, which is outside the organisation's authority and excluded in every manifest.
- Standing consent and window notice
- Standing consent is given once in the contract: test classes, synthetic seeds, the stop right and knowledge rules. Window notice precedes each run, by default ten working days for a managed provider, without saying which cases are tests.
- Sanitiser
- A deterministic export that passes an allow-list of fields and drops everything else, followed by a person's release. The technique identifier, portable detection logic and aggregate results may leave; the target mapping, the gap list and findings stay inside.
- Coordinated vulnerability disclosure
- A weakness in a supplier's product goes to the supplier and, failing an answer, to the CSIRT designated as coordinator under NIS 2 Article 12(1). The test stops at proof. The organisation also publishes a channel for reports about its own estate.
- One loop, local manifests
- For a group in several legal regimes, the stage records and handling classes are the same everywhere, and each entity has its own rules-of-engagement manifest reflecting local law, consent and transfer rules.
Interactive exercises
- Match: One loop, local manifests
- Scenario: A supervisor asks for the gap list
Done on the lesson page, with feedback at once.
Discussion
- A log that cannot cross a border is a coverage gap, recorded as one. How should a group explain such a gap to a board that expects one loop for the whole group?
- Detections received from peers are untrusted input and a draft. What would a loop need to do before a shared detection counts towards coverage?
- A supervisor has the authority to demand the gap list and the organisation has a duty to protect it. Why does the chapter settle on inspection rather than a copy?
Exercise · 35 minutes
A consent and notice protocol
Write a consent and notice protocol for one test of your own organisation, or of Nordvik Region, that crosses a managed provider's boundary. Following the nine steps of the practical guide, classify the crossing and refuse any test against the provider's platform. Record the standing consent the contract annex must contain. If a cloud provider is involved, note which published rules you would read and that you record the page and the date. Set the window notice and its contents. State what the provider may and may not know during the run. Name the stop holders on both sides with deputies and the instrument each uses. Name the stop condition for a vulnerability found in the provider's own service. State the data rule and what the provider receives for each seed.
Deliverable: A one-page protocol from which a reader can name the provider, the clause, the date of notice, the stop holders on both sides and the data rule.
Self-test
Which of the three ways of crossing a provider boundary is excluded in every manifest?
- Seeding an incident into the provider's queue under a contract annex.
- Probing the provider's own platform, its other customers' tenants or its staff.
- Running a scenario on one's own estate to watch the provider's out-of-hours chain.
- Asking the provider to return outcomes for tagged events within five working days.
What may leave the organisation in sector sharing, under the chapter's rule?
- The technique identifier, portable detection logic and aggregate results, after a person's release.
- The names of the systems each scenario ran against, marked TLP:AMBER.
- The list of techniques that are untested or failed, for peers to compare.
- Evidence manifests and telemetry samples that let the receiving peers verify that the detection really works.
During an authorised test, a weakness is revealed in a supplier's product. What does the chapter prescribe first?
- The tester continues to establish how far the weakness reaches.
- The finding is published at once with full detail, so that the supplier's other customers can protect themselves.
- The test stops at proof and the weakness is reported to the supplier, with the CSIRT coordinator as fallback.
- The finding is closed as out of scope and removed from the register.
Module 6 · Part VI · People and Career
Part VI turns to the people of the loop. Participants examine what red and blue are rewarded for and design a shared target and a blameless purple session, protect staff who are exposed to exercises, build a competence framework and an apprenticeship route that survives the automation of routine emulation, and keep a horizon register on whether red and blue will be separate teams in 2036, with every trend labelled observable, expected or speculation.
Chapter 23 · 75 minutes
Red and Blue as People: Trust, Incentives, Safety and Wellbeing
The lesson treats red and blue as people who are rewarded in opposite directions. Participants design a shared target that keeps accountability separate, write ground rules for a blameless purple session, apply three rules for those who are tested, manage the four sources of load, and govern exercises that expose staff outside the teams. The chapter states that its evidence is thin and adjacent and claims no more for it.
Learning objectives
- Explain why a scoreboard of red wins when blue loses makes honest reading costly, and what the shared target replaces it with.
- Draft a purple session charter with rules on recording, readers, pauses and the use of results.
- Classify the four sources of load on a team and propose a spacing rule for tested activities.
- Review a plan for an exercise that involves staff against the four points of governance.
Key concepts
- Shared target
- Red and blue are measured together by the share of priority techniques verified with evidence of a stated age, the time from finding to verified control and the held-out catch, while accountability for findings, changes and closure stays separate.
- Blameless, not accountless
- A session identifies contributing causes without indicting an individual, records roles and systems and not names, and lets no result enter an appraisal. The class owner, the person who granted a level and the closure owner still answer for design and decisions.
- Withhold the instance, not the class
- The charter and works agreement state that seeded incidents, injected records and tagged scenarios exist and in what classes. Only which case, when and with what content is withheld, and a missed seed counts against the system, not a person.
- Four sources of load
- The exercise window, the stop path that requires a reachable holder, the findings that become work, and the reviewing of machine output. Only the last is new, and tested activities are spaced so that a team is not always being tested.
- Governance for staff exercises
- Representatives are informed first; consent is not the legal basis; no result has individual consequences because the result belongs to the control; and staff are debriefed on the same day with a way to talk.
Interactive exercises
- Put in order: Ground rules of a blameless session, in sequence
- Scenario: A request for a name
Done on the lesson page, with feedback at once.
Discussion
- The joint scorecard changes what a good quarter means without changing who closes a finding. Why does the chapter insist on both halves of that sentence?
- Large field studies of phishing simulations found limited value in the training. What does that imply for what an exercise involving staff should be designed to test?
- Nordvik Region's small team is both examiner and examined. Which of the session rules matter most when the same six people will meet again next quarter?
Exercise · 30 minutes
A session charter and a staff notice
Write two short artefacts for your own organisation or for Nordvik Region. First, a purple session charter following the practical guide: purpose, who attends by role, what is recorded and what is not, who may read the record, how long it is kept, that the session opens with what held, that the record names roles and systems and not people, that any participant may pause without giving a reason, and that no result enters an appraisal. Second, an exercise-information notice for a class S4 exercise that involves staff: the kind of exercise, the unit, the window, the working days of notice to the representatives, what staff are told in advance and on the day, and that the result belongs to the control and has no consequence for any person.
Deliverable: A one-page session charter and a one-page staff notice, both dated before the session or window they govern.
Self-test
What does the chapter keep separate, and what does it share, between red and blue?
- It shares accountability for closure and keeps the targets of each team separate.
- It keeps accountability separate where independence needs it and shares the target for learning.
- It shares budgets and reporting lines in one team and keeps the scenario library and the register separate.
- It keeps the purple session separate and shares the individual appraisals.
How does a blameless session record an alert that was closed wrongly out of hours?
- By the name of the analyst who was on shift, in a version of the record visible to the CISO only.
- By the name of the shift, so that the team concerned can be retrained.
- By omitting the event from the record to protect those involved.
- By role and system, for example that a triage class closed the alert, not by person.
On what does the chapter rest the protection of staff exposed to an exercise such as a deepfake test of payment approvals?
- On governance: representatives informed first, no individual consequences and a debrief on the day.
- On the written consent of each participant, collected by the line manager before the exercise window opens.
- On keeping the exercise secret from the representatives until it is over.
- On limiting the exercise to volunteers from the finance unit.
Chapter 24 · 90 minutes
Competences, Training and Keeping the Red Craft
The lesson defines the red craft and the competences of the loop in six areas, from scenario design to evidence, each evidenced in an assisted and an unassisted form. Participants review what studies in other fields measured about skill beside automation, design an apprenticeship route with attempt-first practice, manual-form runs and rotation, write contract terms that buy teaching with testing, and work through the debate on whether automated emulation erodes the craft.
Learning objectives
- Explain the difference between the red craft and routine, and where the framework gives the adversary's mind an address.
- Classify loop competences into the six areas and state why each is evidenced in two forms.
- Design an apprenticeship route with stages, gates, rotation and unassisted practice per usage mode.
- Review the debate on erosion of the craft and state the verdict and what an organisation controls.
Key concepts
- Red craft
- In the book's own definition, the capacity to start from an adversary's objective and constraints, choose a path and adapt when a step fails. Routine is running a known technique in a known form and writing the record.
- Six competence areas
- SD Scenario design, EE Emulation engineering, DE Detection engineering, OA Observation and analysis, XC Exercise control and EV Evidence, with eighteen identifiers in Hansa Mutual's LOOP-CF. The adversary's mind is evidenced in SD-03 and EE-03.
- Assisted and unassisted form
- A competence is shown both with and without AI assistance, and the lower level counts; DE-03 and OA-03, the review of AI output, are assessed in the assisted form alone. The competence record is used for development, not for ranking, pay setting or discipline.
- Attempt first
- For the first six months an apprentice drafts every scenario, variant and detection test before opening any assistant's proposal. The chapter's reading of the studies is that the effect of AI on skill depends on whether it replaces the attempt.
- Craft register
- A team-level count of the people who have shown each loop competence unaided in the last twelve months. Hansa Mutual ties the runner's Level 3 for S1 scenarios to at least two operators who have shown EE-02 unaided.
Interactive exercises
- Sort: Competences of the loop by area
- Match: Unassisted practice per usage mode
Done on the lesson page, with feedback at once.
Discussion
- The case for calm says routine was never the craft. The chapter replies that the routine did teach. What exactly did it teach, and where can that now be learned?
- Hansa Mutual measured 26 per cent human-led hours against a protected 30 and reported it as a finding, not a failing. What does that way of reporting make possible?
- Nordvik Region buys the red competences and keeps the ability to read them. What is the minimum a buyer must be able to do itself?
Exercise · 30 minutes
An apprenticeship route that survives automation
Following the practical guide, design an apprenticeship route for your own team or for Hansa Mutual. List the competences the route must produce by identifier, for example EE-02, SD-01, SD-03, DE-01 and XC-02, and write for each where a junior practises it now that a pipeline runs the routine. Define the stages from shadowing to co-authoring to authoring a held-out scenario to leading a stop drill, with a gate for each. Add the attempt-first rule, manual-form practice on the range under supervision, and a rotation between detection engineering and the red team. Say who decides that a stage is complete and what is reported each quarter at team level. Participants whose organisation buys the loop write the tester annex instead: named testers, declared AI, an open-ended part, pairing days and a handover.
Deliverable: A learning path with competences, stages, gates and quarterly team-level counts, or a contract annex with the seven terms.
Self-test
Where does the competence framework give the adversary's mind an address?
- In OA-01 and OA-02, reading and classifying outcomes.
- In EV-01 and EV-02, writing and reproducing evidence.
- In SD-03 and EE-03, writing a novel scenario and adapting when a step fails.
- In XC-01 and XC-02, coordinating a purple session and holding and exercising the stop path.
What does the chapter's verdict say about whether automated emulation erodes the red craft?
- The evidence shows neither that it does nor that it does not; the response is the organisation's.
- The evidence from red teams shows erosion within two years of automation.
- The evidence shows that automation improves the craft of every operator.
- The question is settled against automation by the studies of colonoscopy, school mathematics and programming.
To what does Hansa Mutual tie the emulation runner's Level 3 for S1 scenarios in AIDR-0885?
- To the number of scenarios the pipeline runs each month.
- To the approval of the vendor that supplies the runner.
- To a pass rate of at least 90 per cent on the known-form scenarios of the library.
- To at least two operators who have shown EE-02 unaided within twelve months.
Chapter 25 · 75 minutes
The Future of the Loop: Will Red and Blue Be Separate Teams in 2036?
The lesson looks ten years ahead with the series' discipline. Participants sort trends into observable, expected and speculation, compare three futures for 2036, two teams with one loop, one engineering practice and a pipeline with a human verifier, by where independence lives, work through the debate on whether red and blue stay separate, examine the author's labelled projection and its five thresholds, and write a horizon register with triggers from the loop's own records.
Learning objectives
- Classify statements about the future of the loop as observable, expected or speculation, with the posture each justifies.
- Compare the three futures for 2036 by who authors scenarios, where independence lives and the main risk.
- Explain the verdict's separation of whether independence is needed from how it is provided.
- Draft a horizon register with labelled trends, triggers, postures and a review date.
Key concepts
- Horizon register
- A record of the organisation's view of the future of its loop in which every trend has a label, a stage, evidence, a trigger, a posture and an owner. Its triggers can be computed from the loop's own records.
- Commit, prepare, keep reversible
- The posture by label: observable trends justify commitment, expected trends justify preparation at low cost, and speculation justifies keeping a decision reversible, especially where it would remove a safeguard.
- Three futures for 2036
- Future A, two teams and one loop; Future B, one engineering practice; Future C, a pipeline with a human verifier. They are devices for thinking, carry no probability, and differ on where independence lives.
- Author's projection
- A section labelled as the author's judgement beyond the evidence: the red team survives as a name and role with changed content. It comes with five thresholds, each a trigger that would change the view.
Interactive exercises
- Sort: Observable, expected or speculation
- Match: Three futures for 2036
Done on the lesson page, with feedback at once.
Discussion
- The case for calm says the separation of red and blue was an accident of scarcity. What would have to be shown before you accepted a converged practice as verified?
- A loop that closes findings without a person at any step is labelled speculation and advised against. What would the first warning signs of drifting towards it look like in your own records?
- Nordvik Region accepts convergence inside the Region and keeps independence in the contract. What makes that defensible for a team of six?
Exercise · 25 minutes
A horizon register for the loop
Following the practical guide, write a horizon register for your own organisation's loop. List at most eight trends that bear on it, including at least one that would be bad news for your present plan. Label each observable, expected or speculation and write evidence a reader can look up; a trend without evidence is speculation. Name the stages each concerns by their exact names. Write a trigger for each trend with a threshold and a direction, drawn from the loop's own records where possible, such as the age of evidence for priority techniques, the held-out catch rate or the seeded-gap catch rate. Set the posture by label and say what keeping reversible means. Name the owner, the approver and the first review date.
Deliverable: A horizon register of up to eight trends, each with label, stages, evidence, trigger, posture and owner, and a review date.
Self-test
On what do the three futures for 2036 differ, according to the chapter?
- On whether independence between tester and tested is needed at all.
- On whether DORA and NIS 2 will still require financial and essential entities to test their ICT systems.
- On whether a named person signs the closure of a finding.
- On where independence lives: between teams, inside one practice, or with a human verifier.
Which posture does a trend labelled speculation justify?
- Commitment: a decision can be taken on it today.
- Keeping a decision reversible, especially where it would remove a safeguard.
- Preparation: decisions that cost little now and save a great deal if the trend arrives.
- No action, because speculation is excluded from the register.
How does the verdict label the two questions it separates about independence in 2036?
- That it is needed is speculation; how it is provided is observable.
- That it is needed is observable; how it is provided is expected.
- That it is needed is expected; how it is provided is speculation.
- Both are speculation, since ten years is too long a horizon.
Module 7 · Part VII · Experience and Conclusions
Part VII tests the design against experience. Participants recognise eight anti-patterns of the loop from public cases, composites and illustrative records, follow Nordvik Region and Hansa Mutual through a year with a failure and a recovery at each, turn a year of records into a two-year plan with at most six measures and three decisions, and weigh the author's verdict on the five-part thesis, the two operating rules and the twelve questions.
Chapter 26 · 90 minutes
Case Studies and Anti-Patterns
The lesson collects the failures of the purple loop. Participants read four public cases as far as their sources go, then eight anti-patterns, from purple theatre and the red team that grades its own homework to the findings register that leaked and the contract with no testing right, each with its mechanism, early signs and control. They run a loop review that finds which patterns their own loop is closest to.
Learning objectives
- Classify a described failure as one of the eight anti-patterns and name the stage it breaks.
- Explain the mechanism and the early signs of each anti-pattern and the control that changes the outcome.
- Compare the four kinds of case material and say what each can and cannot support.
- Review a loop against the seven steps of the loop review and draft an anti-pattern record for one pattern found.
Key concepts
- Purple theatre
- A session with no re-test: findings carry a change identifier and no re-test log, and the closure date equals the merge date. It breaks P5 Verify, and the control is the closure rule with implemented-unverified as a state.
- Grading its own homework
- A check in which tester and tested share a model, training data, prompts or a vendor, so that their errors correlate. The control is the independence statement, held-out scenarios and seeded gaps.
- Coverage by assertion and the green dashboard
- A coverage figure without a date, or a map coloured by the existence of a rule, and a dashboard that counts runs. The controls are a coverage record with last test, result and expiry, and cells that carry age.
- The emulation that became an incident
- An emulation whose blast radius includes what the estate does in response, for example a scope silent on calendars and dependencies. The controls are safety classes, scope as code and a stop path outside the runner.
- Anti-pattern record
- An AP record in the loop repository with the pattern, signs, evidence, owner, control, decision and a re-test date, so that a loop review can find each pattern and follow its remedy.
Interactive exercises
- Sort: Where each anti-pattern breaks the loop
- Scenario: A sample of closed findings
Done on the lesson page, with feedback at once.
Discussion
- The chapter says loops fail where a step was reasonable and a question was not asked. Take one anti-pattern: which reasonable step, and which unasked question?
- Nordvik Region's anti-patterns come from buying a loop and Hansa Mutual's from running one. Which family does your organisation belong to, and does it also carry the other?
- A draft pack at Nordvik Region said nine of nine priority techniques were mapped to a detection. What made that sentence easy to write, and what made it wrong?
Exercise · 35 minutes
A loop review for the eight patterns
Run the chapter's loop review on your own organisation, or on the records of Hansa Mutual and Nordvik Region as the book gives them. Draw a sample of findings closed in the last two quarters and ask for each re-test log and evidence manifest. For five claims on the coverage map, ask for the last test, the result and the expiry. List every agent and tool that plans, runs, drafts or checks, with vendor and model family. For each class of scenario in production, ask who holds the stop and how long a drill took. Inventory where findings live. Read the contracts for a testing right. For each of the eight patterns write found, not found or not testable, and write an anti-pattern record for one pattern found.
Deliverable: A review sheet with an entry per pattern and the evidence read, plus one anti-pattern record with owner, control and re-test date.
Self-test
Which early sign points to purple theatre?
- A closure date that equals the merge date, and findings with no re-test log.
- One vendor's model family named across the planner, the detection drafter and the critic.
- A tester's access that outlives the engagement.
- An agreement that says nothing about testing.
Which control answers the anti-pattern of overfitted detections?
- A register of record with handling classes and expiring access.
- A testing right, scope as code and the consent of third parties written into every supplier contract.
- Detection from a behaviour description, variants, a negative control and the held-out gap.
- Cells that carry age and turn amber at expiry.
What does the chapter conclude from its cases about AI and the loop?
- That AI made the loop unnecessary, since pipelines now find the failures.
- That AI made the loop unsafe, since agents leave the scope of a test.
- That public cases prove each anti-pattern occurs in most organisations.
- That no case shows AI made the loop unnecessary or unsafe in itself.
Chapter 27 · 75 minutes
A Year in the Loop: Nordvik Region and Hansa Mutual
The lesson follows four fictional people through the loop's year to 30 April 2028: Karin Lund and Elin Berglund at Nordvik Region, Lena Vogel and Mareike Hartmann at Hansa Mutual. Participants trace dated failures and recoveries through all five stages, read the two reports that answer AIDR-0800 and AIDR-0850, and turn the year into an annual loop calendar in which every expiry has a re-test dated before it.
Learning objectives
- Explain the failures of the year at each organisation and the stage at which the loop found each.
- Classify the steps of a recovery into containment, fix and independent re-test.
- Compare the two reports of April 2028 by what each claims and what each lists as not shown.
- Draft an annual loop calendar with fixed dates, stage events, delegation levels, stop holders and closure signers.
Key concepts
- Three-part recovery
- The shape every recovery of the year had: a local and immediate containment, such as a lowered level, a rotated token or a retired rule; a fix made by one person; and a re-test run by another.
- Found by the loop
- Every failure of the year, the hours variant, the restraint failure, the shared model family, the collector's log and the slow stop, was found by a scheduled exercise or check and not by an incident.
- The two reports
- AIDR-0841 reports Nordvik Region's exercise programme as a verified share with its list, reasons and age, not as closed findings. AIDR-0891 keeps Hansa Mutual's quarterly session for held-out and human-led scenarios and the findings the pipeline cannot raise.
- Annual loop calendar
- A calendar built backwards from dates that cannot move, such as evidence expiries, the validity of the independence statement, risk acceptances and contract end dates, with a re-test before each expiry, a drill before each window and time reserved for recovery.
Interactive exercises
- Put in order: Nordvik Region, January to April 2028
- Match: The people of the year
Done on the lesson page, with feedback at once.
Discussion
- A year without a failure would have tested nothing, the chapter says. How should a team present its failures of the year to a board so that they are read as evidence that the loop works?
- The chapter notes that the people carried more than the records show. Where in the year did the design rely on one person, and what made that person safer?
- Hansa Mutual's answer to its board was no, with reasons. Which of those reasons would stop being true first if the organisation stopped funding human-led work?
Exercise · 25 minutes
An annual loop calendar
Following the practical guide, write a calendar for the next twelve months of your own loop, or continue the calendar of Nordvik Region or Hansa Mutual. First fix the dates that cannot move: expiries of evidence, the validity of the independence statement, risk acceptances and contract end dates. Place the stage events behind them: a review of the priority list, emulation windows, a purple session after each window, change deadlines and a re-test dated before every expiry. Give each event its delegation level and a stop holder who sits outside the system being stopped, and schedule a stop drill before every emulation window. Reserve time for recovery. Name the person who signs each closure. Label every statement about the future as a plan.
Deliverable: A twelve-month calendar with fixed dates, dated stage events, levels, stop holders, closure signers and reserved recovery time.
Self-test
How were the failures of the year discovered at the two organisations?
- By incidents that reached production and were investigated afterwards.
- By the loop itself, through scheduled exercises, re-tests, reviews and drills.
- By the supervisor during a threat-led penetration test.
- By the vendors of the simulation tool and the managed service.
What does AIDR-0841 decide about how Nordvik Region reports to its audit committee?
- To report nine of nine priority techniques as covered by detections.
- To report the number of findings closed since the first exercise.
- To withhold the coverage page until every technique has been tested.
- To report a verified share with its list, reasons and age, not closed findings.
Which sequence describes the recovery pattern the chapter finds in every case?
- Report to the board, wait for the next quarter, then re-run the pipeline.
- Accept the risk, close the finding and record a lesson learned.
- Contain at once, fix by one person and re-test by another.
- Raise the delegation level, tune the detection and close at merge.
Chapter 28 · 90 minutes
Nordvik Region and Hansa Mutual: Conclusions for Two Organisations
The lesson draws the conclusions for the two organisations. Participants compare what the year verified and left open per stage, the conclusion that a bought loop is worth the terms on which its evidence can be inspected and that a run loop is as good as its independence and its stop, the six measures of each plan, the three decisions asked of each management body, and what each organisation should stop doing. They write a two-year loop plan.
Learning objectives
- Compare the conclusions and plans of Nordvik Region and Hansa Mutual by leverage, risk, independence and limit.
- Explain how each organisation answers the five parts of the thesis from its own records.
- Draft a two-year loop plan with at most six measures, each with an owner, a date and a verification.
- Design a gate for an increase in delegation and tie the plan's triggers to numbers in the loop's records.
Key concepts
- A bought loop
- Nordvik Region's conclusion in AIDR-0846: a bought loop is worth as much as the terms on which its evidence can be inspected. The Region owns the priority list, the register, closure and the stop, and asks the contract to carry what the team cannot.
- A run loop
- Hansa Mutual's conclusion in AIDR-0896: a run loop is as good as its independence and its stop. It keeps the red team's authorship of the held-out set and a person's reproduction of a sample of what the agents report.
- Six measures
- Each plan has at most six measures, normally one per stage and one for people, each with a verb, an owner by role, a date and a verification that produces evidence. They target what is open, not what a passed re-test has closed.
- Delegation gate
- Every increase in delegation is written as a gate: a threshold on the class's own evidence, a period, a named grantor, a stop rule and a first review 90 days after the increase. Hansa Mutual withdraws delegation below a seeded-gap catch of 0.8.
- Three decisions for the management body
- The management body receives one page and at most three decisions. Nordvik Region's are testing as a condition of every agreement, funding the people who hold the loop, and accepting the age of evidence.
Interactive exercises
- Sort: A bought loop and a run loop
- Scenario: A request to raise a level
Done on the lesson page, with feedback at once.
Discussion
- Nordvik Region is asked to accept a re-tender or a change of supplier if a provider refuses a testing-right clause. What does it cost a small public authority to hold that line, and what does it cost not to?
- Hansa Mutual's strongest instrument is described as the discipline not to widen what it cannot yet show it can supervise. How is that discipline made visible to a board that wants the pipeline to do more?
- Both plans end with a measure about people. Why does the chapter require these to be agreed with representatives before the first reading is taken?
Exercise · 40 minutes
A two-year loop plan as a decision record
Following the nine steps of the practical guide, write a two-year plan for your organisation's loop. List the year's decisions as done, in progress or reopened. For each of the five stages name the constraint that limits it most, such as one person, review capacity, a provider's silence or a shared model family. Write at most six measures, normally one per stage and one for people, each with a verb, an owner by role, a date and a verification that produces evidence. Order them by reversibility. Write any increase in delegation as a gate with a threshold, a period, a named grantor, a stop rule and a review after 90 days. Take triggers from your horizon register and tie each to a number in your records. Select at most three decisions for the management body.
Deliverable: A plan of at most six measures with owners, dates and verifications, the delegation gates, the triggers and three decisions for the management body.
Self-test
What is Nordvik Region's conclusion about its loop, as recorded in AIDR-0846?
- A bought loop cannot be trusted and should be brought in-house within two years.
- A bought loop is as good as the pipeline its provider runs.
- A bought loop is worth as much as the terms on which its evidence can be inspected.
- A bought loop needs no coordinator, because the contract closes the findings.
How does the plan treat an increase in a delegation level?
- As a gate: a threshold on recorded evidence, a named grantor, a stop rule and a review after 90 days.
- As an automatic step once the pipeline has run for two consecutive quarters without an incident or a failed drill.
- As a decision of the agent operator, recorded after the fact.
- As a vendor setting that changes with each new model version.
What does the chapter say both organisations should stop counting as progress?
- The verified share within expiry and its age distribution.
- The number of scenarios run or of findings closed.
- The held-out gap and the seeded-gap catch rate.
- Cycle time from finding to verified control.
Chapter 29 · 75 minutes
Conclusions: The Purple Loop with AI
The lesson gives the author's verdict on the book. Participants examine how each of the five parts of the thesis fared, why no part earned a plain held, how part three was restated, whether the refutation conditions occurred, the two operating rules and the six that serve them, the twelve questions with their status, the author's labelled projections and their triggers, and a first step for each role with a first-week plan.
Learning objectives
- Explain the verdict on each part of the thesis and what remains unproved for each.
- Compare what is supported by sources and design with what is only illustrated by the fictional cases.
- Classify the twelve questions as answered by design or open, giving the reason for the open ones.
- Draft a first-week plan for a loop with a concrete output for each of the five days.
Key concepts
- Verdict on the thesis
- Parts 1, 2, 4 and 5 held with qualifications and part 3 held in part. No part earned a plain held, the refutation conditions did not occur, and the fictional cases illustrate the design without testing it.
- Part 3 restated
- A loop closed by machines alone can show that the known still works and cannot by itself show that its coverage is real. Regression, replay and the re-run of a known scenario are mechanical and properly automated.
- Two operating rules
- The closure rule and the independence principle carry the book. Six further rules serve them: coverage with evidence age, expectation before result, scope as code with the stop outside, ceilings with the lowest governing, the register as tier 0, and a named person who decides.
- Twelve questions
- Six are answered by design, which is the book's own conclusion, and six are open because they turn on people over years, on what supervisors and national law decide, and on how long a stop path stays quick.
- First-week plan
- Five working days: Monday inventory, Tuesday target, Wednesday register, Thursday claim and rules, Friday owners. By Friday the organisation can say what it has tested, what is closed and who holds the stop.
Interactive exercises
- Match: A first step for Monday, by role
- Put in order: A first-week plan for a loop
Done on the lesson page, with feedback at once.
Discussion
- The author finds that no part of the thesis earned a plain held, and says the book searched only its own sources. How does that admission change how you would use the book's rules in your organisation?
- The first part of the thesis is partly a definition, and a definition is not refuted by a case. What evidence would persuade you that a separate purple team does worse, or better, than a loop held by roles?
- Of the six open questions, which matters most to your organisation in the next two years, and what would you measure to narrow it?
Exercise · 25 minutes
A first-week plan for a loop
Write the first-week plan of the chapter for your own organisation, with a named person and a concrete output for each day. Monday: list every tool and assistant in testing and detection with its usage mode, owner and stop holder, and mark the first with no stop path outside itself. Tuesday: build the priority list for one service, mark the hard-to-emulate entries and set a review date. Wednesday: count the findings by the five states and check one closed finding for re-test date, variants and signer. Thursday: write the coverage statement for three techniques in five parts and check that a signed manifest names a stop holder outside the system being stopped. Friday: name the closure owner and the coordinator, diarise the review of the independence statement and inform the employee representatives.
Deliverable: A five-day plan with a named person and an output per day, plus the participant's own first step for Monday.
Self-test
How did the third part of the thesis fare in the author's verdict?
- It held without qualification, on the strength of the Hansa Mutual case.
- It was refuted, because pipelines found most of the year's failures.
- It held with qualifications, like the other four parts.
- It held in part and was restated for what a machine-only loop can and cannot show.
Which two rules does the author say carry the book?
- The alert budget and the tune-or-retire rule.
- The closure rule and the independence principle.
- The 30 per cent human-led share and the attempt-first rule.
- The handling classes and the register of record.
What first step does the chapter give the closure owner for Monday?
- Ask of one finding the register calls closed its re-test date, its variants and its signer.
- Drill the stop on one real identity and measure the time to revoked credentials.
- Replace one activity count on the board page with the verified share within expiry, with its list and age.
- Read the contract for a right to test and a duty to declare AI.