Emulate Continuously: Breach-and-Attack Simulation, Atomic Tests and Automated Red Teaming

All courses / AI Purple / Module 2 / Chapter 8

Learning objectives

After the lesson the participant can:

Compare breach-and-attack simulation, atomic tests and automated red teaming by what each is good for.
Design triggers and a cadence that keep evidence for priority techniques inside its expiry.
Explain the four overfitting failures and how a held-out set and the generalisation gap expose them.
Review the debate on the human-led session and state the verdict with its observable, expected and speculation labels.

Key concepts

Three kinds of trigger: Change triggers answer a detection that worked yesterday and not today; schedule triggers answer decay when nothing changed; intelligence triggers answer a world that moved. Hold-backs during a freeze or a major incident are logged with the evidence age.
What runs where: Every new scenario and variant runs first on the red-team range; scenarios above S1 run in a pre-production mirror; production carries S1 on the pipeline's authority and S2 only with white-team approval per run; shadow replay tests a detection change against stored telemetry.
Overfitting failures: Marker leakage, where a rule fires on the telemetry tag; signature fitting to the public form of a test; catalogue closure, where detections converge on the library; and dashboard comfort, where a high pass rate over a stable library is read as safety.
Held-out set and generalisation gap: A share of scenarios and variants hidden from detection engineers and never used for tuning. The generalisation gap is the difference between the pass rate on known forms and on held-out variants; above ten percentage points it is treated as a finding.
Protected share of human-led emulation: A share of red-team hours reserved for novel, human-led work, with a default of 30 per cent that is the book's own figure, and with novel-finding yield measured separately from the pipeline's regression results.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Sort

What runs where

Sort each statement into the environment of the chapter's table that it describes.

Interactive exercise 2 · Scenario

A request to see the held-out set

Apply the chapter's response to overfitting to a request at Hansa Mutual.

Teaching plan

15 min	The teacher presents the three kinds of tool, the environment table and the pipeline definition of HM-SEC-004, stressing the lines that say what the runner cannot do.
15 min	Participants compute the generalisation gap from Hansa Mutual's first quarter of 2028: 37 of 41 known forms against 10 of 14 held-out variants, and decide what follows.
25 min	Exercise: choose triggers, cadence, volume cap and hold-back rules for a pipeline or, for a buyer of the loop, a pinned atomic-test subset in a contract.
25 min	Structured debate in four steps: the case for concern, the case for calm, where the positions meet and what can be done, with two groups arguing each side at its strongest.
10 min	The teacher gives the chapter's verdict in three labels and asks each participant for one design choice they can influence.

Discussion

The case for calm says spending a red-team operator's scarce hours on re-running last year's techniques is a poor use of them. Where does that argument stop being true?
A pipeline that adds alerts consumes analyst attention. Who in your organisation should hold the right to cap its volume, and on what evidence?
If the model that proposes variants also drafts detections, the set is not held out from it. How would you show a board that your held-out set really is held out?

Written exercise

Exercise · 25 minutes

Triggers and cadence for continuous emulation

For ten priority techniques of your own organisation, or of Hansa Mutual as the chapter describes it, write a pipeline definition in outline. Give each technique its evidence expiry and set the schedule trigger to one third of it. List the detections, connectors and mandates that can affect each technique and attach a change trigger with a maximum delay. Define the intelligence trigger as a named intake with deadlines. Estimate the alerts per week the pipeline will generate and compare them with the alert budget of the affected detections. Add hold-back rules and a quarterly stop-path drill with a target time. Participants whose organisation buys the loop instead specify five atomic tests, by technique identifier, for a contracted catalogue run.

Deliverable: A one-page pipeline definition, or contracted test subset, with three trigger types, a volume cap, hold-back rules and a stop-drill target.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

In Hansa Mutual's first quarter of 2028, detections fired on 37 of 41 known-form scenarios and on 10 of 14 held-out variants. What does the chapter conclude?
Show the answer
D. The 19-point generalisation gap is above the threshold and is a finding on the detections.
Ninety per cent on known forms against 71 per cent on held-out variants gives a gap of 19 points. The chapter treats a gap above ten percentage points as a finding on the detections, a threshold later fixed in AIDR-0889.
Which overfitting failure occurs when a detection fires on the telemetry tag that gives ground truth?
Show the answer
B. Marker leakage.
Marker leakage is the case where the tag is visible to the detection logic, so a rule fires on the tag and not on the behaviour. The design answer is a tag that detections cannot see.
How does the chapter's verdict label the claim that continuous automated emulation replaces the human red team?
Show the answer
A. Speculation: no evidence shows it, and a protected share of human-led work remains.
Continuous regression is observable and AI-generated variants at scale are expected, but replacement of the human red team is labelled speculation. The book's conclusion is that the work changes and a protected share of novel, human-led emulation remains.