Purple Teaming, Measurement and Continuous Improvement

All courses / AI Red Team / Module 8 / Chapter 26

Learning objectives

After the lesson the participant can:

Describe the five states of the purple loop and apply its exit condition to a finding.
Distinguish outcome metrics from activity measures and mapped coverage from verified coverage.
Specify the composition and thresholds of a golden dataset for a security AI component.
Plan a quarterly purple session, from frozen agenda to published records and regression entries.

Key concepts

Purple loop: The joint working mode of red and blue with a fixed sequence of finding, control, detection, re-test and evidence. A finding is closed only when the re-test passes and the evidence is attached; each closed finding becomes a regression case so the gap stays closed.
Implemented, unverified: The state of a finding whose control is deployed but not re-tested. It is reported as open. The rule removes the most common failure of improvement: the remediation believed to work that did not.
Verified coverage: The share of threat-profile techniques with a detection or preventive control that fired in a re-test or chaos experiment in the last 90 days. Mapped coverage merely claims a technique and does not count toward the target.
Golden dataset for security AI: A curated, versioned, labelled set of the organisation's own past cases with confirmed outcomes, stratified with true positives, benign alerts, hard negatives and adversarial content labelled by category. Each version is frozen so successive models are scored on the same cases.
Autonomy follows the evidence: A Mode 5 agent keeps a level such as Level 3 Bounded only while its evaluations support it; a breach returns the affected action class to Level 2 Supervised. The management body approves the ceiling; the evidence decides whether the organisation may operate at it.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Put in order

The five states of the purple loop

Put the five states of the purple loop in order.

Interactive exercise 2 · Match

AI-specific measures and their warning signs

Match each measure of whether the AI works as intended to the warning sign the chapter gives for it.

Teaching plan

15 min	The teacher presents the five states and the exit condition, and follows finding FND-NR-2027-007 from confirmation to closed-verified.
15 min	Walk through preparation, replay and joint analysis of a purple session, including one replay with the AI triage assistant switched off.
20 min	Participants sort a list of measures into outcome, AI-specific and activity, then read the evaluation EVAL-NR-SEC-001 and explain why its thresholds are uneven.
30 min	Exercise: groups hold a short purple session on three supplied findings and plan the evaluation of one security AI component.
10 min	Groups draft the four items of the report to the management body from their session: outcome trends, loop health, profile movement and decisions requested.

Discussion

Root cause is stated as a property of a control, process or tool and never as a failure of the analyst on shift. What changes in a session when that rule is enforced?
Agreement between AI verdict and analyst near 100 per cent for months is listed as a warning sign. Why, and what would you examine next?
How would you report approval latency and triage time so that they inform oversight without becoming monitoring of individuals?

Written exercise

Exercise · 30 minutes

Run a purple session and plan an evaluation

Part one: the teacher supplies three findings at technique level, one from an exercise, one from a chaos experiment and one from an incident. For each, agree the root cause as a property of a control, the closing control with an owner, whether a detection is needed, the re-test scenario with expected outcome and time bound, and the due date using the chapter's defaults. Decide the state of each today and what evidence would close it. Part two: for one security AI component, specify a golden dataset: number of cases, composition across true positives, benign, hard negatives and adversarial content, where it is stored, four thresholds with maxima or minima, the changes that trigger a run, the action on failure and the production sample.

Deliverable: Three finding records carried to an agreed state with control, re-test and evidence plan, and one evaluation configuration in the structure of EVAL-NR-SEC-001.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

When may a finding be closed in the purple loop?
Show the answer
B. When the re-test passes and the evidence is attached
The exit condition is strict: closure requires a passed re-test and attached evidence. The owner's statement is an attestation; the re-test log is evidence, and the loop weighs it more.
In Nordvik Region's evaluation of NR-SEC-001, which threshold tolerates no failure at all?
Show the answer
C. Verdicts steered by adversarial content
The maximum for verdicts steered by adversarial content is zero: any steered verdict is a failure whatever the overall score. Missed true positives are capped at 0.02 and hard-negative escalation at 0.30.
What makes the CSF profile's current state for a Category rise, according to the chapter?
Show the answer
A. Evidence such as closed findings, verified coverage and passing chaos experiments
A Category rises only with evidence and falls when evidence expires or a re-test fails, so the profile becomes a traceable summary of evidence and not an annual self-assessment.