The Verification Loop: Purple, Chaos and Coverage Tools

All courses / Tools / Module 4 / Chapter 14

Learning objectives

After the lesson the participant can:

• Describe the five stages of the purple loop and the record each produces.
• Compute the status of a coverage record from its fields with an ordered rule.
• Turn a closed finding into a regression case with variants, a negative control and a cadence.
• Formulate a steady-state hypothesis for a security chaos experiment with rollback, blast radius and stop path.

Key concepts

Purple loop

A method of AI Purple with five stages, P1 Target, P2 Emulate, P3 Observe, P4 Improve and P5 Verify, in which every finding ends as a verified control, detection and re-test with evidence, and closed findings return as regression cases.

Coverage record

An artefact, family COV, stating per technique and environment what was tested, when, how, with what outcome and when the evidence expires. It is derived, and its status is computed from its fields and never edited by hand.

Regression case

A closed finding kept as a repeatable test, family REG, with the base scenario, variants, a held-out variant, a negative control and a cadence. Re-testing only the exact scenario protects the path and not the technique.

Independence between tester and tested

A principle of AI Purple with five kinds of shared dependency that can hide failure: model, data, authorship, vendor and time. Each has its own control, and the kinds are not interchangeable.

Security chaos experiment manifest

A catalogue entry, family SCE, that injects a condition, not an attacker technique, to verify one control property, with rollback, blast radius and failure routing. On failure a finding goes to the purple backlog.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Nordvik Region's coverage page reports nine techniques as five tested, two asserted by the provider and two not tested. Why is this page more useful than a fully green map?
2. A re-test run straight after the fix shares a dependency with the fix in time. Which of your current re-tests would fail the independence table, and on which kind?
3. The chapter says the chaos platform that injects the conditions is itself a tier-0 asset that is tested. What follows for who may operate it and how its credentials are issued?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. What is the distinguishing feature of the purple session method of AI Purple?

   1. The blue team is told the full plan a week before the run
   2. The session is recorded on video so that individual performance can be reviewed
   3. The red team grades the result alone and reports to the board
   4. The expectation is written, hashed and sealed before the run
   Show the answer

   D. The expectation is written, hashed and sealed before the run
   The expectation is written, hashed and sealed before the run, so that the session compares a result with a commitment, not with a memory. The charter excludes audio or video recording.

2. How does the coverage-status rule file treat a manual override of a computed status?

   1. No override is allowed; an extension is a recorded decision with a named person, a reason and a new date
   2. An override is allowed for the detection owner during a change freeze, provided the white-team lead is told within a working day
   3. An override is allowed once per quarter for each priority technique
   4. An override is allowed if the provider asserts in writing that the technique is covered
   Show the answer

   A. No override is allowed; an extension is a recorded decision with a named person, a reason and a new date
   In the rule file the first rule that matches wins and no override is allowed. If evidence must be extended, that is a recorded decision with a named person, a reason and a new date.

3. What does a security chaos experiment manifest inject, according to AI Red Team?

   1. A working exploit against a production control
   2. A condition, such as log forwarding from one host stopping
   3. A new detection rule drafted by an AI system
   4. A full adversary campaign planned by the red team
   Show the answer

   B. A condition, such as log forwarding from one host stopping
   The entry injects a condition, not an attacker technique. In the example, log forwarding from one host stops, and the hypothesis is that the security operations queue shows a silent-source alert within the stated time.