All courses / AI Purple / Module 2 / Chapter 11
Learning objectives
After the lesson the participant can:
- Explain the four clauses of the closure rule and the failure each one answers.
- Design a re-test with a pass criterion, an independent runner, variants and a negative control.
- Draft a regression case and the fields of an evidence manifest for one finding.
- Review a verification report for its denominator, its reasons for unverified techniques and its evidence ages.
Key concepts
- Four clauses of closure
- A finding is closed only when the re-test and its variant pass; the evidence is attached; a named person signs; and closure has an age, so that when the condition lapses the finding is unverified again.
- Variants
- Path variants change how the technique is carried out, context variants change where and when, and rate and shape variants change volume and timing. A held-out variant is written by someone who did not write the fix and is not shown to the fix author.
- Negative control
- A benign look-alike that must not alert, included in each case because a detection can pass its re-test by alerting on everything.
- Regression case as ratchet
- A re-test that has been kept (REG): the exact scenario, its variants and the negative control, run on change, on schedule and on intelligence. A case once added is retired only by a recorded decision when its control is retired.
- Evidence manifest
- The unit of proof (EVD): generated by the run and not written, content-addressed, versioned with its dependencies, signed by a nameable person and minimised. It expires by a clock and by a condition, such as a change to a dependency.
Interactive exercises
Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.
Interactive exercise 1 · Put in order
From finding to closure, and after
Put the steps of the closure path in the order the chapter gives them.
Interactive exercise 2 · Sort
Three kinds of variant
Each line changes one thing about a scenario and keeps the technique. Sort it into the kind of variant the chapter defines.
Teaching plan
| 15 min | The teacher presents the four clauses and the four register states: verified; implemented, unverified; failed; not tested. |
| 15 min | Case reading of REG-NR-0004 and EVD-NR-2028-0003: the exact re-test passed on 2027-11-17, an out-of-hours variant failed on 2028-01-19 and the full case passed on 2028-02-09. |
| 35 min | Exercise: turn one closed finding into a regression case with variants, a held-out variant, a negative control, triggers and an expiry. |
| 15 min | Participants rewrite a one-line board statement of 74 per cent verified into a report with denominator, reasons and evidence age, using Hansa Mutual's 25 of 34. |
| 10 min | Plenary on delegation at P5: the runner at Level 3 Bounded, never the agent that drafted the fix, and closure signed by a person. |
Discussion
- Under a rule that closed findings on a passing exact re-test, Nordvik Region's finding would have been closed in November 2027. What would the Region have reported to its audit committee, and what would it have missed?
- A green run is itself an automation output that people learn to trust. What keeps a nightly regression suite from being read by nobody?
- Evidence that never expires is not credible, the chapter says. How would you explain to a system owner that a finding closed last spring is now unverified again?
Written exercise
Exercise · 35 minutes
From closed finding to regression case
Choose one finding that your register, or the Nordvik Region or Hansa Mutual material, calls closed. Following the practical guide, restate the hypothesis: technique, environment and pass criterion with its time bound and layer. Record the exact scenario as the base case. Describe at least three variants for a priority technique, one of path, one of context and one of rate or shape, each at the level of technique and telemetry and never of procedure. Name who will author the held-out variant, add one negative control, and confirm for each scenario its safety class, tag, clean-up step and stop-path holders. Set the triggers, the cadence and the evidence expiry, and list the versions the evidence manifest must record.
Deliverable: A regression case record with base case, variants, held-out variant, negative control, triggers, expiry and a named closure signer.
Self-test
Choose an answer to see whether it is right, or open the answer below each question.
Why is re-running the exact scenario necessary but insufficient for closure?
Show the answer
D. Because a detection written from a finding tends to fit that finding and can miss a variant.
The exact re-test shows the specific path is now seen. A detection shaped by a finding passes the scenario that shaped it, so variants, a held-out variant and a negative control are needed to give evidence about the technique.Who signs the closure of a finding at P5 Verify?
Show the answer
B. A named person in a named role; never an agent, and never the author of the fix alone.
Closure is a decision about the organisation's risk and is not delegated. A runner may execute the re-test and record the result, but a named role signs, never an agent and never the author of the fix alone.Which two parts make up the expiry of evidence in the chapter?
Show the answer
C. A clock since the last full pass and a condition that ends it early on a dependency change.
An expiry has a clock, a number of days after the last full pass, and a condition that ends the evidence earlier when something it depends on changes. When either triggers, the status changes by rule from verified to unverified.