Improve: From Finding to Change in Detections, Controls, Playbooks and Mandates

All courses / AI Purple / Module 2 / Chapter 10

Learning objectives

After the lesson the participant can:

• Classify a finding by the layer that failed and choose the route, giving a reason for each route rejected.
• Explain the ownership rules, including who must not own each kind of change.
• Review an AI-drafted change against the five review questions and the three risks specific to the loop.
• Draft a finding-to-change decision that a second person could reproduce.

Key concepts

Five routes

Detection, when activity was visible and no detection fired or it was routed wrongly; control, when a preventive control was absent or bypassable; playbook, when response was slow or wrong; mandate, when an agent, role or provider acted outside or short of its authority; recorded risk acceptance.

Change record

The CHG record of P4: the finding, the route, the decision with routes considered and urgency, the owner, the artefact, who drafted and who reviewed, cost, rollout, and the regression case handed to P5 Verify.

Alert budget and tune-or-retire rule

Each detection has a budget of alerts per week. A change is replayed over at least 30 days of telemetry, runs in shadow for 14 days, and is reviewed or rolled back if it exceeds its budget or twice its budget for two consecutive weeks.

Finding half-life

The number of days after which half of the findings opened in a quarter have a passing re-test; it counts only verified closure. Findings are sorted into urgent, standard and deferred, with default targets of 14, 30 and 90 days, by technique importance and evidence age, what the defence did and what is at stake.

Recorded risk acceptance

A decision by a role with authority over the residual risk, for a stated period, with a reason and a review date. The person who would do the work does not accept the risk, and the coverage record keeps showing the technique as failed or declared.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Lowering an agent's level takes effect at once, while raising it needs a named authority and a re-test. Why does the chapter build that asymmetry into review?
2. A loop that turns every finding into a rule ends with an alert queue nobody can read. Who should decide that an older rule is retired to make room for a new one?
3. Nordvik Region cannot edit its provider's closure behaviour. What can a buyer of the loop do at P4 Improve, and where does its leverage end?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. A finding shows that activity was detected but the response was too slow and too wide. Which route does the chapter's table indicate?

   1. Detection: rewrite the logic of the rule that fired.
   2. Playbook: change the playbook manifest, runbook or decision right.
   3. Control: add a preventive control in front of the service.
   4. Mandate: narrow the mandate of the tester who ran the scenario.
   Show the answer

   B. Playbook: change the playbook manifest, runbook or decision right.
   The playbook route applies when the activity was detected and the response was slow, wrong, too wide or too narrow. The artefact changed is the playbook manifest, the runbook or the decision right.

2. Who must not own the detection change that answers a scenario, according to the ownership table?

   1. The detection-engineering lead of the organisation.
   2. The purple-team coordinator who keeps the register.
   3. The tester who wrote the scenario that it answers.
   4. The platform team that runs the log pipeline.
   Show the answer

   C. The tester who wrote the scenario that it answers.
   The table lists the tester who wrote the scenario as the party who must not own the detection route. It carries the independence principle into ownership: the scenario's author does not tune the detection that answers it.

3. What is the status of a finding whose detection change has merged but whose regression case has not yet run?

   1. Implemented, unverified, with the coverage cell still shown as failed.
   2. Closed and verified, with the merge date recorded as the date of closure in the register.
   3. Risk accepted, until the next scheduled run of the pipeline.
   4. Deferred, with a 90-day target for the re-test.
   Show the answer

   A. Implemented, unverified, with the coverage cell still shown as failed.
   A merged change is not a closed finding. In the Hansa Mutual example the finding is implemented-unverified and the coverage cell stays failed until P5 Verify has run the case and its variants and attached the evidence.