Triage Transformed: From Alert Queue to Verified Case

All courses / AI Blue / Module 2 / Chapter 7

Learning objectives

After the lesson the participant can:

Explain why closure is delegated by class and not by alert, and list the properties each class needs
Draft a closure policy for three alert classes with sampling, seeding and a numeric stop condition
Compare the measures of triage quality and explain why closure speed is the wrong one
Evaluate the case for concern and the case for calm on AI closure and state where they meet

Key concepts

Closure policy: The answer, given in advance, to the question for which classes of alert an AI system may close without a person reading, on what evidence and with what check afterwards. It is enforced by a policy point outside the model.
Escape rate by class: The share of closed alerts in a class that a later review shows should not have been closed. It is estimated from the sample, which limits how small a rate the default sampling can demonstrate.
Seeded incident: A known case, synthetic or replayed, injected into the stream so that review can be seen to catch it. The default is at least five per class per quarter.
Verified case: The case record, identified as CASE-*, with five properties: provenance of every conclusion, evidence by reference, the class and level of any closure, timestamps that serve the reporting clock, and reproducibility without the AI.
Mode 0 blind spot in managed triage: A provider's or product's AI closes alerts without a decision recorded by the customer. The answer is a contract that gives the class list, the evidence to close, sampling, seeding, notice of model change and a stop right.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Match

Five triage tasks and how each fails

Match each triage task in which AI takes part with the characteristic failure the chapter's table gives for it.

Interactive exercise 2 · Scenario

The sample finds a closed phishing report

Apply the chapter's idea of the verified case to the Nordvik Region example.

Teaching plan

15 min	Teacher presents the alert as a claim, the three triage decisions of validity, consequence and routing, and the table of five tasks with their characteristic failures.
15 min	Case reading of CASE-NR-0231: participants mark each step where the record says no delegation was recorded, and explain what reopened the case.
35 min	Exercise: groups write a closure policy for three alert classes, modelled on CLOSE-HM-SOC.
25 min	Structured debate in four steps: the case for concern, the case for calm, where the positions meet and what can be done, followed by the chapter's verdict.
15 min	Wrap-up on measures of triage quality and on what the analyst does when the repetitive classes are closed by class.

Discussion

A reopen rate of zero over many months may mean the review is not real. How would you tell a safe class from an unexamined one?
The chapter proposes keeping a share of triage unassisted, one case in ten at Nordvik Region and one in five at Hansa Mutual. What would decide the right share in your team?
Who should answer when a class closes an intrusion, and how does the chapter divide that between the class owner, the approving authority and the analyst?

Written exercise

Exercise · 35 minutes

A closure policy for three alert classes

Follow the practical guide. From 90 days of closed alerts in your own team, or from the candidates the chapter names (registered internal vulnerability scanner traffic, duplicates of an alert already in an open case), choose three classes that are high in volume, narrow in definition and low in consequence when wrong. For each, write the definition and the evidence to close in structured fields, the named owner, the starting level of Level 1 Assisted for four weeks, a sampling rate of 5 per cent with a floor of 20 a month, the seeding plan and a stop condition written as numbers. Add the list of alerts never closed by an agent and a review date.

Deliverable: A closure policy file with three classes, the never-closed list, owners, levels, sampling, seeding, numeric stop conditions and a review date.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

Of the five triage tasks in which AI takes part, which one takes the decision away from a person?
Show the answer
C. Closure
Enrichment, correlation, deduplication and summarisation leave the decision with a person. Closure is itself a decision, which is why it is delegated by class under a policy.
In the practical guide, what happens to a class after one confirmed escape of high severity or one missed seeded incident?
Show the answer
B. It returns to Level 1 Assisted at once, so nothing is closed unread
The stop condition is written as numbers and stops closure: the system recommends and a person closes until the class is reinstated.
Which of these may never be closed by an AI system under the two exclusions that belong in every closure policy?
Show the answer
A. An alert about a tier-0 asset, an executive account or a named person's access to work
No class is closed by an AI system where the alert concerns a tier-0 asset, a privileged or executive account or a named person's access to work, and Level 4 is not used for any class that can hide an intrusion in progress.