Sustain: Problem Management, Post-Incident Learning and Knowledge

All courses / AI Operations / Module 2 / Chapter 12

Learning objectives

After the lesson the participant can:

Distinguish an incident record, a problem record and a post-incident review by the question each answers
Explain the risks of an AI-drafted timeline and the controls that keep the analysis in human hands
Apply the action closure rule and report the state and age of each action
Review a piece of operational knowledge for owner, last verification and expiry

Key concepts

Four outcomes of a problem: A problem becomes a change by a recorded decision: the condition is removed by a change, contained by a compensating control, transferred by a contract, or accepted by a named person for a stated period. A problem that is none of these and ages in a list is a debt without an owner.
Contributing conditions: The form of finding in a blameless, evidence-based review: several conditions that together allowed the incident, each with evidence, in place of a single guilty cause or person. Where regulation asks for a root cause, one is selected for reporting while the conditions are recorded.
Timeline and analysis as separate artefacts: The timeline may be drafted by an AI system at Level 1 Assisted, each entry with a time, a source link and a kind: observed, reported or inferred, and verified by a person. The analysis, with conditions, alternatives and actions, is written by people.
Closure rule: An action is closed only when its re-test has passed, the evidence is attached, a named person has signed and the closure carries an age. Implemented but unverified is reported as open, and an accepted risk is reported apart from closure.
Knowledge hygiene: Knowledge has a named owning role, a link to a service record, a last-verified date with a method, and an expiry. Retrieval by assistants makes stale knowledge fast, so freshness travels with the text, and unowned articles are archived after notice.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Sort

What AI may do in a review

Sort each task by whether the chapter's table allows an AI system to do it in a post-incident review.

Interactive exercise 2 · Scenario

The draft is ready before the meeting

Apply the chapter's controls for a review that uses a machine-drafted timeline.

Teaching plan

15 min	The teacher sets out the three records and their clocks, the triggers for a problem record including recurrence within 90 days, and the four outcomes.
20 min	Presentation of the blameless review and the risks of an AI-drafted timeline: the plausible but wrong narrative, hindsight written into the draft, anchoring, and an agent drafting the account of its own actions.
10 min	Participants read the table of what AI may and may not do in a review and explain the reason given for two rows.
35 min	Exercise: participants apply the closure rule to an action register.
10 min	Debrief on Nordvik Region's two failover actions, shown as done in the tracker and reported as open with an age of 336 days under the rule.

Discussion

Why does the chapter have each participant write a short account of their own before the group sees the drafted timeline, and what is lost if the order is reversed?
Hansa Mutual closes a review action only when a game-day re-test passes. What would your action register look like next quarter under the same rule?
A gap in a timeline is called a finding. What happens to that finding when a drafting system fills the gap with a probable event?

Written exercise

Exercise · 35 minutes

Applying the closure rule to an action register

Take the actions from one post-incident review of your organisation, or the two corrective actions after Nordvik Region's failed failover test RST-NR-2027-002. For each action record the condition it addresses, the owner, the change made, and the re-test that would show the condition is gone: a test, a game day or an authorised, scoped experiment that reproduces the mechanism. Assign one of the five states, and state how each is reported: open, closed with age, or accepted risk apart from closure. Add the date of the recurrence check 90 days after closure. Finish with one sentence for management on what can and cannot be claimed.

Deliverable: An action register with condition, owner, re-test, state, reported status with age, recurrence-check date and a one-sentence statement for management.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

How is an action reported whose change is in production but whose re-test has not yet run?
Show the answer
D. As open, because implemented says nothing about the condition
Implemented, unverified is one of the five states and is reported as open. Implemented is a statement about the change; only a passed re-test with evidence and a signature says something about the condition.
Which task does the chapter allow an AI system to perform in a post-incident review?
Show the answer
B. Draft the timeline from records, with evidence links and kinds, for a person to verify
A timeline may be drafted at Level 1 Assisted with evidence links and kinds. AI may not fill gaps, assign the cause or close an action: a gap is a finding, the review is a diagnosis people own, and a named person signs closure.
Why does Hansa Mutual keep the action log of its IT-operations agent in a store the agent cannot edit?
Show the answer
A. Because the agent takes part in many incidents, so the account of its actions must be independent of it
When an agent was part of the incident and the same platform drafts the account, the account is not independent. The group keeps the log out of the agent's reach and has a person verify every entry about it.