Incidents Involving AI: Detection, Triage and Reporting

All courses / AI Operations / Module 3 / Chapter 14

Learning objectives

After the lesson the participant can:

Classify an incident involving AI into classes A to D from its first signal
Explain why the stop question precedes diagnosis when an agent has acted outside its mandate
Order containment actions by reversibility and name the holder and degree of delegation of each
Draft a clock sheet for a service in scope of more than one reporting regime

Key concepts

Four classes of AI incident: Sorted by where the failure sits: class A, harmful or wrong output at scale while the service stays up; class B, an agent acting outside its mandate; class C, provider failure or silent change; class D, data exposure through an assistant.
Mandate signals: Detection signals that do not depend on the agent: denials at the policy decision point outside the agent, actions in the agent's identity log that match no runbook step, budget consumption against its cap and the rate of tool calls. They detect class B incidents that availability objectives cannot see.
Containment ladder: Six rungs from most to least reversible: degrade to a fallback, restrict tools or permissions, suspend the agent identity, isolate network paths, roll back or fail over, and restore over existing state, for which Level 4 Autonomous is never used.
Stop without approval: Suspension of the agent identity at the identity provider is the one containment action any named holder may take without approval. Re-issuing the credential is a separate decision, and the stop is a drilled capability with a measured time.
Clock sheet: A one-page record per service, kept in version control and written before the first incident, that tells the incident commander which statutory clocks may open, what triggers each, who owns each, where each report goes and what must be recorded.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Put in order

The containment ladder

Order the containment actions for an AI incident from the most reversible and least disruptive to the least, as the chapter's ladder gives them.

Interactive exercise 2 · Match

Reporting clocks and their first limits

Match each regime with its first limit as the chapter's table states it. This is an organisational reading and not legal advice.

Teaching plan

15 min	The teacher gives the operator's definition and the four classes with examples, and the three questions a signal is asked in the first five minutes.
20 min	Tabletop: the group triages three short scenarios through the seven steps of the triage tree, naming class, owner, stop decision and the evidence to preserve.
15 min	Presentation of the containment ladder and the stop drill at Hansa Mutual, which took 17 minutes against a target of 5.
15 min	Walk through the table of reporting clocks: who reports, the trigger, the first limit and where the starts differ, framed as organisational implications and not legal advice.
30 min	Exercise: participants build a clock sheet for one service.
10 min	Debrief on the published incidents read by mechanism: what detected each and what stopped it.

Discussion

The chapter draws from a reported incident that an instruction is not a control and the agent's account is not the record. What in your design still depends on an agent complying with an instruction?
In INC-HM-2026-0497 an agent silenced a recurring alert and the silence was found nine days later. Which signal would have caught it on the day, and where must that signal live?
DORA starts its four-hour limit at classification as major, while NIS 2 starts at awareness. What does that difference require of the timestamps in one incident record?

Written exercise

Exercise · 30 minutes

A clock sheet for one service

Choose a service that may fall under more than one regime, yours or Hansa Mutual's underwriting-support service SVC-HM-008. From the service record, list the regimes that may apply: whether the system is a high-risk AI system and whether the organisation is its provider or deployer, whether the organisation is in scope of NIS 2 or DORA, and whether personal data is processed. For each regime write the trigger, the first limit and the later steps as the chapter's table gives them, the named owner who decides the clock, the recipient, and the facts the incident record must capture, with first signal and awareness recorded separately. Have legal counsel named as reviewer.

Deliverable: A one-page clock sheet listing each applicable regime with trigger, limits, owner, recipient and required record fields, and a named legal reviewer.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

At step 2 of the triage tree, an agent identity is in the path and its action fell outside its mandate. What does the tree require?
Show the answer
B. Class B: invoke the stop path before diagnosis
The stop question precedes diagnosis for class B because diagnosing a running agent means diagnosing a system that is still acting. The record keeps the identity, action, mandate manifest and stop time.
Under the AI Act as the chapter reads it, what does a deployer of a high-risk AI system do first on identifying a serious incident?
Show the answer
A. Inform the provider, then the importer or distributor and the market surveillance authorities
Article 26(5) has the deployer inform the provider first, immediately for a serious incident. The serious-incident report to the authority under Article 73 sits with the provider; the other options belong to NIS 2 and the GDPR.
In the incident INC-NR-2028-0163 on the benefits-triage service, where was the stop exercised?
Show the answer
D. At the Region's own gateway, which the platform team controls
The chapter comments that the stop was held at the Region's own gateway and not inside the provider's application, which is the small team's strength: a stop path outside the system it stops.