The Blame Instinct: When Nobody Designed Who Decides

All courses / Landscape / Module 2 / Chapter 13

Learning objectives

After the lesson the participant can:

Explain why, where software acts, the thing that acted last is often not the thing that decided.
Describe what the incident record names, and state the limits of the author's classification of its entries.
Read a decision-rights view from the seats and identify rows at T0.
Draft the four lines written before go-live: the tier, the holder, the stop path with its budget, and the record.

Key concepts

The blame instinct: The habit, when something has gone wrong, of looking for an agent to hold responsible and stopping when one is found. Here the agent found is the model, the vendor or the agent software, because those acted last.
Moral crumple zone: The term of AI Boss for a human attached to a system's output who can absorb the blame without having had the means to decide. It is why a name is necessary and not sufficient.
Six capabilities of a decider: AI Boss defines the named human decider by six capabilities: identifiable, able to decide otherwise, informed, given time, protected, and answerable. The seat tells you whom to ask; the capabilities tell you whether the answer will be worth anything.
Stop path with a budget: Who may stop the unit without asking anyone, within how many minutes, with the switch kept outside the thing it stops, and drilled. If a stop needs a conversation, the budget is a fiction.
Rule of thumb: the seat before the culprit: Look for the seat before you look for the culprit; if nobody was named to decide, that is the finding. The rule stops applying where a person acted with intent, the largest category of recorded harm.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Scenario

A system in the title

Read the situation and choose what the executive should do, following the chapter's rule of thumb.

Interactive exercise 2 · Put in order

Four lines before go-live

Put the four things written down before a unit goes live in the order the chapter says matters.

Teaching plan

15 min	Opening: the teacher presents the first draft of the post-mortem on the stop drill that took seventeen minutes against five and asks for its recommendation, then presents the second draft and its one-sentence finding.
20 min	Lecture on the evidence with Figures 13.1 and 13.2: named organisations against unnamed categories, the three readings the record cannot tell apart, the proximate-cause and intent fields, and Articles 3(3), 3(4), 25(1), 26(2) and 26(5) of the AI Act as the chapter reads them.
15 min	Guided reading of Figures 13.3 and 13.4 down the right-hand ends of the bars: seventeen rows and seventeen seats, thirteen rows and thirteen seats, and what each view looked like on 1 January.
30 min	Exercise: participants rewrite an incident finding and design the decision rights for one unit before go-live.
10 min	Closing: the two outcomes of a failed stop. One organisation finds no seat for the budget and creates one; the other finds the seat and gives it hands.

Discussion

The second draft finds nothing wrong with anyone and something missing from the design. Why is that draft harder to write, and what in your organisation's review practice favours the first?
A seat on the ambient AI does not make it safe; it makes it somebody's. What changes in practice once a row at Level 4 has a named holder, and what still does not?
The chapter says an organisation that starts from the record writes a name into a field, while one that starts from the tier and the stop path may discover the name was wrong. What does the order of the four steps protect against?

Written exercise

Exercise · 30 minutes

Four lines before go-live

Part one: take an incident report from your organisation with a system in its title, or the first draft of Hansa Mutual's post-mortem. Rewrite its principal finding after asking who had been named to decide that the unit may act at that level, who had been named to decide that it stop, and whether a record dated earlier says so. Part two: for one unit that acts at Level 3 Bounded or above, or the unit in your report, write the four lines in the chapter's order: the tier, from the reversibility and reach of its actions; the holder, by role, checked against the six capabilities; the stop path, with who may stop without asking, the budget in minutes and where the switch sits; and the dated record. Draw the unit's row as a V4 with its seat.

Deliverable: A rewritten finding that names the seat or its absence, and a dated decision record with tier, holder, stop path and budget, with a hand-drawn V4 row.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

What is the one-sentence finding of the second draft of the post-mortem on Hansa Mutual's stop drill?
Show the answer
B. Nobody has been named to decide the time to stop
The second draft keeps every fact of the first and reaches a different finding: the alert reached an analyst without the decision or the mandate, and the record named an owner but not who decides the stop inside the budget.
After Nordvik Region's first out-of-hours stop drill of the benefits-triage system took 26 minutes against a budget of 15, what was the remedy in the scenario?
Show the answer
C. The managed security service provider received a break-glass credential for one action; the seat stayed put
The right had been designed; the gap was the means to exercise it at night. A break-glass credential for one action was given, and the October re-drill came in at 11 minutes. The seat stayed at T2.
The author sorts the thirty most frequent alleged deployers in the AI Incident Database into named organisations and unnamed categories. What does he say the resulting figure shows?
Show the answer
D. How the record is labelled, not whom it blames; the split is partly an artefact of the labelling
An entry such as scammers gathers many actors under one name, where organisations are counted one by one. The sorting is the author's classification, and the figure shows how the record is labelled. No public dataset records whether a decision right was designed.