Tools for Weighing Evidence: Threat Registers, Debate and Foresight

All courses / Tools / Module 7 / Chapter 23

Learning objectives

After the lesson the participant can:

Grade a claim about an AI threat on the five rungs of the evidence ladder and state what would move it.
Label an outlook as observable, expected or speculation and attach the matching posture.
Write a threat register entry with rung, label, exposure, influence, owner, controls and triggers in both directions.
Facilitate a debate in four steps that ends in a crux map and no-regret actions.

Key concepts

Evidence ladder: A five-rung grading of claims from AI Threat: fiction, argument, laboratory demonstration, incident in the wild, pattern at scale. A claim is graded at the highest rung its evidence reaches; the ladder does not measure risk.
Observable, expected, speculation: Three labels defined in AI Governance, Chapter 22. An observable trend is documented now and justifies commitment; an expected one justifies preparation; a speculative one is plausible but not supported by present evidence and justifies reversibility.
AI threat register: A version-controlled register, family THR, with one file per entry carrying scale, trope, rung, label, exposure, influence, owner, controls and triggers. It deliberately holds no probability, and review is triggered by change in both directions.
Constructive debate in four steps: A method: the case for concern, the case for calm, where the positions meet, and what can be done, before a verdict. Each case is stated at its strongest and with the same charity.
Crux map: The recorder's output of a structured discussion: each crux linked to a register entry, an owner and a trigger, with the value questions and the no-regret actions that hold whichever side is right.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Put in order

The rungs of the evidence ladder

Put the five rungs of the evidence ladder of AI Threat in order, from the lowest to the highest.

Interactive exercise 2 · Match

Artefacts for weighing evidence

Match each artefact of Chapter 23 with its purpose.

Teaching plan

15 min	Present Figure 23.1 and the evidence ladder with its three rules. Separate capability from propensity, and the rung from the label.
20 min	Read the register entry THR-NR-02 on AI-accelerated exploitation and the four conditions for an entry. Contrast the two Hansa Mutual entries: one owned and tested daily, one that asks the board to confirm a rule.
15 min	Teach the debate in four steps and the crux map from Nordvik Region's audit committee workshop, then the three labels and the horizon register.
30 min	Exercise: groups grade one claim, write a register entry and run a short debate in four steps on it.
10 min	Debrief: where exactly did groups disagree, in the rung, the label, the exposure or the controls?

Discussion

A behaviour shown in fifty laboratories stays a laboratory demonstration. Why does repetition in the laboratory not move a claim up the ladder?
The register holds no probability because, for contested tail risks, a single figure records a choice between camps of experts. What does a board lose and gain without the figure?
A register that never changes a label is not being tested. How would you build the habit of confirming or changing at least one label per review?

Written exercise

Exercise · 30 minutes

Grade a claim, enter it, debate it

Take one claim about an AI threat that has reached your organisation through a briefing, a news report or a supplier, stated without any operational detail. In groups of four, appoint a grader, a challenger, an owner and a recorder. State the claim precisely, grade it on the evidence ladder at the highest rung its evidence reaches, and label the outlook. Write a register entry: scale, rung, label, exposure given current controls, influence, owner, two controls that do not depend on the system being assessed, one trigger that would raise and one that would lower the entry, and a review date. Then run a ten-minute debate in four steps and record one crux, one value question and one no-regret action.

Deliverable: A graded claim with the challenger's dissent noted, one threat register entry with triggers in both directions, and a crux map of one crux, one value question and one no-regret action.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

Why does the AI threat register keep the rung and the label in separate fields?
Show the answer
C. Because the same rung can support different outlooks
The grade and the label are kept in separate fields because the same rung can support different outlooks: a laboratory demonstration that is cheap to exploit may be expected, while another stays speculation.
Which posture does a trend labelled speculation justify?
Show the answer
D. Reversibility
The postures are the same in every volume: observable justifies commitment, expected justifies preparation and speculation justifies reversibility. The Summary calls reversibility the posture for what cannot be evidenced.
Which of these is one of the four conditions for an entry to belong in the AI threat register?
Show the answer
A. A role can be named to own it
The four conditions are a plausible path to harm, a material change that AI makes to the harm, a claim precise enough to be graded, and a role that can be named to own it. No probability is held.