The Voices of Alarm and Calm: Who Is Warning, About What, and Why Now

All courses / AI Threat / Module 1 / Chapter 3

Learning objectives

After the lesson the participant can:

Describe the five camps of the AI risk debate and state the strongest point of each.
Explain what the researcher survey and the forecasting tournament show about expert opinion, and what they do not show.
Identify the measures the camps share and the measures on which they diverge.
Draft a claim card for an AI risk claim using the ten-question check.

Key concepts

The five camps: Present-harm critics and sceptics see harm already here; optimists and the race case expect harm or gain within a decade; extreme concern holds catastrophe possible with contested timing. They differ on severity and timing more than on measures.
Four sources of catastrophic risk: The division by Hendrycks, Mazeika and Woodside: malicious use, an AI race, organisational risks and rogue AIs. Three of the four involve no machine with goals of its own.
Measures justified at any probability: Testing before deployment, honest incident reporting, the ability to stop a system, keeping people capable of the work, and explanation and appeal. They are worth having whatever the probability of catastrophe.
Claim card: A written record that answers ten questions about a claim: what is claimed, by whom, on which rung, capability, propensity or realised harm, conditions, strongest counter-claim, interests, what would change it, implication and label.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Match

Each camp at its strongest

The chapter states every camp at full strength. Match each camp to the point the chapter calls its strongest.

Interactive exercise 2 · Scenario

Which voices count as evidence

Apply the rule Hansa Mutual's emerging-risk team adopts for weighing statements, scenarios, surveys and findings.

Teaching plan

15 min	The teacher presents the five camps from Table 3.1, each at full strength; participants write down the single strongest point of the camp they find least convincing.
10 min	Review what experts and forecasters believe: the survey of 2,778 authors and the tournament in which neither group moved much; discuss why the figures express uncertainty and not frequency.
10 min	Walk through incentives for developers, investors, critics, states and the press, then list the common ground and the measures whose cost depends on the probability.
30 min	Exercise: each participant completes a claim card for a real claim; a second person who did not select the claim reviews it, as the practical guide requires.
10 min	Plenary: compare labels and decisions across cards, and discuss how Hansa's rule separates evidence, argument and opinion.

Discussion

The one-sentence extinction statement said nothing about likelihood, timing or mechanism. How did that make it both easy to sign and hard to act on?
The race case says unilateral restraint is abdication; its critics say the race is partly self-fulfilling. What evidence would help a European board weigh the two?
The chapter notes that this book also has an interest in its question seeming urgent. How should a reader, or a board, correct for the interests of the people who brief it?

Written exercise

Exercise · 30 minutes

Write and review a claim card

Choose one claim about AI risk that has reached your board, your management team or the press in the last year. Answer the ten questions of the practical guide in writing: separate harm, severity, timing and mechanism; name the claimant and venue; state the rung and whether the evidence shows capability, propensity or realised harm; describe the conditions; name the strongest counter-claim and who holds it; record the interests on both sides; state what observation would move the claim a rung; state your exposure and influence; and assign the label with the decision it justifies. Then swap with a colleague who reviews the card.

Deliverable: A reviewed claim card with counter-claim, interests on both sides, label, decision and a review trigger.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

On which rung of the evidence ladder does the chapter place the statements of 2023 and the Bletchley Declaration?
Show the answer
C. Argument, because they record who was worried, not observations of harm
The chapter states that all of these statements stand on the rung of argument: they record who was worried, not observations of harm.
What was the most striking finding of the Existential Risk Persuasion Tournament, as the chapter reports it?
Show the answer
B. Neither superforecasters nor domain experts moved much towards the other after extended discussion
The median superforecaster put AI-caused extinction by 2100 at 0.38 per cent and the median domain expert at around three per cent, and months of exchanged arguments barely narrowed the gap.
In Nordvik's claim card for the 2023 extinction statement, which label and decision are recorded?
Show the answer
A. Speculation; reversible measures only, with no change to the Region's AI use on this claim alone
The claim stands on the rung of argument and the Region cannot influence the threat, so the card labels it speculation and allows reversible measures only.