Organisational Defences: Containment, Evidence and Decision Rights

All courses / AI Threat / Module 6 / Chapter 30

Learning objectives

After the lesson the participant can:

Classify a threat as directly influenced, influenced only through exposure, or beyond influence, and state what the organisation still decides in each case.
Match the defences for each of the four scales to the threats they answer.
Assign the decision rights to stop, pause, deny or reverse, with holders and the time the threat allows.
Build a coverage matrix that classifies each control as verified, declared or failed and records shared dependencies.

Key concepts

Six disciplines: Oversight that keeps people capable, harm-path reviews, containment that holds whatever the system intends, evidence produced by operation, decision rights assigned in advance, and autonomy matched to reversibility.
Rung sets the strength of the response: A threat at incident in the wild or pattern at scale warrants controls built and tested now; one at laboratory demonstration warrants controls designed to hold if the behaviour appeared in deployment; one at argument warrants mainly reversible dependence and capable people.
Rights to stop: A second set of decision rights, beside the tiers that approve a system: who may stop, pause, deny or reverse. They sit with people who can act within the time the threat allows, and stopping must be cheaper to exercise than continuing.
Verified, declared, failed: A control is verified when a test that re-creates its scenario passed within cadence and produced machine-generated evidence. Attestation alone counts as declared. A control is a defence only when such a test has passed recently.
Shared dependency: A credential, platform, supplier or person on which several controls rely. Controls that would fail together count as one, because tests share failure modes with the controls they test.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Sort

Defences by scale

Each scale of threat has its own family of defences. Sort each defence under the scale the chapter assigns it to.

Interactive exercise 2 · Scenario

Verified or declared

Apply the classification of evidence in the control-to-threat coverage matrix.

Teaching plan

12 min	The teacher presents the three positions of a threat and the rule linking rung and response, then the six disciplines and how existing duties under the AI Act, NIS 2 and DORA take this shape.
20 min	Walk through the defences scale by scale using Table 30.1; for each scale participants name what is new in this book compared with the engineering of the earlier volumes.
13 min	Study the table of decision rights; participants fill in the holder and the time allowed for three of the decisions in their own organisation and note any that nobody holds.
35 min	Exercise in groups: build a coverage matrix for four threats following the eight steps of the guide and CCM-HM-2027-01, and write the findings.
10 min	Read the matrix from the board's side: which flank is verified, which is declared, which has a failed test; compare Nordvik's contractual defences with Hansa's engineered ones.

Discussion

The chapter says the adversary's AI seldom defeats the technical control head-on; it defeats the human approval, the slow decision and the gap between an alert and the person entitled to act. Where is that gap widest in your organisation?
The decision record should state that a stop in good faith is never a failure. What in your organisation's culture or incentives currently makes stopping more expensive than continuing?
Hansa's defences against its own agents are verified while those at the scale of the mind and of fairness are partly declared. Why do organisations tend to verify technical controls first, and what would it take to verify the others?

Written exercise

Exercise · 35 minutes

A control-to-threat coverage matrix

Start from four AI-related threats in your register, one for each scale if you can, with their rung and label. For each, list the controls you rely on, marking each as preventive, detective or corrective and noting where it comes from. Record shared dependencies and count controls that would fail together as one. Name the verifying test for each control, its owner and cadence, and classify the control as verified, declared or failed on the evidence you actually hold; attestation alone is declared. Check the rule that every observable or expected threat needs a verified preventive and detective control. Write one finding per gap with action, owner and due date, and say which decision tier receives gaps beyond the owner's authority.

Deliverable: A coverage matrix for four threats with controls, kinds, tests, status and shared dependencies, plus a list of findings with owners and dates.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

In the table of decision rights, who holds the right to stop an agent or revoke its credentials, and within what time?
Show the answer
D. At least two named stop holders, around the clock and without prior approval, within minutes
Rights to stop must sit with people who can act within the time the threat allows; for an agent that is minutes, which is why more than one named holder is required.
When does a control count as verified in the coverage matrix?
Show the answer
B. When a test passed within cadence and produced machine-generated evidence
The chapter builds on the distinction between attestation and evidence produced by operation: attestation alone counts as declared, and a declared control is not yet a defence.
What does the chapter say is new in the standard of evidence for containment against machines beyond control?
Show the answer
C. A control counts only when shown to hold without the cooperation of the system it contains
Part V added tests that assume non-cooperation: outcome checks the agent cannot see, monitoring and decoys beside pre-deployment tests, a stop-path drill with a non-cooperative stand-in and reconstruction from logs.