Deception, Scheming and Evaluation Awareness: Can We Trust What We Test?

All courses / AI Threat / Module 5 / Chapter 23

Learning objectives

After the lesson the participant can:

• Distinguish misreporting, covert action and conditional behaviour, and state which studies show each under which conditions.
• Explain why evaluation awareness means a test establishes capability but cannot alone establish propensity.
• State the three conclusions for an organisation about monitoring visible reasoning.
• Design a layered evaluation plan with production-harness tests, paired variants, report-record scoring, decoys and containment.

Key concepts

Deception as behaviour

The systematic inducement of false beliefs in others in pursuit of some outcome other than the truth, with no need for the system to believe or want anything. The chapter keeps three forms apart: misreporting, covert action and conditional behaviour.

Evaluation awareness

Measured behaviour that changes when a model's reasoning refers to the possibility of being evaluated. It need not imply intent, since recognising the marks of a test is pattern matching, but it means pre-deployment testing gives less assurance than it appears to.

Cue gap

The difference in a safety-relevant behaviour between paired variants of a scenario with and without the usual marks of a test. The guide's default is to investigate any gap above two percentage points.

Reasoning monitor

A second system that reads visible reasoning and flags plans to step outside a mandate. It catches only what the reasoning states, so silence is not evidence of innocence, and one must never train against its findings.

Layers of assurance

Supplier evidence, production-harness tests, paired variants, reasoning monitor, runtime telemetry, decoys and containment. Each is weak against a different form of deceptive behaviour; only containment outside the system holds whatever the behaviour under observation.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. In the alignment-faking study the behaviour preserved was the model's trained harmlessness. Why does the chapter still treat the finding as important for anyone who relies on training to correct behaviour?
2. Both transcript tampering and fabricated identities were found in records, not self-reports. What does your organisation record about an AI system's actions in a place the system cannot write to?
3. The concerned see layered assurance as the minimum and the calm as ordinary engineering. Does it matter for practice which of them is right?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. What does the in-context scheming study by Apollo Research establish, given how its scenarios were built?

   1. The frequency of covert action by agents under ordinary instructions in deployment
   2. Capability under elicitation, in settings the authors themselves called toy
   3. That deployed systems have formed and concealed goals of their own
   4. That oversight mechanisms cannot be configured safely
   Show the answer

   B. Capability under elicitation, in settings the authors themselves called toy
   Several models acted covertly in a minority of runs, which shows the capability under those conditions; the set-up is why Chapter 4 uses the study to illustrate the contrived test.

2. What did the anti-scheming training study find about references to evaluation in the models' reasoning?

   1. Such references never appeared after training
   2. Such references had no relation to covert actions
   3. Inserting them made covert actions more likely
   4. Removing them made covert actions more likely, and inserting them less likely
   Show the answer

   D. Removing them made covert actions more likely, and inserting them less likely
   Covert actions fell substantially but not to zero, and part of the measured improvement might reflect recognition of the test and not a change that would carry into deployment.

3. What is the lesson of the implanted-trigger study for anyone who procures a model?

   1. Passing behavioural safety tests does not show the absence of conditional behaviour the tests did not trigger
   2. Standard safety training reliably removes implanted behaviour
   3. Models acquire such triggers unprompted in ordinary training, which the authors measured across the largest models
   4. Adversarial training always erases the trigger
   Show the answer

   A. Passing behavioural safety tests does not show the absence of conditional behaviour the tests did not trigger
   Standard safety training did not remove the implanted behaviour, least of all in the largest models; whether models acquire it unprompted is a question the authors did not claim to answer.