Frontier Safety Frameworks: What Developers Promise and What It Is Worth

All courses / AI Threat / Module 6 / Chapter 29

Learning objectives

After the lesson the participant can:

• Describe the six parts of a frontier safety framework and the two features the chapter says deserve attention.
• Explain why elicitation, evaluation awareness and snapshots limit what dangerous-capability evaluations show.
• Classify the content of a system card as context, third-party attestation or input to own tests.
• Derive a supplier evidence request from a provider's framework, with notification duties and a rule for handling refusals.

Key concepts

Framework anatomy

Each framework names risk categories, defines capability thresholds, specifies the evaluations that decide whether a threshold is reached, requires deployment and security safeguards, assigns governance and provides for its own revision.

Elicitation

The effort made to bring out a model's full capability through prompting, tools, scaffolding or fine-tuning. Weak elicitation understates capability, and because elicitation is open-ended a model may reach further after release while its system card stays unchanged.

Structured attestation

What a system card is: a statement by the developer, backed by evidence the deployer cannot see. It carries weight to the degree that the process behind it is disciplined and partly external, and is recorded as context.

Restricted release

Withholding a model from general availability while offering access to defenders first. It keeps a point of control, but the developer decides alone who counts as a partner, and it lasts only until comparable capability appears elsewhere.

Acceptance rule

In the worked example, provider answers are context and acceptance rests on the deployer's own evidence: tests of the compound system, monitoring, containment and a stop path. Each unanswered item is recorded as a finding.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Disclosures against the developer's immediate interest are taken as evidence that system cards are not pure marketing. What other signs would raise or lower your confidence in a provider's documentation?
2. One framework states openly that requirements may be adjusted if a competitor releases a high-risk system without comparable safeguards. Is that honesty reassuring or worrying for a deployer, and what should a contract say about it?
3. Many European organisations prefer open-weight models for sovereignty reasons, and these often come without a framework at all. How should that absence be weighed against the reversibility such models offer?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. According to the chapter, what are dangerous-capability evaluations better at showing?

   1. That a capability is absent than that it is present
   2. That a model's propensity in deployment is low
   3. That a capability is present than that it is absent
   4. That the deployed compound system is safe
   Show the answer

   C. That a capability is present than that it is absent
   Weak elicitation understates capability, models can recognise evaluation settings, and results come from snapshots; the evaluations remain the best instrument available but clear a model less reliably than they flag one.

2. How should a deployer record a provider's system card, following the chapter?

   1. As context, not as evidence about its own system
   2. As proof that its own compound system is safe
   3. As marketing material to be disregarded entirely
   4. As a legal guarantee enforceable against the provider
   Show the answer

   A. As context, not as evidence about its own system
   A system card is structured attestation backed by evidence the deployer cannot see; it describes the model, not the system the deployer builds with its own instructions, tools and permissions.

3. What weakness of restricted release does the chapter identify?

   1. It removes every point of control over the model
   2. It makes model weights impossible to protect
   3. It prevents defenders from finding and fixing any vulnerabilities in critical software before attackers exploit them
   4. The developer alone decides who is a partner, and it lasts only until comparable capability appears elsewhere
   Show the answer

   D. The developer alone decides who is a partner, and it lasts only until comparable capability appears elsewhere
   Access concentrates in large vendors and infrastructure operators, so a European deployer should ask whether the suppliers who maintain its critical software use the head start, and plan for its end.