Roles, Usage Modes and Autonomy Levels: People in Command of Security AI

All courses / AI Red Team / Module 2 / Chapter 6

Learning objectives

After the lesson the participant can:

• Define the five autonomy levels and say who decides each action at each level.
• Assess an action against reversibility, blast radius, confidence, exposure to attacker-controlled input and the decision tier.
• Explain why the line between Level 2 Supervised and Level 3 Bounded is the most important in the scheme.
• Draft an autonomy-level decision record that names owner, operators, authority and stop-path holders.

Key concepts

White team

The function that controls an exercise: it approves scope and rules of engagement, can pause or end the exercise, deconflicts red activity from real incidents and keeps the record. The white team, not the AI, holds the stop decision for an exercise.

Level 3 Bounded

The AI executes pre-approved, reversible actions within a codified playbook; a person is informed and can stop it. The playbook is written as code and enforced by a policy decision point outside the model.

Corroboration from a deterministic source

If the trigger for an action rests on content an adversary can write, Level 3 requires corroboration from a deterministic source, such as an endpoint detection, so that free text alone never triggers an action.

Owner, operators and authority

Every machine identity has an owner in a line role accountable for purpose, evaluation and residual risk; operators named through a rota who approve, watch and stop; and an authority that may raise the level within the ceiling. Lowering the level is open to every stop-path holder.

Human command by design

Three elements make command real: an interruption path outside the system it stops, an override log of every approval, rejection, modification and stop, and measured approval quality, with time to approval, rejection rate and later reversals per operator.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. The chapter says that at Level 3 the approval moves from the action to the playbook, which concentrates accountability instead of diluting it. Do you agree, and what would make it untrue?
2. Approval-quality metrics are themselves employee monitoring. How would you agree them with employee representatives and still learn what you need?
3. How can an organisation keep analysts competent to supervise and to work in degraded mode when AI triages most alerts?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. At Level 2 Supervised, who decides each action?

   1. The AI decides within a playbook approved in advance, and a person is informed
   2. A person approves each action explicitly, and the AI then executes it
   3. A person reviews the AI's draft and then executes the action personally
   4. The AI selects and executes actions within bounds set in advance
   Show the answer

   B. A person approves each action explicitly, and the AI then executes it
   The table defines Level 2 Supervised as execution by the AI after explicit human approval of each action. Drafting for a person to execute is Level 1 Assisted; a pre-approved playbook is Level 3 Bounded.

2. According to the chapter, what happens when a Mode 4 triage service is given write tools?

   1. It stays in Mode 4 but must move to Level 0 Advisory
   2. It becomes a Mode 3 Agentic work tool and is confined to sandboxes in isolated ranges
   3. It becomes Mode 0 Ambient AI from the SOC's perspective
   4. It moves to Mode 5, a change in governance that triggers reclassification
   Show the answer

   D. It moves to Mode 5, a change in governance that triggers reclassification
   Mode 4 services sit at Level 0 to Level 2 because their identity is read-only. Write tools move the service to Mode 5, which triggers reclassification just as a new model version does.

3. What default evidence does the practical guide ask for before an action is raised to Level 3 Bounded?

   1. A vendor benchmark showing precision above the market average
   2. Thirty days of operation at Level 4 on read-only actions
   3. At least 200 evaluated cases with a precision of 0.9 or more
   4. A signed attestation from every operator on the rota
   Show the answer

   C. At least 200 evaluated cases with a precision of 0.9 or more
   The defaults to adapt include at least 200 evaluated cases with a precision of 0.9 or more before Level 3, a first review 90 days after any increase, and a stop target of two minutes.