Adversary Emulation and Continuous Red Teaming with AI

All courses / AI Red Team / Module 3 / Chapter 12

Learning objectives

After the lesson the participant can:

• Distinguish a threat-informed emulation from a generic penetration test and state what AI may and may not decide in scenario design.
• Assign an autonomy level per step of an emulation plan according to whether the step reads or changes state.
• Map the three controls that failed in July 2026 to their counterparts and design responses in an emulation programme.
• Draft an emulation plan extract and a structured finding record with a closing control and a re-test.

Key concepts

Threat-informed emulation

An emulation that asks whether the adversaries most likely to target this organisation, with the capabilities they have shown, would succeed and whether anyone would notice. Each step maps to an ATT&CK technique and names the detection or control it is expected to meet.

Executing AI as a Mode 5 system

AI that executes emulation steps acts with credentials and network access of its own, so it is treated as Mode 5 for the exercise whatever product it is built on, and receives the full containment baseline, including a permission manifest, budgets and an external stop path.

Tempo

Emulating AI-enabled adversaries includes emulating their speed. In Hansa Mutual's exercise the detection fired, but the alert was queued at medium severity and triaged after 47 minutes: the detection was not missing, its severity assumed a human tempo.

Continuous validation as code

Catalogued, benign technique emulations replayed against the estate as regression testing for Protect and Detect. AI plans and interprets; deterministic components act. A failed test is a finding, closed only when the same test passes again.

Finding record

A machine-readable entry naming the plan step and technique, the expected and observed outcomes with times, the control and never an analyst on shift, and the closing action and re-test. A red-team member confirms every finding and the control owner accepts it.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Why does an agent whose task is to probe for weaknesses need containment more, not less, than a production agent?
2. The finding record names a control, never an analyst on shift. What does that choice protect, and what would be lost without it?
3. What must a white team understand about AI components to control an exercise in which steps complete in seconds?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. What is the ceiling the chapter sets for AI on emulation steps that change state on production systems, use harvested credentials or move data?

   1. Level 1 Assisted at most
   2. Level 2 Supervised at most
   3. Level 3 Bounded within a playbook
   4. Level 0 Advisory only
   Show the answer

   A. Level 1 Assisted at most
   Read-only steps may run at Level 2 Supervised, but steps that change state on production systems, use harvested credentials or move data run at Level 1 Assisted at most, and Level 4 is never used for offensive actions.

2. In an emulation programme, what is the design response to the July 2026 failure of a package mirror that was also an egress path?

   1. A larger action budget for each AI-executed scenario
   2. Exercise alerts routed to the red-team lead on shift, who then decides whether to inform the white team
   3. No shared supporting services between ranges, and every egress route named
   4. A second package mirror held in reserve for exercises
   Show the answer

   C. No shared supporting services between ranges, and every egress route named
   The table maps the mirror to a shared repository, proxy or mirror between ranges, and the response is no shared supporting services with every egress route named.

3. In continuous validation, which components hold estate credentials and act?

   1. The AI planner, which selects tests from coverage gaps
   2. Deterministic components, while AI only plans and interprets
   3. The AI evaluator, which drafts findings on missing outcomes
   4. The red team's coding agents, under the engineer's identity
   Show the answer

   B. Deterministic components, while AI only plans and interprets
   The chapter states that AI plans and interprets and deterministic components act: neither the planner nor the evaluator holds estate credentials or generates emulation logic.