AI Red Team: the course

About the course

The course follows AI Red Team, Book 4 of The Transformed Architecture, through NIST CSF 2.0. It starts with the evidence on how attackers use AI, the framework read through three AI focus areas, and a reference architecture that places security AI on the governed platform. It then works through Govern, Identify, Protect, Detect, Respond and Recover, and closes with security chaos engineering and the purple loop. Red teaming, with AI and of AI, runs through every module, always authorised, scoped by rules of engagement and ending in a verified control. The course matters because exploitation now commonly precedes the patch, and defenders in European enterprises and public authorities must act at machine speed without giving up human command or their obligations under NIS 2, DORA and the AI Act.

After the course the participant can

Label AI-enabled threats as observable, expected or speculation and translate them into requirements per CSF Function
Draft a risk appetite for security AI with autonomy ceilings per usage mode and assign an autonomy level to each action of a capability
Write rules of engagement as a signed, enforceable manifest for an authorised exercise in which AI proposes or executes steps
Design the placement, identity, guardrails and degradation modes of a security AI capability on the governed platform
Review AI-drafted threats, detections, findings and triage verdicts for soundness against evidence
Plan red teaming of a compound AI system and a containment test for an agent, with residual risk accepted by a named owner
Map one incident to every reporting clock and bound machine-speed containment in a catalogue of reversible actions
Run restore drills, security chaos experiments and the purple loop so that every finding ends as a verified control

Who it is for

Chief information security officers and heads of security operations
Red-team leads and AI red-teaming practitioners
Blue-team leads, detection engineers and threat-intelligence analysts
Security and AI architects
Risk, compliance and internal-audit functions
Incident-response and recovery owners

Before the course

Working knowledge of security operations: SIEM, EDR, incident response and vulnerability management
Familiarity with MITRE ATT&CK as a vocabulary of adversary techniques
Helpful but not required: Architecture as Code, AI Governance and AI Platform and Security, whose conclusions the course summarises where it builds on them

Material

Three ways to run it

The same material in three lengths. Each row of an agenda names the lessons it draws on.

Executive briefing90 minutes

For management bodies, executive teams, chief risk officers and CISOs who must decide how far security AI may act

A briefing on the decisions only the management body can take: the highest usage mode and autonomy level it accepts for security AI, the risk appetite behind them, and the evidence it should receive. It uses the two fictional organisations to show the same programme shape at two scales and ends with the three tests from Chapter 27.

0:00	Why now: exploitation before the patch, AI as operator for attackers, and what active defence does and does not mean Ch. 1, Ch. 2
0:15	Usage modes and autonomy levels: who decides each action, with whose identity, and the line between Level 2 and Level 3 Ch. 6
0:30	Risk appetite and ceilings: error, autonomy and reversibility, and the vendor features that already act automatically Ch. 5, Ch. 8
0:45	The neglected Function: recovery as a measured capability reported beside patch velocity Ch. 22, Ch. 24
1:00	Evidence for oversight: chaos experiments, the purple loop and a CSF profile that moves only with evidence Ch. 25, Ch. 26
1:15	The mandate, inventory and evidence tests applied to the participants' own organisation, and the decisions to take next Ch. 27, Ch. 28

One-day seminar1 day

For security leaders, architects, SOC and red-team leads who will design or run an AI-enabled defence programme

A working day that follows the six CSF Functions with one artefact per block: a decision record with autonomy per action, a rules-of-engagement manifest, a guardrail policy, a containment catalogue and a chaos experiment. Every exercise is a design, review or tabletop exercise on the participants' own or the fictional organisations' systems.

09:00	Foundations: the threat picture labelled by evidence, CSF 2.0 with three AI focus areas, and security AI on the governed platform Ch. 1, Ch. 2, Ch. 3, Ch. 4
10:00	Govern: appetite, ceilings and an autonomy-level decision record written action by action Ch. 5, Ch. 6
11:00	Govern: rules of engagement as code and the effective autonomy of supplier AI Ch. 7, Ch. 8
12:00	Lunch break
13:00	Identify: adversary emulation under safeguards and red teaming of the compound AI system Ch. 12, Ch. 13
14:00	Protect: approvals that resist synthetic media, guardrails outside the model and safe degradation Ch. 14, Ch. 16, Ch. 17
15:00	Detect and Respond: grounded triage, the investigation agent, reporting clocks and a containment catalogue Ch. 18, Ch. 20, Ch. 21
16:00	Recover and verify: restore before repair, chaos experiments and the purple loop Ch. 22, Ch. 25, Ch. 26
16:45	Close: the three tests and each participant's first ninety days Ch. 27, Ch. 28

Full course8 weeks, one module a week

For security teams, architects, red and blue team members and risk functions who will build the programme and its artefacts

The full course takes one part of the book each week: a lecture on the part, one lesson per chapter with an exercise that produces a real artefact, and a self-test. Participants build towards the final assignment, a programme design for AI-enabled defence in their own organisation or in one of the two fictional organisations.

Week 1	Foundations: thesis and rules of the book, the threat landscape, CSF 2.0 with an AI focus and the reference architecture Ch. 1, Ch. 2, Ch. 3, Ch. 4
Week 2	Govern: context and risk strategy, roles with usage modes and autonomy levels, policy and rules of engagement, oversight and supply chain Ch. 5, Ch. 6, Ch. 7, Ch. 8
Week 3	Identify: asset management, threat modelling, vulnerability and exposure, adversary emulation and red teaming AI systems Ch. 9, Ch. 10, Ch. 11, Ch. 12, Ch. 13
Week 4	Protect: identity and access, awareness and training, data and platform security, infrastructure resilience Ch. 14, Ch. 15, Ch. 16, Ch. 17
Week 5	Detect: the AI-assisted SOC, and detection engineering, threat intelligence and deception as code Ch. 18, Ch. 19
Week 6	Respond: incident management and analysis, reporting, communication and mitigation Ch. 20, Ch. 21
Week 7	Recover: recovery plan execution, recovering AI systems and security AI, proactive recovery Ch. 22, Ch. 23, Ch. 24
Week 8	Verification and conclusions: security chaos engineering, the purple loop, the two organisations and the eight conclusions; presentation of the final assignment Ch. 25, Ch. 26, Ch. 27, Ch. 28

Modules and lessons

One module for each part of the book and one lesson for each chapter. A lesson has learning objectives, key concepts, a teaching plan, discussion questions, an exercise and a self-test.

Module 1 · Part I · Foundations

Part I sets the ground for the whole course. Participants meet the argument that AI must become an active part of cyber defence, the evidence on how attackers use AI, NIST CSF 2.0 read through three AI focus areas and mapped to European obligations, and a reference architecture that places security AI on the governed platform. They leave with a labelled threat profile entry, a CSF profile entry and a placement decision for one security AI capability. Lecture: Foundations: from threat picture to a governed place for security AI

Module 2 · Part II · Govern

Part II covers the Govern Function for AI in cyber defence. Participants write a risk appetite with autonomy ceilings per usage mode, assign usage modes and autonomy levels per action, turn policy into enforceable rules and signed rules of engagement, and extend oversight and supply-chain risk management to the AI that arrives inside products and managed services. They leave with four artefacts: an appetite statement, an autonomy-level decision record, a rules-of-engagement manifest and supplier assurance requirements. Lecture: Govern: who decides what security AI may do, and who can stop it

Module 3 · Part III · Identify

Part III covers the Identify Function. Participants inventory an estate that includes models, agents, prompts, connectors and security AI; keep threat models and risk assessment continuous with AI drafting and named people deciding; manage vulnerabilities and exposure when every patch is an exploit specification; and run authorised adversary emulation and red teaming of their own AI systems and agents. Every finding in this part ends with the control that closes it and the test that verifies it. Lecture: Identify: know the estate, the paths through it and whether the controls hold

Module 4 · Part IV · Protect

Part IV covers the Protect Function. Participants design identities for people, machines and agents, move approvals off appearance and onto call-back, dual control and out-of-band confirmation, build resilience to AI-generated deception by testing processes and not people, protect security data and place guardrails outside the model, and make infrastructure and security AI degrade safely. They leave with a credential profile, an exercise design record, a guardrail policy and a degradation-mode definition. Lecture: Protect: controls that hold however convincing the input

Module 5 · Part V · Detect

Part V covers the Detect Function. Participants place AI step by step in the SOC workflow, require grounded triage in which every claim cites its evidence, govern investigation agents through a permission manifest and five conditions for acting without per-action approval, and monitor AI systems including the SOC's own. They then build what feeds the SOC: detections as code with tests and a rationale, graded threat intelligence in STIX, and deception registered as code and kept inside the estate. Lecture: Detect: grounded signals, bounded agents and analysts in command

Module 6 · Part VI · Respond

Part VI covers the Respond Function. Participants learn to declare, prioritise and coordinate incidents with AI as assistant and people in command, to reconstruct timelines and scope with re-runnable impact queries, to keep AI analysis defensible under chain of custody, and to handle incidents in which an AI system is target, channel, subject or instrument. They then map one incident record to every reporting clock, draft reports under human sign-off, and bound machine-speed containment in a catalogue. Lecture: Respond: faster work, human command and one record for every clock

Module 7 · Part VII · Recover

Part VII gives the Recover Function a part of its own, because it is the most neglected Function and the one AI can change most. Participants write recovery plans as code, sequence and verify restoration in a clean room when the adversary may still be present, restore AI systems and agents from the evidence bundle with re-evaluation before return to service, and turn recovery into a measured property through monthly drills, findings routed to other Functions and prepared communication. Lecture: Recover: from an assumed plan to a proven capability

Module 8 · Part VIII · Verification, Improvement and Conclusions

Part VIII closes the loop. Participants design security chaos experiments with steady-state hypotheses and the safety boundaries of a platform that disturbs controls on purpose, run the purple loop from finding to verified control, measure defensive AI by outcomes and evaluate it against golden datasets, compare how Nordvik Region and Hansa Mutual applied the book, and draw the eight cross-cutting conclusions into a plan for their own organisation's first ninety days. Lecture: Verification and improvement: evidence that the controls still hold

Final assignment

Capstone

A programme design for AI-enabled defence, from mandate to verified control

Design an AI-enabled defence programme for your own organisation, or for Nordvik Region or Hansa Mutual, that would pass the mandate, inventory and evidence tests of Chapter 27. The design covers all six CSF Functions but goes deep on one security AI capability of your choice, carried from charter to verification. Everything is a design, review or tabletop artefact: no offensive tooling, test inputs or procedures are written, any exercise you plan is authorised and scoped by rules of engagement against your own or the fictional organisations' systems, and every finding you anticipate ends in a closing control and a re-test. Use the book's terms for usage modes, autonomy levels and CSF Categories exactly.

Steps

Write a one-page programme charter with scope in CSF terms, ceilings on usage mode and autonomy by type of action, decision rights and named stop-path holders.
Inventory the security AI in use, including vendor features that act automatically, with usage mode, effective autonomy, owner and identity.
For one capability, write the decision record with autonomy per action, the placement and guardrail policy, and the degradation modes.
Draft rules of engagement for one authorised exercise or AI red-teaming campaign that tests the capability, with objectives written for the controls that should hold.
Define two chaos experiments, one restore drill and the evaluation that gate the capability, each with a hypothesis, time bound and evidence output.
Describe how findings from all sources pass through the purple loop into the CSF profile and the report to the management body.

Deliverables

Programme charter and inventory of security AI with effective autonomy
Capability dossier: decision record, guardrail policy and degradation modes
Rules-of-engagement manifest and test plan written as control objectives
Verification plan with chaos experiments, restore drill, evaluation thresholds and a sample finding carried to closure
A first-90-days plan with milestones mapped to CSF Categories

Assessment criteria

Every autonomy level is set per action, justified by reversibility, blast radius and confidence, and recorded with the forum that decided it
Controls that bound the capability sit outside the model, and a named person can stop it through a path that does not depend on it
All offensive activity is authorised, scoped and defensive in purpose, described by technique without procedures, tooling or test inputs
Every claim of capability is tied to a dated test, drill or experiment, and every finding to a closing control and a re-test
The design uses the book's terms exactly and attributes legal statements to instrument and article as architectural implications