Security Chaos Engineering with AI: Verifying Security Capabilities Continuously

All courses / AI Red Team / Module 8 / Chapter 25

Learning objectives

After the lesson the participant can:

• Distinguish penetration testing, breach-and-attack simulation and security chaos engineering by what each is for.
• Write a falsifiable steady-state hypothesis with condition, expected outcome, observation and time bound.
• Explain why the AI planner and evaluator hold no credentials to the estate and what the abort controller does.
• Draft a chaos experiment manifest with injection, rollback, blast radius, autonomy level and evidence.

Key concepts

Steady-state hypothesis

A falsifiable statement of what should happen, measured from outside the control being tested, naming the condition, the expected outcome, how it is observed and the time within which it must occur. The statement that the EDR works is not a hypothesis.

Conditions, not attacks

Most experiments contain no attacker technique: they inject conditions, including failures of security controls themselves, and measure whether the organisation notices and acts. Where a technique is emulated, a catalogued, benign emulation is used.

Three zones

The planning zone holds the catalogue and the AI planner and evaluator, with no estate credentials. The control zone holds the policy decision point, scheduler and approval gate, abort controller, ledger and evidence store. The execution zone holds narrow actuators and observers.

Safety boundaries are the design

Eleven boundaries, among them: separate planning from execution, catalogued experiments only, reversibility by construction, bounded blast radius, synthetic data, abort conditions independent of the planner, an out-of-band stop path and treatment of the platform as a tier-0 asset.

Sealed ledger

The experiment's ledger entry is sealed from the SOC for the detection window, so analysts treat the alert as real. The duty shift lead can ask the ledger whether an alert is an experiment at any time without seeing the schedule.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Why is an aborted experiment a finding about the platform or the estate, and not simply a run to repeat?
2. Announced experiments no longer verify detection, and unannounced ones can erode trust in alerts. Does the sealed ledger resolve the tension for your SOC?
3. The platform avoids Mode 5 by design. What would be lost and what would be risked if an AI component held actuator credentials?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. Under the safety boundaries, what may the AI planner do with the experiment catalogue?

   1. Generate new injection logic when it finds a coverage gap
   2. Write emulation code for techniques seen in new threat reports
   3. Execute catalogued experiments directly with actuator credentials
   4. Choose and combine catalogued types within their parameter ranges
   Show the answer

   D. Choose and combine catalogued types within their parameter ranges
   The AI may choose and combine catalogued types within their parameter ranges but may not generate new injection logic, emulation code or attacker behaviour. A new type is a reviewed catalogue change, and the planner holds no credentials.

2. Which component watches guard metrics and triggers rollback independently of the planner and actuators?

   1. The abort controller in the control zone
   2. The AI evaluator in the planning zone
   3. The observers in the execution zone
   4. The purple backlog outside the zones
   Show the answer

   A. The abort controller in the control zone
   The abort controller watches guard metrics such as service health, error rates and business transactions, and triggers rollback independently, including when the platform itself behaves unexpectedly.

3. By the chapter's defaults, how many safe runs in test does an experiment type need before promotion to production?

   1. Five safe runs, with the SOC informed each time
   2. Ten safe runs, of which two are unannounced
   3. Twenty safe runs
   4. One hundred safe runs across all environments
   Show the answer

   C. Twenty safe runs
   The default values include 20 safe runs before promotion, at most one per cent of targets in production, experiments of at most 30 minutes and at most three concurrent.