When AI Is the Threat: AI-Enabled Attackers and the Collapsing Exploitation Window

All courses / AI Platform and Security / Module 4 / Chapter 18

Learning objectives

After the lesson the participant can:

• Explain why publishing a fix starts a race the defender begins with a handicap.
• Classify vulnerability findings into tiers by known exploitation, exploitation probability and exposure.
• Design an approval process for high-value or irreversible actions that does not rely on appearance.
• Draft a 30-60-90-day plan that sequences the response for an organisation.

Key concepts

Exploitation window

The time between a patch or advisory and exploitation. The chapter reports measurements falling from weeks to days and an estimate in which exploitation routinely precedes the patch, so a monthly patch cycle no longer matches the threat.

Patch velocity

The time from advisory to protection, treated as a measured service level with an emergency path. In the chapter's policy, tier 1 findings are protected within 48 hours by a patch or a compensating control.

Compensating control

A measure such as virtual patching, disabling the feature or isolating the system that carries the risk when a patch cannot be applied in time. It is prepared in advance and recorded with an expiry date.

Synthetic-media-resistant approval

For actions that move money, change payee details, grant privileged access or deploy to production: verification through an independent channel registered in advance and approval by a second person who did not receive the request.

The shared principle

Every boundary around an AI workload, and every approval of a consequential action, should be enforced by something that neither an attacker nor the workload itself can change. None of the responses relies on predicting the actor.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Only a small fraction of disclosed vulnerabilities is exploited, which is why prioritisation works at all. What does your organisation prioritise by today, and does it include exposure?
2. An alert to a person not empowered to stop the workload produces a record, not a response. Who in your organisation may stop a production workload without a meeting?
3. Why is refusing an urgent request from a senior executive described as protected behaviour, and what would have to change for staff in your organisation to believe it?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. Why does the chapter say that the defender begins the race after a published fix with a handicap?

   1. Because the attacker needs one exploit and the defender needs every instance patched
   2. Because vendors publish patches to attackers before they reach customers
   3. Because defenders may not use AI to triage advisories under the AI Act
   4. Because patches for AI components cannot be tested before deployment
   Show the answer

   A. Because the attacker needs one exploit and the defender needs every instance patched
   A security patch tells the attacker where the flaw was; the attacker then needs a single working route, while the defender has to find and protect every instance.

2. What lesson does the chapter draw for architects from the Arup case?

   1. That video conferencing must be removed from finance departments and replaced by signed e-mail for every payment request
   2. That the firm's internal systems had been compromised months earlier
   3. That deepfake detection software would have stopped the transfers
   4. That the approval process was the vulnerability: it treated a familiar face and voice as proof of identity and authority
   Show the answer

   D. That the approval process was the vulnerability: it treated a familiar face and voice as proof of identity and authority
   Arup stated that none of its internal systems were compromised. Nothing was hacked; the process trusted appearance, which is why independent-channel verification and a second approver are required.

3. In the patch-velocity policy, what does the service level for tier 1 require?

   1. A patch deployed to every instance within 30 days of the advisory, verified by closing the change ticket for each system
   2. A risk acceptance signed by the CISO within seven days
   3. A patch or compensating control in place within 48 hours, through a pre-approved emergency change path
   4. A vendor confirmation that no exploit exists before any change is made
   Show the answer

   C. A patch or compensating control in place within 48 hours, through a pre-approved emergency change path
   Tier 1 covers known-exploited findings and internet-facing findings with a high exploitation probability; protection, not only patching, is required within 48 hours.