Prove Security: Secure Code, Supply Chain and the Toolchain as Attack Surface

All courses / AI Developer / Module 2 / Chapter 10

Learning objectives

After the lesson the participant can:

Explain why the chapter treats generated code as untrusted input and how to measure its effect in one's own code
Design the five controls on the path a dependency name must travel
Classify the inputs of a coding agent by who can write them and limit what the agent can do after reading them
Draft the toolchain controls of a mandate, with separated identities and a stop path

Key concepts

Package hallucination: A coding tool proposes imports, and some of the names it proposes do not exist; one study found a substantial share of suggested packages hallucinated and treats this as a threat to the supply chain. The defence is the path a name must travel, not the tool's accuracy.
A denial that is an event: The fifth dependency control: when the package proxy refuses a name, the refusal is logged with the run that asked and reaches an owner. At Hansa Mutual the proxy held against a non-existent package but the denial had no reader.
Permissions set the cost of being steered: Prompt injection is architectural and not a patchable defect. What an agent can be made to do is what its permissions allow, so the limits are input classification, no unneeded secrets, egress denied by default, reading separated from acting, and gates kept out of reach.
The toolchain as Tier 0: The systems that turn intent into running software are among the most privileged in the estate. Developer, agent, build and release identities are kept separate, and no one identity combines authoring, approving and releasing.
Provenance record: The record PRV of where a piece of software came from. It claims origin only as far as a log supports, says nothing about the share of code a human wrote, and leaves AI comments out of the count of approvals.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Sort

Four identities in the toolchain

Sort each description to the toolchain identity it belongs to in the chapter's design.

Interactive exercise 2 · Scenario

The proxy said no, three times

Apply the fifth dependency control: a denial that is an event.

Teaching plan

15 min	Lecture on what the studies of generated code measured and their three shared limits, and on how to measure one's own merged changes within a quarter.
15 min	The teacher tells the two lessons: the connection string pasted into an unapproved chat assistant at Nordvik Region, and the package that did not exist at Hansa Mutual. The room names the control that held and the one that was missing.
15 min	Whiteboard: draw the toolchain trust boundary with the four identities and mark where signing keys, gate configuration and the stop path sit.
30 min	Exercise: groups write the controls for a coding agent following the eight steps of the practical guide.
15 min	Tabletop: each group explains how its stop path works when the pipeline does not, and when the limits would be re-tested.

Discussion

The studies measure code at the moment of generation, not what survives review and scanning. What would you need in order to state a figure for your own merged code?
At Hansa Mutual the proxy refused the package and nobody read the denial. Which refusals in your own toolchain have no reader today?
The chapter says an untested limit is declared, not verified. How would an authorised exercise under rules of engagement test your agent's limits without creating new risk?

Written exercise

Exercise · 30 minutes

Write the controls for a coding agent

For a coding agent in your organisation, or the agent under MND-HM-DEV-02 at Hansa Mutual, design the controls. List what the agent reads and label each source by who can write it: the team, the organisation, a supplier, anyone. List what the agent can do and strike everything the task does not need. Specify a separate identity with task-scoped, short-lived credentials issued outside its runtime. Route packages through the proxy with denied direct egress, a waiting period and a named approver for any new name. Place the secret filter and log at the gateway. Put gates, signing keys, ownership rules and the mandate out of reach. Write the stop path and the date of its drill.

Deliverable: A toolchain-controls section for a mandate: labelled inputs, pruned permissions, identity, package path, gateway filter, protected assets, stop path and drill date.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

Which of the following is one of the five dependency controls the chapter lists?
Show the answer
D. A waiting period before new versions of existing packages are installed
The five controls are an approved-registry proxy, a human approval for every new dependency, a lock file with hashes, a waiting period for new versions, and a denial that is an event.
What happened in the incident INC-NR-2030-0071 at Nordvik Region?
Show the answer
A. A consultant pasted a test-database connection string into an unapproved public chat assistant
In March 2030 a consultant on the case platform pasted a connection string into an unapproved public assistant while asking for help with a SQL view. The secret in the prompt is the Region's lesson.
What does the provenance record in the chapter say about the share of code written by a human?
Show the answer
B. It says nothing about it, and claims origin only as far as a log supports
The chapter stresses that the record makes a claim about origin only as far as a log supports, says nothing about the share a human wrote, and does not count AI comments as approvals.