Policy, Pipeline and Evidence Tools

All courses / Tools / Module 2 / Chapter 5

Learning objectives

After the lesson the participant can:

Explain the four conditions under which a rule becomes a tool in the sense of the chapter.
Read a Rego gate and state its input document, its deny reasons and its default.
Distinguish a bill of materials, a provenance attestation and a signature by the question each answers.
Select one shape of evidence manifest for a given question and justify the choice.

Key concepts

Policy decision point: The place where an engine evaluates a structured input against policy and returns allow or a set of deny reasons. In the series it is not controlled by the author of the change and stays outside the agent at every degree of delegation.
Change gate policy: An illustrative Rego policy of AI Operations that computes the change class from the plan, requires an independent named human approver for agent-authored change and refuses release outside the window. The author's declared class is treated as a claim.
Model approval policy: A Rego policy of AI Governance, Chapter 9, that approves a model version only if its model card is complete, its evaluation meets the thresholds of its risk tier, its licence is allowed, its provenance attestation verifies and its register entry is within appetite.
Evidence manifest: One idea in six shapes across six volumes, binding artefacts to controls, a release to obligations, a promotion to its checks or a re-test to a finding. A team should choose one shape, keep it in version control and index it.
Evidence pack index: An artefact of AI Governance, Chapter 11, family EP: a signed index of audit questions, the artefacts that answer them and their hashes, generated by query so a second auditor can regenerate it. An empty section is itself a finding.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Match

Which policy gates which decision?

Match each Rego policy or constraint of Chapter 5 with the decision it gates.

Interactive exercise 2 · Sort

Three questions about an artefact

Supply-chain evidence answers three questions. Sort each tool by the question it answers in the chapter.

Teaching plan

15 min	Present Figure 5.1, the policy, pipeline and evidence chain, and its two recurring properties: the decision point sits outside the author, and the pipeline writes the evidence store.
20 min	Read two gates side by side: the model approval policy and the change gate policy. Participants name the input document, the deny reasons and the default of each.
15 min	Teach supply chain and evidence: SLSA and in-toto, Sigstore and OpenSSF Model Signing, the ML-BOM, the six evidence manifests, and the warning that a scanner is itself a dependency.
30 min	Exercise: groups specify a gate and its evidence for one change type, following the five steps of the unsigned-model example at Hansa Mutual.
10 min	Debrief on what each group's gate does not prove, and where a check on the resulting state would sit.

Discussion

The worked example's gate checks that a signature was verified, not that the signer was the right party or that the model is safe. Which other tools of the chapter carry those claims?
The series records that a scanner one of its volumes lists to rely on was compromised. What follows for the way a team pins, verifies and reviews the tools that produce its evidence?
Why does the series insist that an agent-authored change needs a human approver who is neither the requester nor the author, even at Level 3 Bounded?

Written exercise

Exercise · 30 minutes

Specify a gate and the evidence it leaves

Choose one kind of change in your own organisation that should not reach production unchecked, or use the promotion of a model at Hansa Mutual. Specify a pipeline gate for it without writing code: the input document the policy reads, at least four deny reasons in plain sentences, the default decision, who controls the decision point, and the test you would write for each deny reason. Then specify the evidence the run leaves: which shape of evidence manifest you choose from the six in the chapter and why, what it binds, who signs it and when it expires. Finish with two things the gate does not prove.

Deliverable: A one-page gate specification with input, deny reasons, tests, decision point owner, chosen evidence manifest shape, signer, expiry and stated limits.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

According to the chapter's Summary, which four things make a rule a tool?
Show the answer
B. An engine can evaluate it, it has tests, a decision point its author does not control evaluates it, and every decision leaves a record
The chapter's own conclusion names exactly these four conditions. Open Policy Agent and Rego are the common engine and language, but the conditions, not the product, make the rule a tool.
Which property of evidence does AI Purple add that the earlier evidence manifests lack?
Show the answer
D. The evidence carries an expiry
The chapter states that AI Purple adds the property that the earlier manifests lack: evidence expires. Its manifest also records the versions tested, the runs including the failed one, and who reproduced and signed it.
What does the reconciler sync policy of AI Operations say about an agent identity and the right to sync a state engine?
Show the answer
A. An agent identity with the right to sync is something the artefact must never contain
In the table across the degrees of delegation, the operations agent of the example may read state but cannot sync, and an agent identity with the right to sync is a thing the artefact must never contain.