Verifying What AI Proposes: Review for Soundness When the Reviewer Depends on the AI · The Transformed Security Architect: the course

All courses / The Transformed Security Architect / Module 3 / Chapter 14

Learning objectives

After the lesson the participant can:

Define soundness in advance through grounding, completeness, consistency with principles and stability under re-asking.
Apply independent re-derivation, oracles and provenance to a proposed change and explain why reading the rationale is not enough.
Assign a review tier to a change by rule from reversibility, blast radius and artefact type.
Design a team-level measurement of review quality that does not monitor individuals.

Key concepts

Correct-looking versus sound: A proposal is correct-looking when it is well formed and carries a rationale that reads well; it is sound when its reasoning would survive scrutiny by someone who knows the system. Soundness is defined before review starts so the proposal does not define it.
Expectation note: Two or three lines the reviewer writes before opening the diff, stating what the change should contain and what it must not touch. It is the lighter form of independent re-derivation and anchors the review in the reviewer's reasoning.
Oracle: A check that decides whether an output is acceptable without reference to how it was produced. Oracles must be outside the proposer's reach, a proposal that changes one takes the highest tier, and a missing test for the property at risk is a finding.
Review tiers: Tier 0 routine, Tier 1 standard, Tier 2 significant and Tier 3 irreversible or self-referential, assigned by rule before any human reads the proposal. A human may raise a tier but not lower it without a recorded reason.
Signals of rubber-stamping: Time to approval relative to size and tier, a rejection and amendment rate near zero, reversals of approved changes, and the seeded-error detection rate, the only signal that measures review directly. Each is a finding about the review design, not about a person.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Sort

Which review tier

Sort each AI-proposed change under the tier the chapter's protocol assigns it by rule.

Interactive exercise 2 · Scenario

Two runs, two different changes

Apply the soundness property of stability under re-asking.

Teaching plan

15 min	Lecture on the open question, correct-looking versus sound, and the four soundness properties with the mechanical part the pipeline checks.
15 min	The teacher presents the three techniques and the design of the critic agent; participants explain why agreement between two agents is not verification.
15 min	Walk through REV-HM-0014: how the defect was found by a test the reviewer wrote, and how one critique objection was disposed of, not obeyed.
35 min	Exercise: participants tier three proposed changes, write an expectation note for one, name the oracle that would catch its likely defect and draft the review record.
10 min	Debrief on Nordvik's seeded errors, the missed citation S3 and the change it caused in the protocol.

Discussion

The chapter says slowing the agents is cheap to reverse, whereas thinning the review is invisible. How would you make that case to a manager who measures the team by throughput?
When an approved change later proves defective, the record shows whether the review or the tier rules failed. Why does that distinction matter for how the organisation responds?
What would have to be agreed with employee representatives before you could measure time to approval and seeded-error detection in your team?

Written exercise

Exercise · 35 minutes

Review an AI-proposed change by tier

Take three AI-proposed changes from your own practice, or use these: an evidence-link refresh, a change to one agent's tool policy for a single market, and a new requirement in a tender annex. Assign each a tier by rule from reversibility, blast radius and artefact type, and state the techniques and human review that tier requires. For the tool-policy change, write the expectation note before reading any proposed text, name the oracle that should pass and the property for which a test might be missing, list the provenance you would require, and dispose of two imagined critique objections with reasons. Record the result as a review record with tier, oracles, critique, reviewer, minutes spent and decision.

Deliverable: A three-row tiering table and one review record in the form of REV-HM-0014 with expectation note, oracle results, objections and dispositions, and a reasoned decision.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

Which soundness property does the chapter call the most specific to AI?
Show the answer
A. Stability under re-asking: the same inputs should produce materially the same change
When two runs on identical inputs produce materially different changes, the proposal was not determined by the inputs alone. Instability does not prove a proposal wrong, but it is a cheap and reliable reason to look harder.
How was the defect in the policy change of review record REV-HM-0014 found?
Show the answer
C. The reviewer saw no test covered market binding, wrote one, and the proposal failed it
The oracle tests passed, but the domain security architect found that no test covered market binding for the new endpoint, wrote one, and the proposal failed it: the rule allowed the endpoint for claims from every market.
Which tier does a change to an oracle, to the review protocol or to a mandate manifest always take?
Show the answer
B. Tier 3, the highest, with full re-derivation and two reviewers or the decision forum
A change to an oracle, to the review protocol or to a mandate manifest always takes the highest tier, because it changes the standard every later proposal is judged against. Tier 3 covers irreversible or self-referential changes.