Observer, Stop Path and Evaluation Tools

All courses / Tools / Module 6 / Chapter 19

Learning objectives

After the lesson the participant can:

Specify an observer that meets the three tests of the observer principle for one service function.
Compare the five kinds of independence in AI Purple with the six in AI Operations and apply them as a union.
Design a seeded test with an author, a scope, a way to stop and a catch-rate rule.
Plan a timed stop drill and state what its record must show.

Key concepts

Observer principle: Three tests from AI Operations: verification is independent of the action, the observer does not depend on the observed system, and the stop path works when the platform does not. An agent that reads the health endpoint its own restart resets has a mirror, not an observer.
Observer manifest: An artefact, family OBS, that declares the probes, heartbeat and stop path verifying an agent's actions, and the observer's independence in model, data, authorship, credential, failure domain and time. Its statement of independence expires.
Seeded-fault record: A record, family SDF, of a deliberately introduced, authorised and scoped fault that tests whether the agent's verification and the independent probe notice it. Three kinds of seed: silent failure, observer loss and stop path.
Stop-path test record: A record of AI Threat, family SPT, of a test that an agent can be stopped by a route it cannot block. A failing result blocks any widening of autonomy until a re-test passes.
Promotion and demotion policy: An artefact of AI Operations stating the evidence needed to raise an agent's level and the events that lower it at once. Promotion needs shadow, offline, seeded and stop evidence; demotion is automatic.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Match

Records of observation, seeding and stopping

Match each identifier family of Chapter 19 with the record it names.

Interactive exercise 2 · Scenario

A drill that failed

Choose what the stop holders should conclude from the test record, following AI Threat, Chapter 24, as the chapter reports it.

Teaching plan

15 min	Present Figure 19.1, the observer principle as a set of tools, and the two sets of kinds of independence. Explain why the series uses them as a union.
20 min	Read the observer manifest OBS-HM-0001 and the seeded-fault record SDF-HM-2028-002, in which the agent's own verification reported success and an independent probe said otherwise 61 seconds later.
15 min	Teach stop paths: the agent stop policy, the failed stop-path test at Hansa Mutual where a scheduler was not on the stop-path map, and the timed drill records.
30 min	Exercise: participants specify an observer and plan one seeded fault and one stop drill as a tabletop for a service.
10 min	Debrief: which probes could the agent influence, and who outside the platform holds each stop.

Discussion

The source says the value of the stop-path test record at Hansa Mutual is that it fails. Why is a failed drill worth more than a policy that has never been exercised?
A record that says only that a different model was used proves one kind of independence. Which of the other kinds is hardest to achieve in your organisation, and why?
Seeded errors are also planted in the queues of human reviewers. What agreements with staff and their representatives must exist before such a programme starts?

Written exercise

Exercise · 30 minutes

An observer, a seed and a stop drill on paper

Choose one service function on which an agent acts or will act, in your organisation or at Hansa Mutual. First specify the observer: one probe that measures an outcome and not an endpoint the agent can influence, the failure domain it runs from, a heartbeat and its receiver outside the platform, and the date on which the statement of independence expires. Check it against the six kinds of independence and mark the weakest. Second, plan one silent-failure seed for a pre-production mirror: authorising role, author outside the agent's team, hypothesis, how to stop and how to undo. Third, plan a timed stop drill: who invokes it, by which route outside the agent, the time budget and what the record must show.

Deliverable: A one-page observer specification with independence check, a seeded-fault plan with authorisation and undo, and a stop drill plan with route, holder and time budget.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

Which two kinds of independence does AI Operations name that the list of AI Purple does not contain?
Show the answer
B. Credential and failure domain
AI Purple names model, data, authorship, vendor and time; AI Operations names model, data, authorship, credential, failure domain and time. The sets differ by two items, and the series uses them as a union.
What does the field stop_message_sent_to_agent: false in the stop-path test record signify?
Show the answer
D. That the stop used a route outside the agent; a stop that asks the agent to stop is a request, not a control
The chapter calls it the field to watch: a stop that works by telling the agent to stop is a request, not a control, so the record shows that the stop used mechanisms the agent cannot block.
What does a catch rate from seeded tests carry with it, according to the chapter?
Show the answer
A. A stop rule that lowers or freezes delegation when the rate falls below a floor
All the seeded tests carry a catch rate and a stop rule. The catch rate is the share of seeds found within the target time, and the stop rule lowers or freezes delegation when the rate falls below a floor.