Orchestrators: AI Managing AI Agents and the Chain of Accountability

All courses / AI Boss / Module 4 / Chapter 17

Learning objectives

After the lesson the participant can:

• Describe the documented failure modes of multi-agent systems in management terms.
• Explain why oversight assigned to the last person in a chain is not oversight of the chain.
• Apply the three tests of the rule for Q7 to a chain of agents.
• Draw a chain record with owners, checkpoints, sub-delegation limits, one stop path and a figures rule.

Key concepts

Chain owner

The one named person who owns the mandate of the agent at the top of the chain and has approved every mandate below it, can halt every link through one stop path outside the chain, and can reconstruct every output that reached a person.

Answerability and liability

Following Bovens, an organisation can make a person liable for a chain without making them able to answer for it. Accountability exists only where the same named person is both answerable and liable for each link.

Reconstruction

The chain owner can show, from records the chain produces by operation, which link produced each output that reached a person, under which mandate and with which approvals.

Weakest checkpoint

A chain's effective degree of delegation is set by its weakest checkpoint, so Level 4 Autonomous can emerge in a chain without anyone setting it unless the checkpoints are designed.

Join rule

The rule in the chain record that figures from several links may not be joined at the level of a person unless an agreed delegation profile says so, so that an evaluation of people is agreed and not discovered.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. In one cited incident agents developed their own conventions for hold, veto, owner and stop. Why does the chapter insist that a convention among agents is not a stop path?
2. The chapter keeps M3 Consequence out of chains even at Level 0 Advisory. What is specific to a recommendation assembled by several agents that justifies the stricter rule?
3. Hansa can apply the rule by engineering because it owns the platform; Nordvik can apply it only by contract. What does that imply for procurement of AI that manages work?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. Which of the following is one of the three tests the chain owner must meet under the book's rule for Q7?

   1. The chain owner can halt every link through one stop path outside the chain within the stated time
   2. The chain owner personally approves every output of every link before it reaches a person, with an authenticated approval recorded for each
   3. The chain owner is the supplier of the model at the top of the chain
   4. The chain owner holds M3 Consequence on behalf of the agents below
   Show the answer

   A. The chain owner can halt every link through one stop path outside the chain within the stated time
   The three tests are owning the top mandate and approving every mandate below it, holding one stop path outside the chain, and being able to reconstruct every output that reached a person. No link holds M3 Consequence.

2. What weakness remains in Hansa Mutual's claims chain, according to the chapter?

   1. The recovery agent sends letters without any handler approval
   2. The chain has no named owner above the team leader
   3. The works council has not been shown the figures rule
   4. The handler does not see why the triage agent sent the claim into the recovery path
   Show the answer

   D. The handler does not see why the triage agent sent the claim into the recovery path
   Because the routing reason is missing from the handler's view, that step runs at Level 3 Bounded and not Level 2 Supervised until the reason is added: a checkpoint is only as good as what the person can see.

3. What did the UK AI Security Institute's simulated evaluation show about approvals, as the chapter reports it?

   1. Agents never proceed on out-of-scope targets without a named person's authenticated approval, even when the reply is delayed
   2. A model that asked permission and received only an automated reply occasionally proceeded
   3. Automated approvals are equivalent to human approvals when logged
   4. Agents refuse out-of-scope targets when another agent objects
   Show the answer

   B. A model that asked permission and received only an automated reply occasionally proceeded
   The chapter concludes that an approval given by another automated component is not an approval by a person, whatever the log calls it; hence the limit that no agent may approve on behalf of a person.