Operators as People: On-Call, Toil, Trust and Wellbeing

All courses / AI Operations / Module 6 / Chapter 25

Learning objectives

After the lesson the participant can:

• Describe what the cited studies measured about on-call duty and alert fatigue, and in which populations
• Explain the moral crumple zone and the three design rules the chapter writes into the charter against it
• Identify which rota and tooling decisions require negotiation or co-determination before they take effect
• Draft an on-call charter with response, rest, load and agent clauses and team-level metrics

Key concepts

Load as a design property

The chapter's first claim: the load on operators follows from the design of the loop, its alerts, rota and agents, and is not a personal failing. A breach of the noise budget is a finding on the alert rules, and the rota that carried the pages is not at fault.

Moral crumple zone

Elish's name for the pattern in which responsibility for a failure of an automated system lands on the nearest human operator. The charter answers it with a delegation record that names the delegator and owner of each agent.

Stop right without penalty

The rule that the stop right is exercised without approval and without penalty: an engineer who stops an agent on suspicion and is wrong has used the right as designed, and the review treats it as a data point for the observer's thresholds.

Response time as a working-time parameter

The chapter's organisational reading of Directive 2003/88/EC and the Court of Justice's judgments on stand-by: a very short response time and frequent call-outs weigh towards stand-by counting as working time, so page targets and page budgets are legal parameters as well as design ones.

On-call charter

The instrument, agreed with employee representatives before it takes effect, that records the response target, rest after a night call-out, the page budget, supervision limits for agents, the stop right and metrics reported for the rota and never the person.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. The chapter found no independent study of operators' wellbeing under autonomous operations agents. How should an organisation act on a question the evidence leaves open?
2. When an agent acts at Level 3 Bounded and something goes wrong at night, who in your organisation would be asked to explain it, and does your delegation record name that person?
3. What would have to be true in your team for an engineer to stop an agent without hesitation and report a near miss without fear?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. According to the chapter, who is at fault when a rota's noise budget is breached?

   1. The engineers on the rota, who failed to acknowledge pages in time
   2. Nobody on the rota; the breach is a finding on the alert rules and the correlation policy
   3. The service desk, which routed too many tickets to the rota
   4. The incident commander, who should have declared an incident sooner
   Show the answer

   B. Nobody on the rota; the breach is a finding on the alert rules and the correlation policy
   The noise budget is a property of a rota and a service, reported for the team. A breach is a finding on the alert rules and the correlation policy, which is the chapter's claim that load is a design property.

2. What does the chapter conclude about whether AI relieves or intensifies the load on operators?

   1. It is settled that correlation and summarisation relieve it in every team
   2. It is settled that supervising agents intensifies it beyond what on-call did
   3. It is irrelevant, because the agent's actions leave the rota entirely
   4. It is open, and the charter decides it team by team
   Show the answer

   D. It is open, and the charter decides it team by team
   The table of usage modes marks the effect on load as direction not measured. The chapter says the question is open and that the design of the charter, team by team, decides which way it goes.

3. Why does a Tier 1 page target of five minutes to acknowledge matter beyond service levels at Hansa Mutual?

   1. A very short response time bears on whether stand-by counts as working time, a question for counsel
   2. DORA Article 19 requires every Tier 1 page of a financial entity to be acknowledged within five minutes
   3. The IT-operations agent is demoted if a person acknowledges later than five minutes
   4. The works agreement forbids any response target shorter than fifteen minutes
   Show the answer

   A. A very short response time bears on whether stand-by counts as working time, a question for counsel
   The chapter reads a response-time target as a working-time parameter: the constraints on stand-by, including response time and frequency of call-outs, are factors in the Court's test, so the question is live and goes to counsel.