Case Studies and Anti-Patterns

All courses / AI Leader / Module 7 / Chapter 30

Learning objectives

After the lesson the participant can:

• Describe the three mechanisms that recur across the public cases, with one record that anchors each.
• Distinguish what a record states from what its headline suggests, as with the supervisory decision and the company release.
• Match each of the seven anti-patterns to the test it fails, its early signal and its control.
• Review one consequential decision against the anti-pattern catalogue and record partial and unassessed results honestly.

Key concepts

Missing owner of the whole

Every part had an owner and the combination had none. The Dutch inquiry committee found it in a ministry structure, the regulator in a firm whose control reviews had not considered a router malfunction. The test: name the person who would have been asked on the day.

The explanation nobody could give

The caseworker could not see why the model had selected an application, and ministers could be given a reconstruction only with difficulty. These are failures of Explain and of design, not of intelligence: the system gave the person nothing to explain with.

Assurance that outran its evidence

A certification that said 'processes', a dashboard that says logins, a release that gives a figure without a method. The Knight Capital order is the book's clearest public example of an assurance that certifies a process and not the result of a test.

Anti-pattern catalogue

Seven patterns in which every step looks reasonable: AI-theatre board, delegated apology, missing owner, cut layer, green adoption dashboard, briefing monoculture and unreachable accountable person. Each is listed with functions, failing test, anchor, early signal and control.

Thin base of success

Failures produce inquiries, orders and reviews that are public and detailed; a decision that kept a system out of trouble leaves no record. The chapter is the right base for a checklist and the wrong base for a rate, and it makes no claim about rates.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. The chapter says no case shows a leader who was lazy; the records show reasonable steps in structures that made them end badly. How does that change the way you would review a failure in your own organisation?
2. Knight's certification passed through nine people who each believed the firm complied. What statement about AI does your board sign that describes a process and not the result of a test?
3. The Swedish agency withdrew its risk profile before the inspection and the supervisor closed the case without a finding on lawfulness. What can and cannot be concluded from such a record?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. What did the certification signed by Knight Capital's chief executive in March 2012 state, according to the regulator's order?

   1. That the order router had passed a test of its controls that month
   2. That eight senior employees had refused to sub-certify compliance
   3. That the firm had 'processes' in place to comply, not that its controls and procedures complied
   4. That the firm would halt trading automatically after a stated loss
   Show the answer

   C. That the firm had 'processes' in place to comply, not that its controls and procedures complied
   The drafting error was not intentional and the chief executive believed he was certifying compliance. The book reads the case as an assurance that certifies the existence of a process and not the result of a test.

2. Which test does the anti-pattern 'delegated apology' fail, and what is its control?

   1. Reach; the stop holder is named, reachable out of hours and placed outside the platform itself
   2. Name; the apology names the accountable leader and describes the system by what it did
   3. Explain; every figure carries its method and its date
   4. Explain; summaries link to the passages they summarise
   Show the answer

   B. Name; the apology names the accountable leader and describes the system by what it did
   A statement that names the system, the supplier or a junior approver as the actor places responsibility with something that cannot answer. The contrast is the Atlassian letter in which two co-chief executives take the responsibility themselves.

3. Why does the chapter make no claim about how often AI-involved leadership fails?

   1. Because failures leave public records and successes leave none, so no rate follows
   2. Because the cases it analysed were all drawn from one country, one sector and a single year
   3. Because none of the cases it analysed involved a decision taken by a leader
   4. Because the regulators concerned asked that failure rates remain confidential
   Show the answer

   A. Because failures leave public records and successes leave none, so no rate follows
   Inquiries, orders and post-incident reviews follow failures. A decision that kept a system out of trouble produces no record, so the chapter offers a checklist and labels any claim about rates as speculation.