From Pilot to Production: Scaling AI Across the Organisation

All courses / AI Platform and Security / Module 5 / Chapter 23

Learning objectives

After the lesson the participant can:

• Explain why pilots built beside the enterprise estate stall on the way to production.
• Describe what a golden path gives a team and what leaving it costs.
• Review a use case against a codified readiness checklist with evidence, exceptions and owners.
• Draft a go or no-go decision record with options, conditions and measurable stop criteria.

Key concepts

Pilot trap

The state in which an organisation can start AI initiatives but not finish them: proofs of concept with no owner, model code in a notebook on a laptop, and no funding. The chapter calls it a failure of architecture, not of talent.

Golden path

A documented, templated, supported way of building a particular kind of AI service with the controls already wired in. A team that leaves it must prove equivalence at the readiness review.

Intake record

A short structured file in the use-case portfolio repository with sponsor, value hypothesis, decision impact, data sources, likely AI Act role and prioritisation scores. It makes the decision to proceed reviewable and seeds the risk classification.

Production readiness review

The gate between a pilot and a service: a codified checklist whose items are verified by evidence in the repository, most of them automatically. An item passes, fails, or passes with a recorded exception that has an owner and an expiry.

Stop criteria

Conditions in the go-live decision record, measurable from the observability configuration verified at the gate, any one of which triggers rollback. A rollback then depends on a threshold and not on a meeting.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. The single failure in Hansa Mutual's gate run belonged to the business owner: nobody had agreed the baseline that defines success. Who would fail that check for your current pilots?
2. Reuse potential is the prioritisation criterion most organisations omit. Which of your proposed use cases would force the platform to add a capability the next five also need?
3. What do your abandoned pilots still hold in the way of data extracts, credentials and indexes, and who is responsible for closing them?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. What is the deeper cause of stalled pilots, according to the chapter?

   1. The models used in pilots are too small for production workloads and must be replaced before the readiness review
   2. Steering committees approve too few pilots each year
   3. Most pilots are built beside the enterprise estate, so controls must be retrofitted at higher cost
   4. Business units lack the talent to write evaluation sets
   Show the answer

   C. Most pilots are built beside the enterprise estate, so controls must be retrofitted at higher cost
   A pilot in a sandbox with its own identity model and logging has no path to production; the question is whether it was built on a path that can carry it there without being rebuilt.

2. What happens at the readiness gate when an item's evidence is missing or older than the last model or prompt change?

   1. The pipeline blocks deployment unless an exception is recorded with an owner and an expiry
   2. The review board discusses the item and records the outcome in minutes
   3. The item is waived automatically for systems that are not high-risk
   4. The deployment proceeds and the gap is listed in the next quarterly report
   Show the answer

   A. The pipeline blocks deployment unless an exception is recorded with an owner and an expiry
   The review is a gate, not a meeting, and an expired exception fails the gate like a missing artefact.

3. Which option did Hansa Mutual choose in the go-live decision AIDR-0034 for the claims triage assistant?

   1. A full go for all claim types and all handlers on the planned date
   2. A conditional go: property claims only, 30 per cent of handlers, for six weeks
   3. No go until both exceptions had been closed
   4. A go for the German market only, with the other markets after a year
   Show the answer

   B. A conditional go: property claims only, 30 per cent of handlers, for six weeks
   The record keeps the rejected options, sets conditions such as closing the two exceptions before expansion, and lists stop criteria that trigger rollback to manual triage.