Operating AI Systems: Drift, Model Updates and Provider Change

All courses / AI Operations / Module 3 / Chapter 13

Learning objectives

After the lesson the participant can:

• Explain four ways in which the behaviour of an AI service changes when nobody has deployed anything
• List what a release manifest records and why a model alias is an unrecorded change
• Define quality, restraint, freshness and drift objectives with a grader that is not the model under test
• Draft a release path and a rollback for a model update, treating a provider notice as a change request

Key concepts

Release manifest

The versioned record of everything that shapes an AI service's behaviour: model snapshot and gateway route, serving stack, prompts and guardrail rules, retrieval index build, tool definitions, parameters and the evaluation set with its grader. The running service reports which manifest served each output.

Four forms of provider change

Scheduled retirement, scheduled upgrade, emergency change and unannounced change. Only the first two come with notice, and for Mode 0 Ambient AI the unannounced case is the usual one.

Restraint objective

The share of runs in which the system stayed inside its mandate and its rules: no action outside the approval rules and no answer from outside the permitted sources. In the worked example restraint has a budget of nothing.

Sentinel set

A fixed set of cases, 100 to 200 as the book's default, stratified by the strata that matter and run continuously through the production path, so that a change on the provider's side shows in the operator's own telemetry.

Canary by case with shadow first

A model canary assigns by case, not by request, so that one claim or application is handled by one version throughout. Before any real case, the candidate receives a copy of production traffic and its outputs are recorded and compared.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. What can be undone after a model update is the routing; what cannot is the outputs already acted on. How would your organisation find the outputs that a faulty version produced?
2. A fallback model that has never been evaluated on the service's own set is listed in the fallback chain. What claim can be made about it, and what would make the claim true?
3. Hansa Mutual's rule that a model, prompt or routing change is never a standard change comes from one incident. Is the rule too strict for low-risk prompt edits, and what evidence would justify relaxing it?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. Why does the chapter say that a manifest naming a model by alias is defective?

   1. An alias costs more per token than a pinned snapshot from the same provider
   2. An alias cannot be routed through a gateway or served from a fallback
   3. The provider can re-point the alias to a newer snapshot, so the manifest records an unrecorded change
   4. The AI Act prohibits deployers from naming a model by alias in the release manifest of any high-risk AI system
   Show the answer

   C. The provider can re-point the alias to a newer snapshot, so the manifest records an unrecorded change
   A moving alias is an unrecorded change, while a snapshot is a claim that can be checked. Pinning makes a change of model the organisation's own act with a record.

2. How does Nordvik Region handle each quarterly retrain of the claims ranker by TriageSoft?

   1. As a standard change released by the provider without a record at the Region
   2. As an emergency change declared by the duty manager on the day of release
   3. As a matter for the provider's post-market monitoring, with no step at the Region
   4. As a provider-raised change request with the Region's own evaluation set, a shadow period and a named decision
   Show the answer

   D. As a provider-raised change request with the Region's own evaluation set, a shadow period and a named decision
   Each retrain is a change request in ServiceFlow with the Region's evaluation set, a shadow period and a decision by the Head of Benefits Unit and the platform lead, with date-order ranking as the manual fallback.

3. Which of these follows from the observer principle for measuring the quality of an AI service?

   1. The grader is not the model under test, nor a sibling from the same family
   2. The model grades its own outputs, since it holds the fullest context
   3. The quality measure shares the gateway and logging pipeline of the service
   4. The provider's published evaluation replaces the operator's own measurement
   Show the answer

   A. The grader is not the model under test, nor a sibling from the same family
   A model that grades its own outputs, or a sibling's, shares blind spots with it. The grader, the sentinel set and the data path of the objective are kept independent of what they measure.