A Reference Architecture for AI in Large Organisations

All courses / AI Platform and Security / Module 1 / Chapter 4

Learning objectives

After the lesson the participant can:

• Describe the five functional layers and the cross-cutting control plane of the reference architecture.
• Explain the role of the AI gateway, the model registry, the evaluation service and the prompt and policy repositories.
• Compare the cloud, on-premises, hybrid and sovereign deployment variants layer by layer.
• Derive a checkable variant of the reference architecture as an overlay on a pinned reference model.

Key concepts

Control plane

The cross-cutting part of the architecture for identity, policy, observability and evidence. It is the architectural expression of policy as code and evidence as code, and it remains the organisation's own in every deployment variant.

AI gateway

A single, mandatory entry point for model calls. It authenticates the calling workload and the human behind it, consults the policy decision point, resolves approved model and prompt versions, applies guardrails, routes the request, enforces budgets and emits a trace.

Model registry

The authoritative list of models the organisation may use, including external foundation models. Entries move through stages such as candidate, evaluated, approved and retired, and pipelines perform every promotion, never a person by hand.

Market overlay

A file per market on top of the shared reference model and the group baseline, recording regime, placement and timing. An overlay may add or tighten controls but never relax the group baseline, and a pipeline check fails any overlay that tries.

Reduced variant

The architecture for an authority that mostly procures. It keeps the gateway, the registry, evaluation as acceptance suites and the control plane, and leaves experiment tracking, training pipelines and feature stores to the provider, with the contract carrying what they would have produced.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. The chapter says that moving the evidence store or the policy repository into a provider's console moves accountability with it. Where do yours sit today?
2. Applications never call models directly in the reference architecture. Which applications in your estate do, and what would it take to route them through a gateway?
3. Why may an overlay tighten but never relax the group baseline, and what would go wrong in a multi-jurisdiction group if it could?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. Which structural decision does the chapter call the single most important one in the reference architecture?

   1. Placing the feature store inside the data foundation instead of the model and feature layer
   2. Running one control plane per legal entity in a group
   3. Keeping the prompt repository apart from the policy repository
   4. Separating the serving and inference layer from the orchestration layer above it
   Show the answer

   D. Separating the serving and inference layer from the orchestration layer above it
   The separation lets the organisation change which model answers a request, or where that model runs, without changing any application.

2. What rule keeps the structure of shared reference model, group baseline and market overlays sound?

   1. Each market keeps a full copy of the platform manifest that it may edit freely
   2. An overlay may add or tighten controls but never relax the group baseline
   3. An overlay may remove the evaluation service where no local law requires it
   4. The group baseline is rewritten whenever a market overlay conflicts with it
   Show the answer

   B. An overlay may add or tighten controls but never relax the group baseline
   The chapter states the rule and adds that a pipeline check fails any overlay that tries to relax the baseline.

3. In the reduced variant for an authority that mostly procures, what happens to experiment tracking, training pipelines and feature stores?

   1. The authority builds them first, because the gateway depends on them
   2. They are replaced by the public algorithm register of the authority
   3. They belong to the provider, and the contract carries what they would have produced
   4. They are moved into the control plane together with the archive
   Show the answer

   C. They belong to the provider, and the contract carries what they would have produced
   The test for each component is whether the organisation's own obligations depend on it; for these, the contract supplies model documentation, evaluation results, release notes and the logs to retain.