Cloud AI Services versus On-Premises AI: Different Challenges

All courses / AI Platform and Security / Module 3 / Chapter 14

Learning objectives

After the lesson the participant can:

Distinguish the three consumption models and say what provider and customer hold in each.
Compare the cost risk and the model-change risk of a cloud AI service and a self-hosted model.
Apply a workload placement policy and record the rule that produced the placement.
Review a platform against the self-hosting readiness gate or a service against the cloud controls file.

Key concepts

Three consumption models: An AI feature embedded in a SaaS product, a model API or managed AI platform, and a self-hosted open-weight model. Responsibilities shift at each boundary, and control and burden grow from the first to the third.
Utilisation risk: The cost risk of self-hosting: the unit cost of a token depends on how busy the hardware is, and a model serving office-hours users pays for capacity it uses a fraction of the day. Cloud cost risk is volume instead.
Forced migration: A model behind an API is not a fixed artefact: the provider decides when it is retired and with how much notice. A cloud-hosted AI system therefore has a migration and a re-run of the evaluation suite built in.
Workload placement policy: A rule set evaluated at intake. Exclusion rules come first, then capability available only as a service, then steady volume with a passed readiness gate, then a default; every placement is recorded in an AI Decision Record with a review date.
Self-hosting readiness gate: The checks that must pass before a self-hosted model reaches production: licence, no unauthenticated endpoint, patching service levels, weights from the registry only, evaluated safety filters, connected monitoring with on-call, and confirmed capacity with a cloud fallback.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Interactive exercise 1 · Put in order

Placing a workload

Put the steps of the chapter's practical guide for placing a workload in the order it gives them.

Interactive exercise 2 · Sort

Who holds what with a model API?

For a model API or managed AI platform, sort each responsibility by who holds it according to the chapter's shared-responsibility table.

Teaching plan

15 min	The teacher replaces the binary framing with three consumption models and the shared-responsibility table; participants place their own AI uses in the three rows.
25 min	Work through cost, capacity, model change, security, data terms and regulatory roles side by side, with attention to what is a reported figure, a vendor claim or the author's reading.
15 min	Read the placement policy and Hansa Mutual's weighted decision matrix for claims correspondence, and identify which rule decided before the scores did.
40 min	Exercise: groups place one workload, then run either the readiness gate or the cloud controls review on it and report the items they could not evidence.
10 min	Debrief with the hybrid patterns and the public-sector placement rules that must enter the policy as an explicit input.

Discussion

The chapter says published break-even analyses exclude staffing and should be treated as lower bounds. What would an honest three-year cost comparison include in your organisation?
Behaviour of a cloud model can change without any change to the model identifier. How would you notice, and how quickly?
What binds you most to your current AI provider: prompts tuned to one model, fine-tunes held by the provider, managed features or licence conditions?

Written exercise

Exercise · 40 minutes

Place a workload and test the chosen route

Choose a planned AI workload in your organisation, or use Nordvik Region's case-note summariser. Register it with data class, risk class, volume profile and any sector or national rule. Apply the placement rules in order and name the rule that decides. If the result is self-hosting, go through the seven items of the readiness check and mark each as passed with evidence, open with an owner, or unknown. If the result is a cloud service, fill in a controls file: retention, residency, version pinning, deprecation notice, exit assets with a named substitute model, and regulatory position. State what would trigger a re-evaluation of the placement.

Deliverable: A placement record naming the deciding rule and review date, with either a completed readiness checklist or a cloud controls file and its open items.

Your work

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

How does the chapter summarise the difference in cost risk between a cloud AI service and self-hosting?
Show the answer
B. Cloud cost risk is volume, managed at the gateway; self-hosted cost risk is utilisation and staffing
A model API is operating expenditure that scales with use, whereas self-hosted unit cost depends on how busy the hardware is and on the people needed to run it.
Which responsibilities does the customer hold in every consumption model, according to the chapter?
Show the answer
D. Data classification and lawful basis; identity, access and keys; human oversight and acceptable use
These remain with the customer whether the model is an embedded SaaS feature, an API or self-hosted; control and burden over the rest grow from the first to the third row.
In the placement policy, what happens first when a workload's data class is special-category, classified or base-registry?
Show the answer
A. Rule P1 places it self-hosted; if no open-weight model passes evaluation, the answer is redesign or decline
Exclusion rules are applied first. In Hansa Mutual's example rule P1 already routes injury claims with health data to the self-hosted tier before any scoring.