Setting Up an Effective DevSecMLOps Loop

All courses / AI Platform and Security / Module 2 / Chapter 8

Learning objectives

After the lesson the participant can:

• Name the artefacts that shape an AI system's behaviour and say where each is versioned.
• Explain why a release is the set of versions evaluated together and why a manifest records versions, not aliases.
• Design a catalogue entry for a shared skill with owner, digest, permitted tools and consumers.
• Map elements of the delivery loop to the ISO/IEC 27002:2022 controls they evidence.

Key concepts

DevSecMLOps loop

The delivery loop in which development, security, machine learning and operations share one loop instead of four handovers. Every stage has an owner, a versioned artefact and a gate the next stage trusts.

Revision management

The backbone of the loop: every artefact that can change the system's behaviour has a version, an owner and a place in version control or in a registry with immutable versions. It decides whether a team can say within an hour what is running.

Release manifest

The signed record of the exact set of artefact versions that passed evaluation together, pinned by digest. If any one changes, the evaluated combination no longer exists and the change needs its own evaluation.

Alias

A mutable reference such as production, used for rollout and rollback in prompt and model registries. Aliases are pointers, not versions, so a release manifest records the version an alias resolved to.

Governed catalogue

One catalogue, owned by the platform, that carries every skill, agent definition, shared prompt and connector to users. Its entry is the unit of governance: owner, version, digest, review and evaluation status, permitted tools and data, distribution and deprecation.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. Could your team say within an hour exactly what is running, what changed since yesterday and how to go back? Which artefact would be hardest to name?
2. The chapter requires that an agent-authored change still needs a second human approval and that agents never hold release credentials. Which of these rules would meet most resistance in your organisation, and why?
3. Where do skills, agent definitions and connector lists live in your organisation today, and who would own a catalogue of them?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. How does the chapter define a release of an AI system?

   1. The model version that most recently passed the evaluation gate, together with its model card and training manifest
   2. The exact set of artefact versions that passed evaluation together, pinned by digest and signed
   3. The commit on the default branch that the production alias points to
   4. The deployment record written by the platform team after admission
   Show the answer

   B. The exact set of artefact versions that passed evaluation together, pinned by digest and signed
   If the prompt, the index, the guardrail policy or a skill changes, the evaluated combination no longer exists and the change needs its own evaluation.

2. What does the chapter conclude from the reported figures on the security of generated code and on secrets leaked in agent co-authored commits?

   1. Coding assistants should be banned from every repository that holds prompts, policies or evaluation sets
   2. Agent-authored changes should merge without review once the test suite passes
   3. Generated changes pass the same gates as human ones and still need a second human approval
   4. Only the platform team may use coding assistants, with release credentials
   Show the answer

   C. Generated changes pass the same gates as human ones and still need a second human approval
   Commits by an assistant or agent are identified in their metadata, agents never hold release credentials, and an agent-authored change still needs a second human approval.

3. In the practical guide for the repository and revision model, what is the bypass list of the ruleset limited to?

   1. A break-glass group of two named people whose use of it raises an alert and an incident record
   2. All repository administrators, reviewed once a year by the security architect
   3. The release pipeline and the coding agents that open pull requests
   4. The model team during training runs, for the life of the training branch
   Show the answer

   A. A break-glass group of two named people whose use of it raises an alert and an incident record
   Roles on a bypass list can override the rules, so the list is kept empty except for the break-glass group, and its use is alerted and recorded.