Frame: Problems, Requirements and Intent

All courses / AI Developer / Module 2 / Chapter 5

Learning objectives

After the lesson the participant can:

• Classify an ambiguity as one of vocabulary, scope or condition and choose its remedy
• Explain why a change to an acceptance criterion is a change to the oracle
• Review an AI-drafted specification for the six failure modes of framing
• Draft a specification record with provenance, assumptions, out-of-scope list and dissent

Key concepts

Three kinds of ambiguity

Vocabulary (a word such as household means different things in the form and in the case system), scope (a request with no edge) and condition (a rule that does not say how it is established). Each needs a different remedy: a glossary entry, an out-of-scope list, a boundary scenario with a named rule owner.

Specification as the source of intent

A specification kept as code in version control, so that it can be tested, versioned with the change and found by an agent. Traceability runs both ways between need, requirement, criterion and test.

The same ambiguity, moved

A generating tool does not stop at a gap in the specification; it fills it with something plausible. The control is an assumptions field and a review step asking which decisions an artefact embodies that no record contains.

Circular verification

The most consequential failure for the book: one assistant drafts requirements and criteria, another generates the code and a third writes the tests, so the loop is closed by machines alone. The control is criteria owned by a person who is not the generator, written before code is generated.

Specification record

The artefact SPC, for example SPC-NR-2030-0001, that links need, requirement, criterion and test, names the owner and the oracle owner, and records what the assistant did. Generation of code begins only after the criteria are committed.

Interactive exercises

Do them here and get feedback at once. Choose a card and then where it belongs, or drag it.

Teaching plan

Discussion

1. A summary of seven interviews shows the majority view. How would you make sure that the case handler who works the hardest cases is still heard in the specification?
2. Assistants tend to agree with the framing they are given. What does that mean for a product owner who asks an assistant whether a proposed scope is right?
3. Nordvik Region sets specify at Level 0 Advisory and Hansa Mutual at Level 1 Assisted. What in the two organisations explains the difference, and which would you choose for your team?

Written exercise

Self-test

Choose an answer to see whether it is right, or open the answer below each question.

1. The form says household and means the applicant's family; the case system uses the same word for everyone at the address. Which kind of ambiguity is this, and what remedy does the chapter give?

   1. Scope ambiguity, remedied by a list of what is out of scope with the reason
   2. Condition ambiguity, remedied by a boundary scenario and a named rule owner
   3. Vocabulary ambiguity, remedied by a glossary entry agreed with the case handlers
   4. Interface ambiguity, remedied by a contract test between the two systems
   Show the answer

   C. Vocabulary ambiguity, remedied by a glossary entry agreed with the case handlers
   The chapter's table gives this as the example of vocabulary ambiguity, with a glossary entry in the specification agreed with the case handlers as the remedy.

2. In the Nordvik Region specification record, how is the baseline for returned applications recorded, and why?

   1. As not measured, because the claim can only be tested against a baseline that exists
   2. As an estimate by the assistant, because no telemetry was available for the form
   3. As the figure from the pilot survey, because nine of twelve developers answered it
   4. As confidential, because the public-access rules prevent its disclosure
   Show the answer

   A. As not measured, because the claim can only be tested against a baseline that exists
   The chapter calls this a deliberate detail: the baseline is recorded as not measured rather than estimated, since a claim of fewer returned applications needs a real baseline to be tested against.

3. Where do the Level 4 prohibitions bite on framing, according to the chapter?

   1. An assistant may never draft a user story from a meeting transcript
   2. No agent changes the acceptance criteria its own output is tested against
   3. A product owner may never use a conversational assistant to build a prototype
   4. A specification may never be stored in the same repository as the code it describes
   Show the answer

   B. No agent changes the acceptance criteria its own output is tested against
   The chapter treats the criteria as the oracle: changing a criterion changes what counts as correct, which falls under the prohibition on a system changing its own test oracles, so a named person makes the change.