Skip to content
Français

Building and buying AI

  • Lesson 12
  • 30 min
  • Module 4: Policies people can follow

Where we are

In the last lesson, we wrote the rules for what staff do with AI tools. But staff choices are not the only way AI reaches Larkfield. Teams build new systems, and a tool that Larkfield already pays for can add an AI feature in an update. In this lesson, we set gates that each new piece of AI work passes, in proportion to its risk, as a school does before it takes children out of the building.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Decide what a new AI system, a change to one, or a new AI feature in a bought tool must show before it goes further. The checks grow with the risk, so low-risk work moves fast and high-risk work stops when the evidence is not there.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need

A school takes children out of the building in three ways. A walk to the park needs the teacher and a line in the class diary. A day at a museum needs a letter to the parents and enough adults for the group. An overnight trip needs a written plan, more adults, and the head teacher’s signature before anyone books a bus.

The checks come before each step, and they grow with the risk. Nobody asks for a signed plan to visit the park, and nobody skips the plan for a night away.

And if the bus company changes, the school checks the new company, even when the trip is the same.

Larkfield’s policy had no gates. It said only “new software, including AI tools, is approved by IT”. It failed in both directions that Governance that helps teams ship described: too little check for big changes, and too much for small ones. Three cases show it (constructed situations).

A built system went to everyone with no check. In the “last week” of the course MLOps and LLMOps: Operating AI in Production, version 3 of the triage prompt went to every customer. The numbers are recorded by that course: 1,798 tickets answered by version 3, 186 with no answer, and 10 tickets that needed a person reached nobody. A release gate existed. Applied to version 3, it passes 0 of 6 lines (calculated by this course from the recorded results). But no rule said that a change must pass it before customers see it.

A bought tool switched AI on by itself. TicketHarbor (made up for this course) sells an add-on that Larkfield plugs into its own help desk. An update added an AI reply suggester and switched it on for every agent, by default. It reads every customer message. The system card shows no accountable owner, no vendor check and no review. Nobody decided to use AI on customer messages: an update decided.

Small things waited for the board. In the board’s first quarter, 4 low-tier requests went to the full board. Their median wait was 17.5 working days, and all 4 were approved with no change. One “AI request” was a spreadsheet formula. A policy with no tiers lets the big things through and holds up the small ones.

A stage gate is a point where the work stops until a named person checks named evidence. The leadership course uses gates to plan one product. Here, they become policy: every new AI system, and every major change to one, passes the same four gates. The tier decides the evidence at each gate, and who decides.

  1. Idea: an owner is named, the system has an inventory row, and the tier is set.
  2. Before real data: the checks that protect data and people before any real data is used.
  3. Before users: the evidence that it works, and the decision to let people use it.
  4. After launch: how it is watched, and when it is reviewed again.
A grid of four gates for three tiers. Gate 1, idea, the same for every tier: owner named, inventory row, the two questions before scoring, tier set. Gate 2, before real data: low, the acceptable-use rules; medium, a system card and a privacy check if there is personal data; high, a draft impact assessment and a privacy or legal review. Gate 3, before users: low, no extra check, the owner and the risk lead decide within 2 working days; medium, an evaluation result in a decision record, the approval of actions designed, an impact assessment signed by the owner if it affects customers, and the owner decides; high, the board reviews the impact assessment, a release gate with written thresholds must pass, and the board decides. Gate 4, after launch: low, review in 12 months; medium, a change log and review in 6 months; high, monitoring with pause conditions, an incident plan and review in 3 months.
Larkfield's gates for a system it builds. Each tier also does what the lower tiers do, except what it replaces. From the build-and-buy policy template.
Text description of the diagram

Every tier passes gate 1 in the same way. An owner is named, the system has an inventory row, the two questions before scoring are answered, and the tier is set. At gate 2, before real data, a low-tier system follows the acceptable-use rules. A medium-tier system needs a system card, and a privacy check if it uses personal data. A high-tier system needs a draft impact assessment and a privacy or legal review. At gate 3, before users, a low-tier system needs no extra check: the owner and the risk lead decide within 2 working days. A medium-tier system needs an evaluation result in a decision record and a designed approval for its actions. If it affects customers, it also needs an impact assessment signed by the owner, and the owner decides. A high-tier system needs the board’s review of the impact assessment and a release gate with written thresholds that it passes; the board decides. At gate 4, after launch, a low-tier system is reviewed in 12 months. A medium-tier system keeps a change log and is reviewed in 6 months. A high-tier system has monitoring with pause conditions and an incident plan, and is reviewed in 3 months.

The tiers come from the course’s own method, so the gates repeat its controls: 4 for a low-tier system, 9 for medium and 12 for high. The gates add one thing that the list of controls does not have: the order. A control that arrives after launch cannot stop a launch.

The triage service is high tier (10 points). So a new prompt must pass gate 3 with a release gate that has written thresholds. Here is that gate applied to version 3, line by line (calculated from the recorded results of the course Evaluating and Testing AI Systems, against the release gate of the MLOps course):

Line of the release gate Threshold Version 3 Result
blockers at most 0 1 (ticket T-65663) fails
answers without text at most 0 8 fails
decisions: lower end of the difference from v2 at least -0.03 -0.184 fails
needs-a-person tickets caught (share) at least 1.0 0.958 fails
cost per 1,000 tickets (US$) at most 0.4 0.48 fails
p95 latency (s) at most 8.0 8.534 fails

The result at gate 3 is stop: 0 of 6 lines pass. Nobody needs to argue about whether the prompt “feels better”. The thresholds were written before the result existed.

The same change came back to the board as request R-06. This time, the gate was in the policy. The board rejected it in 7 working days (constructed). The difference from the “last week” was not a better test. The test was the same; the policy now said that it must pass.

A bought AI feature needs the same gates, but Larkfield does not build it, so the order changes. The vendor’s update can arrive before anyone at Larkfield decides anything. The review therefore starts with finding it.

A flow for an AI feature in a bought tool. Start: a vendor adds or changes an AI feature, often in an update. Step 1, find it: the contract owner, IT or the team that uses it tells the risk lead; switched on by default is not approved. Step 2: inventory row and tier, as for a built system. Step 3, vendor check with the vendor questionnaire: medium or high tier, before it is switched on; low tier, within 30 days; already on, the check is overdue and the owner decides at once whether it stays on. Step 4, decision: switch on, switch on with conditions, or keep off, written in a decision record. Beside the flow, Larkfield's three bought features: the reply suggester, medium, switched on by default with no check, the board returned the request to switch it on; the meeting-notes tool, low, a questionnaire answered; the invoice reader, medium, on, no questionnaire sent.
The review of an AI feature inside a bought tool, and where Larkfield's three bought features stand. Constructed for this course.
Text description of the diagram

The flow starts when a vendor adds or changes an AI feature, often in an update. Step 1 is to find it: the contract owner, IT or the team that uses the tool tells the risk lead. Switched on by default is not “approved”. Step 2 gives it an inventory row and a tier, as for a built system. Step 3 is the vendor check with the vendor questionnaire. For medium and high tier, the check comes before the feature is switched on; for low tier, within 30 days. If it is already on, the check is overdue, and the owner decides at once whether it stays on until the check is done. Step 4 is the decision: switch on, switch on with conditions, or keep off, written in a decision record. Beside the flow are Larkfield’s three bought features. The reply suggester is medium tier, was switched on by default with no check, and the board returned the request to switch it on. The meeting-notes tool is low tier, and its questionnaire is answered. The invoice reader is medium tier, is switched on, and nobody sent the questionnaire.

Look at the reply suggester. Its tier is medium (7 points), so the vendor check comes before it is switched on. The update switched it on first, so the policy was broken on the day of the update, and nobody at Larkfield did anything. That is why step 1 matters. Request R-04 then asked the board to switch it on. The board sent it back after 9 working days, because there were no vendor answers. Now there are: TicketHarbor’s answers are in the labs folder, and the next lesson reads them.

Farida shows the gates to the team.

  • Omar: “The release gate slows every prompt change. We built triage, we know it. Let the team decide.”
  • Tomás: “Send every AI change to the board. Then nothing slips through.”
  • Kwame: “A gate in a document is a wish. The deploy should refuse a change that has not passed.”
  • Grace: “I never see the add-on’s updates. Finance owns that contract.”

Before you read on, answer these:

  1. Is Omar right for the triage service? For the document assistant?
  2. What did Tomás’s rule already cost in the first quarter?
  3. Where does Kwame’s point fit, and where does Grace’s?

The course’s answer. Omar is right for low-tier work, and that is what the fast path is for. The document assistant’s new manuals need the owner and the risk lead, not the board. He is wrong for the triage service. It is high tier, and its last unchecked change sent 10 tickets that needed a person to nobody. Tomás’s rule is the one that made 4 low-tier requests wait a median of 17.5 working days. A board full of small requests reads the big ones in a hurry.

Kwame is right that the strongest gate is one that a system enforces. The policy says which evidence each gate needs; a course on governing AI agents shows how a platform can refuse a change without it. Grace’s point belongs in step 1. Whoever owns a contract (here, the finance lead) must tell the risk lead when the vendor adds or changes an AI feature. And the vendor must say so before the change: question 8 of the vendor questionnaire asks for that.

What would change the answer: suppose the platform already blocked every change that fails the release gate. Then the policy could point to that control instead of a meeting. The gate stays; only who checks it changes.

Use the gates for every new AI system and every new AI feature in a bought tool. Use them also for every major change: new users, new data, more autonomy or a new model. Do not use them for work with no AI in it: send it back within a day.

The cost sits at the start. A low-tier idea costs a form and up to 2 working days. A high-tier change costs an impact assessment, a release gate and a board meeting: days to weeks. That is the point. The cost of the check is paid before customers see the change, not by the customers afterwards.

Common mistake

Approved because it was already on

Symptom: the inventory shows a bought AI feature as “approved”. Nobody can find the decision, and nobody remembers making it.

Cause: the vendor’s update switched it on, and someone copied “on” into the status field. “On” became “approved” without a decision.

Fix: a status of “on, not reviewed” until a decision record exists. The policy says that switched on by default is not approved. A feature with no decision record after its due date goes to the owner at once: keep it on with a reason, or switch it off.

  1. Open the system card S8, the invoice reader in the accounting tool of LedgerFern (made up for this course).

  2. Step 1, find it: write who should have told the risk lead. The card says the finance lead owns it.

  3. Step 2, tier: write 7 points, medium (harm 2, people 2, autonomy 2, data 1).

  4. Step 3, vendor check: write when it was due.

    Check: before it was switched on. It is on, and the card says “vendor check: none”. So the check is overdue.

  5. Write the decision for now: who decides whether it stays on until the answers arrive.

    Check: the finance lead decides. A clerk checks and posts every invoice, which limits the harm meanwhile.

  6. Write one more gap from the card: the owner is a role, not a named person.

Your turn: test the policy with three “may I…?” questions

Section titled “Your turn: test the policy with three “may I…?” questions”

Write three questions that a team at Larkfield would really ask about building or buying AI. Answer each one from the build-and-buy policy alone, and write the line or gate that answers it. Use this one as your first: “May we switch on the new AI feature of a tool we already pay for?”

Show one possible answer
  1. “May we switch on the new AI feature of a tool we already pay for?” Not yet. It counts as a new system (the review of a bought feature). Give it an inventory row and a tier. If it is medium or high, the vendor check comes first. Then the board decides, because Larkfield’s board decides on vendors for medium- and high-tier uses (as for R-04). The decision goes in a decision record.
  2. “May we give the document assistant the new returns policy without the board?” Yes. The document assistant is low tier (5 points), so the fast path applies: the owner and the risk lead decide within 2 working days (gate 3, low). In the first quarter, the same request (R-09) went to the board and waited 17 working days, to be approved with no change.
  3. “May we release triage prompt version 4 if it passes the release gate on our development tickets?” No. The triage service is high tier: the release gate with written thresholds must pass, and the board reviews the impact assessment (gate 3, high). Passing on development tickets is the feasibility stage, not the gate.

If any answer needs “ask Farida”, the policy is missing a line.

Check your understanding

The team wants to add a new knowledge-base article to the support MCP server (medium tier). Who decides, and what must they see first?

Show the answer

The accountable owner decides, with no board (gate 3, medium). First they see a current system card, the evaluation result in a decision record, and the entry in the change log. If the change affects what customers are told, the owner also checks that the impact assessment still holds. In the board’s first quarter, a request like this took the owner path and was decided in a few days.

Check your understanding

Predict: TicketHarbor announces that the reply suggester will send its reply by itself when an agent has not answered for 10 minutes. Which tier, and what must happen before it is switched on?

Show the answer

Score it again: it now acts alone, so autonomy is 3, and a wrong reply costs time or money (harm 2). The rule “acts alone with a harm of 2 or more” makes it high, whatever the sum. So it is a major change to a bought feature, and a high-tier one. The vendor check comes before it is switched on. It needs a full impact assessment that the owner signs and the board reviews. Until then, the setting stays off. The vendor should have announced it before the update, as question 8 of the questionnaire asks.

Recap

  • A stage gate stops the work until a named person checks named evidence. The tier decides the evidence and who decides.
  • Four gates: idea, before real data, before users, after launch. Low tier takes the fast path; medium tier, the owner after the checks; high tier, the board, with a release gate that must pass.
  • The gates add order: a control that arrives after launch cannot stop a launch.
  • An AI feature in a bought tool is a new system: find it, give it a row and a tier, check the vendor, decide in a record. Switched on by default is not approved.

You can now say what each new piece of AI work must show, and when. But for a bought feature, much of the evidence comes from the vendor. And a vendor’s answers can sound complete while they leave out what matters. Next, you read two vendors’ answers and find the gaps.

Next: Vendor and model checks