Roll it out to an organisation
- Lesson 21
- 40 min
- Module 7: Build, buy and roll out
Where we are
In the last lesson, we compared our gateway with four products and found that the features do not decide alone. But a gateway that nobody moves to protects nothing, and one that blocks everyone on day one makes people go around it. In this lesson, we plan Larkfield's rollout phase by phase, we measure adoption and friction, and we decide build or buy. We start with a new expense system.
Lesson 21 of 21 · Governing AI Agents: Gateways, Registries and ControlsRoll it out to an organisation
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
Decide whether your organisation needs its own gateway at all. Then roll the platform out in phases: registration first, new agents through the gateway, existing apps one by one, then close the side doors. Give exceptions an owner and an end date, and measure two numbers: adoption and friction.
Before you startWhat to know and to have ready
What you need to know
- The comparison sheet and the must-haves, from Gateways compared.
- The provider bill and discovery, from Find unregistered AI.
- How an app moves to the gateway: its base URL and its own credential, from One identity per agent.
What you need on your computer
- The project as the last lesson left it (194 tests pass), in
~/projects/ai-estate, with the virtual environment active. - The estate running (
python -m estate up), with the registrations and credentials of the earlier modules.
Tested on macOS (Apple silicon) with Python 3.12.13. The Windows and Linux commands are written, not run. The month’s numbers are replayed traffic with a simulated bill and egress log.
A new expense system
Section titled “A new expense system”A company replaces paper expense forms with an app. On Monday, the finance team does not throw the paper forms away. New employees start with the app from their first day. Then each department moves in turn, with a person from finance next to them for a week. One department has a trip abroad that the app cannot handle yet. It gets written permission to use paper until the end of the month, signed and dated.
Finance counts two things every week. How many claims came through the app? And how many people called the help desk because the app refused a claim? When almost every claim comes through the app, finance stops accepting paper.
The problem: day one, two ways to fail
Section titled “The problem: day one, two ways to fail”There are two ways to get a rollout wrong, and Larkfield’s replayed month shows the size of both.
Block everything on day one. In the replayed month, the egress rule came on 12 September, after the last app had moved. Before each move, the apps sent 12,977 requests straight to the provider. If the rule had come on 1 September, every one of those requests would have been blocked. The triage service would have sent its tickets to people, and the workflow would have stopped. This is a what-if, computed from the month’s records. People who see their work break on day one learn one lesson: go around the platform.
Never close the side doors. The digest script ran on 7 September with a personal key, to a provider without a contract. Nothing stopped it until the egress rule. Priya’s personal key made 202 judge calls that nobody saw until discovery. A platform that is only optional is a second way to call models, not a control.
When not to run your own gateway
Section titled “When not to run your own gateway”First, the question that comes before any rollout: does this organisation need a gateway of its own?
A gateway pays when several teams share models, providers and budgets. Then one place for keys, limits and records saves each team from building its own, differently. One team with one provider and one or two apps is a different case. The provider’s own project keys, quotas and cost alerts may be enough; check what your provider offers. A managed gateway is the next step. Even then, register every app: the registry is about knowing what runs, not about traffic.
Text description of the diagram
Start with the number of teams that share models and budgets. With one team, ask how many providers and apps it has. With one provider and one or two apps, the provider’s own project keys, quotas and cost alerts may be enough. Register every app anyway. With more, use a managed or an open-source gateway. With several teams, ask whether a must-have exists that no product has. If not, ask whether the requests may pass through a vendor’s service. If yes, use a managed gateway or a self-hosted product. If no, use a self-hosted product. Is a must-have missing everywhere, and can you staff the work for years? Then build that part yourself, next to a product if you can. Every gateway path ends at the same box: keep the registry, the catalog, the policy and the evidence as your own files.
Larkfield has four teams and its own hardware for gemma3:4b. It has a contract with one provider in two regions; the second region is simulated in this course. Its model calls through the gateway cost US$21.31 in the replayed month. The workflow spent US$14.28 and triage US$6.48. The judge spent US$0.42, the document assistant US$0.13. Money is not why Larkfield needs a platform. The digest, the refunds and the shared keys are.
The idea: phases, exceptions, adoption and friction
Section titled “The idea: phases, exceptions, adoption and friction”A phased rollout moves the organisation in steps that each have a measure. Larkfield’s plan is in docs/rollout-plan.md:
| Phase | What happens | What we measure |
|---|---|---|
| 0. Inventory | every AI use: the known apps, the provider bills, the egress log, a scan of the code | findings outside the registry |
| 1. Registration first | every app fills in the form; high risk also gets a review | registrations; time from request to active |
| 2. New agents through the gateway | every new agent gets its own credentials and calls only the gateway | share of new agents through the gateway |
| 3. Move existing apps one by one | each changes its base URL and credential | provider calls through the gateway (the bill) |
| 4. Close the side doors | revoke the old keys; the egress rule lets only the gateway out | calls around the gateway; blocked connections |
| 5. Run it | budgets, drills, reviews, the monthly report, exceptions | friction: refusals per agent, and why |
Registration comes first because it costs a team almost nothing and changes no traffic. New agents come before old apps because they have no old habits to move.
An exception is written permission to skip one rule, for one agent, until a date, with a reason and an approver. It is the paper form for the trip abroad. Without an exceptions process, people make their own exceptions in secret.
Adoption is the share of provider calls that came through the gateway. Count it from the provider’s bill, not from the gateway’s records: the gateway cannot count the calls that go around it. Friction is the share of each agent’s calls that the gateway refused, and why. Adoption shows whether people use the platform. Friction shows what it costs them.
Text description of the diagram
Phase 0, inventory: from 1 to 7 September, every app called the provider straight; the measure is what is outside the registry. Phase 1, register first: on 8 September, every app was registered, 7 forms; the measure is the number of registrations and the time to active. Phase 2, new agents only through the gateway, each with its own credentials; the measure is the share of new agents through the gateway. Phase 3 moves the apps one by one. Triage moved on the 8th, documents on the 9th, the workflow on the 10th and the judge on the 11th. The bill showed 75.9% of the month’s calls through the gateway. Phase 4, close the side doors: the egress rule on 12 September blocked the digest 3 times. The measure is calls around the gateway and blocked connections. Phase 5, run it: the remap gate on the 16th, the outage on the 17th, the runaway on the 22nd. The friction was 19 normal calls refused on the 22nd.
A tiny worked example: adoption from the bill
Section titled “A tiny worked example: adoption from the bill”-
Copy this lesson’s files from the snapshot
end-of-m07, then run the tests:Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m07cp $S/docs/rollout-plan.md docs/cp $S/estate/adoption.py estate/cp $S/estate/cli/adoption.py estate/cli/cp $S/tests/test_adoption.py tests/cp $S/README.md .python -m pytest -qTerminal window $S = "..\nextia-learning-labs\C32\snapshots\end-of-m07"Copy-Item $S\docs\rollout-plan.md docs\Copy-Item $S\estate\adoption.py estate\Copy-Item $S\estate\cli\adoption.py estate\cli\Copy-Item $S\tests\test_adoption.py tests\Copy-Item $S\README.md .python -m pytest -qTerminal window S=../nextia-learning-labs/C32/snapshots/end-of-m07cp $S/docs/rollout-plan.md docs/cp $S/estate/adoption.py estate/cp $S/estate/cli/adoption.py estate/cli/cp $S/tests/test_adoption.py tests/cp $S/README.md .python -m pytest -qThe last line says
195 passed, 1 deselected. The new test counts calls through and around the gateway on a small bill. -
Open
evidence/azure-bill-2026-09.csv, the replayed month’s simulated bill. It has one line per key and deployment. Add up the lines by key:Key on the bill Whose key ( config/provider-keys.yaml)Requests az-key-9(6 lines, both regions)the gateway’s own key 19,563 + 2,677 + 18,391 + 55 + 13 + 55 = 40,754 az-key-1(3 lines)the service team’s old shared key 5,880 + 769 + 5,793 = 12,442 az-key-2the data team’s old key 324 az-key-3Priya’s personal key: not in the inventory 202 around the gateway 12,442 + 324 + 202 = 12,968 -
Let the command do the same:
Terminal window python -m estate adoption --month 2026-09 --bill evidence/azure-bill-2026-09.csvProvider calls through the gateway: 40,754 of 53,722 (75.9%); around it: 12,968.Friction: the share of each agent's calls that the gateway refused.
The command finds the same numbers as your table. It reads each key’s status in provider-keys.yaml: only a key with status: gateway counts as “through”. So an unknown key, like Priya’s, counts as “around”, which is the safe side. The friction part is empty: your own records have no calls in September. The month’s records are in the report of Module 6.
75.9% is the share for the whole month, and the month is mostly the days after the moves. Per app, the share depends on the day that the app moved:
Text description of the chart
Four bars. ticket-triage, moved on 8 September: 76.1% (37,226 of 48,899 requests). doc-assistant, moved on the 9th: 73%. support-workflow, moved on the 10th: 69.7%. release-judge, moved on the 11th: 33.3%, because two of its three evaluations ran before the move, with Priya’s personal key.
Friction: measure what you refuse
Section titled “Friction: measure what you refuse”Now measure friction on your own records. Make one refusal on purpose: the triage service asks for a capability that its registration does not list, as in Module 4.
-
Send the request:
Terminal window TOKEN=$(python -m estate token ticket-triage)curl -s -X POST http://127.0.0.1:8322/v1/chat/completions \-H 'Content-Type: application/json' -H "Authorization: Bearer $TOKEN" \-d '{"model": "strong-reasoning", "messages": [{"role": "user", "content": "Hello"}]}'Terminal window $TOKEN = python -m estate token ticket-triagecurl.exe -s -X POST http://127.0.0.1:8322/v1/chat/completions `-H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" `-d '{\"model\": \"strong-reasoning\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}'Terminal window TOKEN=$(python -m estate token ticket-triage)curl -s -X POST http://127.0.0.1:8322/v1/chat/completions \-H 'Content-Type: application/json' -H "Authorization: Bearer $TOKEN" \-d '{"model": "strong-reasoning", "messages": [{"role": "user", "content": "Hello"}]}'{"error":{"code":"capability_not_allowed","message":"ticket-triage's registration does not list strong-reasoning.","type":"gateway","check":"capability"}} -
Run the report without a month, so that it reads all your records:
Terminal window python -m estate adoption --bill evidence/azure-bill-2026-09.csvOur run, on a fresh state with three triage tickets and two document questions before the refusal:
Provider calls through the gateway: 40,754 of 53,722 (75.9%); around it: 12,968.Friction: the share of each agent's calls that the gateway refused.ticket-triage 1 of 7 refused (14.3%): capability 1doc-assistant 0 of 2 refused (0.0%)Your lines are longer and your numbers differ: your records hold every call that you made in this course.
Each friction line names the agent, the share refused and the check that refused. capability 1 says what to change: the registration, or the app. A high share is a signal to talk to the team, not to blame it.
Two limits of this number. First, it counts model calls only. A suspended agent gets no token at all, and a tool call refused by the tool gateway is in another table, so neither appears here. Second, a refusal is not always friction. In the replayed month, the workflow had 35,182 refused calls. Of them, 35,163 were the runaway loop, which the limits stopped on purpose. Only 19 were normal work: 19 of 1,787 normal workflow calls (1.1%), refused by the daily quota after the loop had used it up. Those tasks went to people. So read the “why”, and separate the controls that worked from the work that they blocked.
Larkfield: the order of the moves, and one exception
Section titled “Larkfield: the order of the moves, and one exception”Kwame moved the apps in an order with reasons. Triage first: the most calls, medium risk, and a move that is only a base URL and a credential. The document assistant next: low risk, a small team. The workflow third: high risk, with approvals; its owner Amira had seen triage move first. The judge last: its pin and its evaluation needed Priya’s time.
Mei asks for an exception. The document assistant’s second tenant cannot leave its old key until its tenant tests pass. Kwame records it:
python -m estate exception add doc-assistant --rule "the gateway-only egress rule" --until 2026-11-30 --reason "its second tenant moves after the tenant tests pass"Exception 1: doc-assistant may skip the gateway-only egress rule until 2026-11-30.The course ran this on a fresh state. If you kept the exception from Evidence for governance, yours is exception 2, and the list below has both.
python -m estate exception list 1 doc-assistant the gateway-only egress rule until 2026-11-30: its second tenant moves after the tenant tests pass (approved by usr-kwame)The exception is a record: it names the agent, the rule, the reason, the end date and the approver, and the monthly report lists it. It does not open the network by itself. The network team adds the matching rule, and removes it on the date. The end date is the important field: an exception without one becomes the new rule.
Common mistake
75.9% adoption, but of which bills?
Symptom: the monthly report says that 75.9% of provider calls came through the gateway. Kwame reports it as “three calls in four, at every provider”.
Cause: the command counts only the bills that you give it. Add the Fernway expense claim:
python -m estate adoption --month 2026-09 --bill evidence/azure-bill-2026-09.csv --bill evidence/fernway-expense-2026-09.csvProvider calls through the gateway: 40,754 of 53,723 (75.9%); around it: 12,969.Friction: the share of each agent's calls that the gateway refused.One more call around the gateway: the digest’s call to Fernway. The share hardly moves, but the risk does: that call sent customer messages to a provider without a contract. And some AI use is on no bill at all. The document assistant’s embeddings run on the CPU of its own machine. The three blocked digest runs are only in the egress log.
Diagnose: list every provider that the inventory, the expense claims and the egress log name. Check that each one has a bill in the command.
Recover: pass every bill and expense export with --bill. Report the share together with the list of findings from estate discover, never alone.
Your turn: build or buy, for Larkfield and for a larger company
Section titled “Your turn: build or buy, for Larkfield and for a larger company”This is the independent task of the module. Write a recommendation of about one page, for two companies:
- Larkfield: the company of this course, its four teams, its one provider in two regions, its six systems and its platform owner, Kwame.
- A larger company: 40 teams, three providers, a security team that collects every trace, and data that some countries require to stay in the country.
For each company, answer four questions, with reasons from the comparison sheet, the rollout and the month’s numbers:
- Does it need its own gateway, or are the provider’s own controls enough? Use the decision flow.
- Build, adopt an open-source product, or buy a managed one? Name the must-haves that decide, and run
score_gateways.pywith them. If you deleted it, create it again from Gateways compared. - What stays the company’s own, whatever the product?
- In which order does it roll out, and which two numbers does it watch?
One possible answer
Larkfield. Four teams share one provider and budgets, so a gateway pays: the provider’s controls alone would not stop the next digest. Larkfield should not run the course’s gateway in production: it is a teaching project with no security review, and Kwame alone would maintain it. With Larkfield’s must-haves, only LiteLLM and our gateway passed. A self-hosted LiteLLM means running PostgreSQL and Redis. Azure API Management fits Larkfield’s one provider, if Larkfield accepts token quotas per capability instead of money budgets. It then redoes them after every remap. Either way, Larkfield keeps its registrations, catalog, remap gate, approval policy and evidence as its own files, and keeps the tool gateway’s approval grants. It rolls out in the order of this lesson and watches the bill’s adoption share and the friction per agent.
The larger company. Many teams and providers: a gateway is needed. OpenTelemetry and guardrails are must-haves, so only LiteLLM passed with the sheet. Its guardrails per team are in the enterprise edition, so it needs a licence. The data-residency rule asks for a self-hosted gateway in each country, or a managed one with regional processing: check the current documentation. No product holds the registry with risk tiers and review dates. So the company builds that part as its control plane, and a platform team runs it. It moves new agents first, one business unit at a time, with exceptions that end, and it reads friction per team every week.
When to use each part, and what it costs
Section titled “When to use each part, and what it costs”Registration first costs each team a form and a review, and changes no traffic. Start there even if the gateway is months away.
Moving apps one by one costs time: in the replayed month one app a day, in a real company weeks each. It keeps every incident small and teaches each team from the one before.
Closing the side doors costs a network rule and the revoked keys. Do it only when adoption is near complete and the exceptions are written down. Otherwise you break work and teach people to go around.
Exceptions cost a record and a review on their end date. Without the end date, they cost the rule itself.
Adoption and friction cost one command a month. Report them together. High adoption with high friction means that people are forced, not helped.
Knowledge check
Section titled “Knowledge check”Check your understanding
The friction line for support-workflow says that 35,182 of its calls were refused in September. Is the platform too strict for Amira's team?
Show the answer
No: read the “why” first. 35,163 of those refusals were the runaway loop on 22 September, stopped by the rate limit and the daily quota, as designed. Only 19 were normal work, refused by the quota after the loop had used it up: 19 of 1,787 normal workflow calls, 1.1%. Those 19 are the real friction. The fix is about the loop, for example a step limit that catches it earlier, not a weaker limit.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
A team asks to keep its old provider key 'for a while' because its app is not ready to move. What does a good exception record, and why is the end date the most important field?
Show the answer
It records the agent, the rule it may skip, the reason, the approver and the end date, as estate exception add does. The monthly report lists it. Someone must also change the real control, for example the egress rule, and change it back on the date. Without an end date, the exception never ends: it becomes a second, silent rule, and the side door stays open.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Larkfield picks a gateway product. In two years, it wants to change to another one. What would the change cost, and what in the design keeps that cost low?
Show the answer
The cost: each app that calls models changes its base URL and gets a new credential. The platform writes a new translation of its files into the new product. That means the catalog’s mappings and fallbacks, each registration’s keys, budgets and limits, the guardrails and the tool rules. It runs each consumer’s evaluation through the new product before it moves, as the remap gate does. It exports the old usage rows, so that the year’s evidence continues. Then it moves the apps one by one, with exceptions for the late ones, and repeats the kill-switch drill. What keeps it low: the apps speak the OpenAI-compatible API and ask for capability names, never a product’s route or a model. Larkfield’s own files are the source of truth, and the product holds only a translation. And the usage evidence is in Larkfield’s own format.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- One team with one provider may need only the provider’s own controls; several teams sharing models and budgets need a gateway. Build only what no product has, and only if you can run it for years.
- Roll out in phases: inventory, registration, new agents, apps one by one, then close the side doors.
- An exception names the agent, the rule, the reason, the approver and an end date.
- Adoption comes from the bills: 40,754 of 53,722 calls (75.9%) in the replayed month. Friction comes from the refusals and their reasons: 19 normal workflow calls, not 35,182.
- The cost of changing product stays low when the apps ask for capabilities and the knowledge stays in your own files.
Your project now matches the snapshot end-of-m07, the course’s end: 195 tests pass. You can build the platform, compare it, roll it out and measure it. But every number in this course came from Larkfield’s own recordings. Next, in the case studies, you build a capability catalog from public evaluation results that anyone can check.