Five agents and nobody knows
- Lesson 1
- 30 min
- Module 1: Why agents need platform controls
Where we are
In the agents course, we built one support workflow and kept it safe: explicit state, limits, and a person's approval before every change. But one safe agent is not a safe organisation. Larkfield now runs several AI systems, each built by a different team, with its own keys, and one of them nobody knew about. In this lesson, we look at what a company cannot answer when nobody sees all of its AI, the way a family cannot read a shared card statement.
Lesson 1 of 21 · Governing AI Agents: Gateways, Registries and ControlsFive agents and nobody knows
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
See what an organisation cannot answer about its own AI when every team calls model providers its own way. You meet Larkfield’s AI estate and read its only record, the provider’s bill. Then you write down the questions that nobody at Larkfield can answer today.
Before you startWhat to know and to have ready
What you need to know
- What an agent is: a program that calls a model, uses tools and acts, from AI Agents and Workflow Orchestration. This course uses that course’s support workflow.
- What a model provider and an API key are: you send a request with a key, and the provider answers and bills the key’s account. Building Reliable Applications with LLM APIs explains both.
What you need on your computer
- Nothing for the first part of the lesson. At the end, you download the course project. You need Git, or a browser to download a ZIP file.
You need no account, no key and no payment in this course. Every model answer comes from a recording, as the last lesson of this module shows.
A shared credit card
Section titled “A shared credit card”Think of a family that shares one credit card. Four people have a copy of it. At the end of the month, the statement arrives. It lists shops, dates and amounts, but not who bought what. Most lines look normal. One line is from a shop that nobody in the family knows, for a small amount.
Now the family has questions. Who bought this? Is somebody’s copy of the card lost? One family member moved out last month: does that person still have a copy? Nobody can answer from the statement, because the statement records the card, not the person. And the small amount is not the real worry. The worry is that the card works in places that nobody chose.
A company that uses AI can be in the same position. Its “card” is a provider key, and its “statement” is the provider’s bill.
The problem: a script that nobody knew about
Section titled “The problem: a script that nobody knew about”This is a constructed practice case. Larkfield, its people and the provider Fernway are made up for the course.
Larkfield sells home and garden products and runs a help desk. A few months ago, Lee, an analyst on the service team, wanted to save an hour every Monday. Lee wrote a small script, weekly_digest.py. It reads the week’s customer messages, sends them to a model, and prints a summary for the Monday meeting. The script runs on a build machine, build-runner-3, every Monday at 06:00.
Lee used Fernway, a provider that Lee already had an account with, and a key from that personal account. These two lines of the script show it:
FERNWAY_URL = os.environ.get("FERNWAY_URL", "http://127.0.0.1:8320/fernway/v1")FERNWAY_KEY = "sim-fw-personal-7Q2K9XW4" # my own account, for nowIn the course, the address is the provider simulator on your computer, and the key is a practice key. In the story, they are a real provider and a real personal key.
Lee left Larkfield on 31 August. The script did not leave. It kept running every Monday, with Lee’s key, from a machine that nobody connected with AI. Each run sent 40 customer messages to Fernway, a provider that Larkfield has no contract with. So nobody had agreed how Fernway may store or use those messages.
The first sign was a line in the expenses:
month,provider,key_id,deployment,requests,tokens_in,tokens_out,cost_usd2026-09,fernway,fw-7Q2K,fernway-swift,1,2764,152,0.001349One request, 2,764 tokens in, 152 out, US$0.0013. The amount is almost nothing. The data is the problem: customer messages, with their complaints and order numbers, went to a company that Larkfield never chose.
Larkfield even has an AI policy, approved by management. The script broke at least four of its twelve rules. The rules ask for an inventory of all AI, approved providers only, customer data only under a contract, and no keys in code. The policy is a document. Nothing in Larkfield’s systems read it, so nothing stopped the script.
Larkfield’s AI estate today
Section titled “Larkfield’s AI estate today”The digest is not the only AI at Larkfield. Earlier courses built several AI systems for the company, and each team now runs its own. Together, the course calls them Larkfield’s AI estate. The name means every AI system that the company runs, like the buildings of an estate that one owner looks after.
| App | Owner | From | Model it calls | Key |
|---|---|---|---|---|
| Ticket triage | Amira (service team) | the MLOps course | chat-small, and embed-small for search | az-key-1, the service team’s shared key |
| Support workflow | Amira | a pilot of the agents course workflow | chat-strong | az-key-1, the same shared key |
| Document assistant | Mei (data team) | the capstone course | chat-small | az-key-2, the data team’s key |
| Release judge | Priya (evaluation) | the evaluation course | chat-strong | az-key-3, Priya’s personal key |
| Support MCP server | the platform team | the MCP course | none: it gives agents tools | none |
| Weekly digest | Lee, who left | nobody registered it | fernway-swift | fw-7Q2K, Lee’s personal key |
In September, the support workflow ran as a pilot: an early version of the workflow that the agents course finishes in October.
Five of these systems call models: they are the five agents of this lesson’s title. Kwame, Larkfield’s solution architect, knew about three of them when he started, and about the MCP server. He did not know about the release judge or the digest. Priya’s judge is a normal, useful tool: it checks the support replies before each release. But it calls the provider from Priya’s laptop, with a key that she made for herself.
Text description of the diagram
On the left are the apps and the machines they run on. Ticket triage runs on triage-svc-01 and the support workflow on workflow-01; Amira owns both. The document assistant runs on docs-01 and belongs to Mei. The release judge runs on priya-laptop. The weekly digest runs on build-runner-3, and its owner, Lee, has left; it is drawn in red.
In the middle are the keys. Triage and the workflow share az-key-1, the service team’s key. The document assistant uses az-key-2, the data team’s key. The judge uses az-key-3, Priya’s personal key. The digest uses fw-7Q2K, a key from a personal account, in red.
On the right are the providers. The three Azure keys reach Azure in the East US 2 region, where Larkfield has a contract. fw-7Q2K reaches Fernway, where Larkfield has no contract. The support MCP server makes no model calls and has no key. Both providers send a bill with one line per key and model. The bill has no app name, no owner and no purpose.
Each app made sense when its team built it. The problem is the sum: five paths to two providers, four keys written in code, and no single place that sees a call.
A tiny worked example: read the bill
Section titled “A tiny worked example: read the bill”The provider’s bill is the only record that covers every app. So let us try to answer three simple questions from it. These are the bill lines of the replayed month that the course uses in Module 6. They cover the first days of September, when every app still called the provider with its own key. The traffic is replayed from real recordings; the bill itself is simulated.
| Key | Requests | Cost |
|---|---|---|
az-key-1 |
12,442 | US$6.48 |
az-key-2 |
324 | US$0.05 |
az-key-3 |
202 | US$0.83 |
fw-7Q2K |
1 | US$0.0013 |
Question 1: what did the support workflow cost? The workflow uses az-key-1. But triage uses az-key-1 too. The bill splits each key by model, and in this estate only the workflow uses chat-strong. So you can guess: the chat-strong line of az-key-1 is the workflow. That guess breaks on the day the triage service also calls chat-strong. The bill cannot tell two apps on one key apart.
Question 2: whose is az-key-3? The bill names an account, not a person. Somebody must search the provider’s console, find that Priya made the key, and then ask her what calls it. The answer comes from a conversation, not from a record.
Question 3: which app sent customer data to a provider without a contract? The bill shows a key at Fernway. It does not show the app, the machine, the data or the owner. Larkfield learns that something happened, after it happened.
So the bill answers “how much, per key”. Every question that governance asks is about apps, owners and data, and the bill knows none of them.
Cost is not risk
Section titled “Cost is not risk”It is tempting to look at the biggest number first. The chart shows why that is the wrong place to look.
Text description of the chart
Four bars, one per key. az-key-1, the service team’s shared key for triage and the workflow, is the highest at US$6.48 for 12,442 requests. az-key-3, Priya’s personal key, is US$0.83 for 202 requests. az-key-2, the data team’s key, is US$0.05. fw-7Q2K, the personal Fernway key, is US$0.0013 for 1 request: the smallest bar by far, and the only one that sent customer messages to a provider without a contract.
The Fernway line is about 4,800 times smaller than the shared key’s line. But it is the one that breaks the company’s own rules. A budget alert would never fire on it. Risk depends on what data goes where, and who decided, not on the amount. So the controls in this course start with knowing what runs, and only then look at money.
The course uses the prices of its recordings, per million tokens in and out: chat-small US$0.10 and US$0.50, chat-strong US$2 and US$10. Fernway’s price is made up. Prices change: check the provider’s current price page.
What Larkfield cannot answer today
Section titled “What Larkfield cannot answer today”Kwame becomes the owner of Larkfield’s AI platform. His first task is to write down what the company cannot answer. Sam, a support agent who is new to all of this, asks the simplest question first: “Can’t we just ask every team?” Kwame tried. The teams told him about four systems. The other two came from a bill and a key that nobody had mentioned.
| Question | What Larkfield has today | Why it is not enough |
|---|---|---|
| What AI do we run? | Kwame’s list, from asking teams | It missed the judge and the digest. A list that people keep by hand is out of date the day someone writes a new script. |
| Who owns each system? | Each team’s memory | The digest’s owner left. Nobody knew that it was still running. |
| Which models and providers does each one call? | Each app’s code | Five codebases to read, and one of them nobody knew to read. |
| What data goes to which provider? | Nothing | Customer messages went to Fernway, and no record says so. |
| What does each app cost? | The bill, per key | Two apps share az-key-1. A personal key has its own account. |
| Who holds which key? | The provider’s console | Keys are in code and on a laptop. When Lee left, nobody stopped the key. |
| Which systems act, and which only suggest? | The agents course’s design notes | The support workflow writes refunds, after a person approves. Nothing outside the app records that it can. |
| Can we stop one system now? | Delete its key | Deleting az-key-1 stops the triage service and the workflow together. |
| Did a model change behind an app? | Each app’s settings | Nothing records which model version answered each call. |
Every row has the same cause. There is no single place where every AI call passes, so there is no place that can check a call, record it, or stop it. Each team built a good app. Nobody built the platform between the apps and the providers.
Tomás, an AI engineer on the platform team, has a quick fix: “Let’s block Fernway on the firewall.” Kwame agrees that it helps for Fernway. But the next script will use another provider, or another personal key at Azure. Blocking one name answers one row of the table. The course answers all of them.
What this course builds
Section titled “What this course builds”Here is how the course answers each row. You build each part on your computer, in Larkfield’s project.
- Module 2 gives every app a capability to ask for, such as “fast-small” or “strong-reasoning”, instead of a model name. The platform decides which model serves it, and records which model answered.
- Module 3 registers every agent before it may call anything: owner, purpose, risk, data, tools, budget, review date. Each agent gets its own identity, so no key is shared and no key is personal.
- Module 4 puts one model gateway between the apps and the providers. It checks, limits and records every model call, and the only provider key is the gateway’s.
- Module 5 puts a tool gateway in front of the tools and writes the approval rules as policy. Its switches stop one agent, or all writes.
- Module 6 turns the records into evidence: cost per agent and team, and the unregistered AI found in bills, network logs and code.
- Module 7 compares the course’s gateway with products, and plans how to roll it out in a real organisation.
When does an organisation need this?
Section titled “When does an organisation need this?”Not always. One team with one app and one provider does not need a gateway, a registry and policy files. The app’s own code and the provider’s console are enough. Microsoft’s architecture guide for AI gateways says the same: a gateway solves “specific scenarios that might not be present in every workload”.
The platform pays when several teams or agents share models, providers, keys and budgets, which is where Larkfield is now. It has a cost. A gateway is a new shared service that must stay up, and every check in it adds a little time to every call. Registration is work for every team. And a control that is easy to go around is worth little, as this course shows more than once. Each module says what its control costs and what it cannot do.
Get the project
Section titled “Get the project”You run Larkfield’s estate on your computer in the last lesson of this module. Download it now, and look at the two files of this lesson.
-
Open a terminal and move to your projects folder. If it does not exist, make it first with
mkdir ~/projects.Terminal window cd ~/projects -
If you have Git, clone the labs repository. If you cloned it for an earlier course, run
git -C nextia-learning-labs pullinstead.Terminal window git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.gitWithout Git, download the ZIP file of the repository. Unzip it in
~/projects. Its folder isnextia-learning-labs-main: use that name in the next steps. -
Copy the snapshot
startinto a new folder,ai-estate. Then copy Larkfield’s AI policy into it, from the snapshotend-of-m01:Terminal window cp -R nextia-learning-labs/C32/snapshots/start ai-estatemkdir ai-estate/docscp nextia-learning-labs/C32/snapshots/end-of-m01/docs/ai-policy.md ai-estate/docs/Terminal window Copy-Item -Recurse nextia-learning-labs\C32\snapshots\start ai-estateNew-Item -ItemType Directory ai-estate\docsCopy-Item nextia-learning-labs\C32\snapshots\end-of-m01\docs\ai-policy.md ai-estate\docs\Terminal window cp -R nextia-learning-labs/C32/snapshots/start ai-estatemkdir ai-estate/docscp nextia-learning-labs/C32/snapshots/end-of-m01/docs/ai-policy.md ai-estate/docs/ -
Open the folder
ai-estatein VS Code, or in another editor. -
Open
apps/digest/weekly_digest.py. Find the two lines that this lesson showed: lines 19 and 20. -
Open
docs/ai-policy.md. Read its twelve rules. You sort them in the next lesson.
The project’s README.md says what each folder holds. You do not need Python yet. The snapshot start is the estate before governance, and the snapshot end-of-m01 is the project at the end of this module. If your project breaks later, the snapshots README shows how to start again from a snapshot.
Common mistake
The course folder is missing
Symptom: the copy stops with a message like cp: nextia-learning-labs/C32/snapshots/start: No such file or directory. On Windows, Copy-Item says that it cannot find the path.
Cause: your clone of the labs repository is older than this course, or you are in the wrong folder.
Diagnose: run ls nextia-learning-labs (Windows: dir nextia-learning-labs). If there is no C32, the clone is old. If the command fails, you are not in ~/projects.
Recover: run git -C nextia-learning-labs pull, or download the ZIP file again. Then do step 3 again. If ai-estate already exists from a failed try, delete it first.
Your turn: one more question
Section titled “Your turn: one more question”The table above has nine questions. Add one more question that Larkfield cannot answer today, from your own work or from the estate’s diagram. For your question, write two lines: what Larkfield has today, and why it is not enough. Keep it for the next lesson, where you decide which control point could answer it.
Some answers other learners gave
- “Which customer data did the digest send, and can Fernway delete it?” Today: nothing but the expense line. Not enough: no record of the request’s content, and no contract that obliges Fernway to delete anything.
- “When the triage service fails, does a person see the ticket?” Today: the MLOps course’s design. Not enough: nothing outside the app records what happened to each call.
- “Is any team’s AI spending close to its budget?” Today: the monthly bill. Not enough: it arrives after the money is spent, and only per key.
Knowledge check
Section titled “Knowledge check”Check your understanding
The Fernway line on the bill is US$0.0013, the smallest line of all. Why is it still the most serious one?
Show the answer
Because the risk is the data and the decision, not the amount. The line shows that customer messages went to Fernway, a provider that Larkfield has no contract with. So nobody agreed how Fernway may keep or use them. It came from a personal key in code, in a script that nobody registered, and whose owner had left. A budget alert would never notice it.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Kwame must stop the support workflow at once, for example because it made a wrong refund. With today's estate, what can he do, and what else stops?
Show the answer
The only tool is the key. He can delete or rotate az-key-1, the service team’s shared key. That also stops the ticket triage service, which uses the same key. He cannot stop one app alone, and he cannot stop only the workflow’s writes while it still answers. Per-agent identity and kill switches, later in the course, fix this.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Which question can the provider's bill answer about Larkfield's AI, and name two that it cannot answer. Why not?
Show the answer
The bill answers how many requests and tokens each key used, per model, and what they cost. It cannot answer which app made a call, who owns it, what data it sent, or whether a model version changed. The bill records keys, not apps. Two apps can share a key, and one person can have a key that nobody else knows about.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Larkfield’s AI estate is six systems built by different teams: ticket triage, the support workflow, the document assistant, the release judge, the support MCP server and the weekly digest.
- Each app calls the provider its own way, with a shared key, a team key or a personal key in code. The constructed digest script sent customer messages to Fernway, a provider without a contract.
- The provider’s bill records keys, not apps. It cannot say who owns a call, what data it sent, or which app shares a key.
- Cost is not risk: the most serious line on the bill was the smallest.
- The cause of every unanswered question is the same: there is no single place where every AI call passes.
Your project now: ~/projects/ai-estate, a copy of the snapshot start, with docs/ai-policy.md. You can now say what Larkfield does not know. But Larkfield already has a policy that forbids what the digest did, and it changed nothing. Next, you sort the policy’s rules into those that code can enforce and those that only a person can check.