Skip to content
Français

Five agents and nobody knows

  • Lesson 1
  • 30 min
  • Module 1: Why agents need platform controls

Where we are

In the agents course, we built one support workflow and kept it safe: explicit state, limits, and a person's approval before every change. But one safe agent is not a safe organisation. Larkfield now runs several AI systems, each built by a different team, with its own keys, and one of them nobody knew about. In this lesson, we look at what a company cannot answer when nobody sees all of its AI, the way a family cannot read a shared card statement.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

See what an organisation cannot answer about its own AI when every team calls model providers its own way. You meet Larkfield’s AI estate and read its only record, the provider’s bill. Then you write down the questions that nobody at Larkfield can answer today.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need on your computer

  • Nothing for the first part of the lesson. At the end, you download the course project. You need Git, or a browser to download a ZIP file.

You need no account, no key and no payment in this course. Every model answer comes from a recording, as the last lesson of this module shows.

Think of a family that shares one credit card. Four people have a copy of it. At the end of the month, the statement arrives. It lists shops, dates and amounts, but not who bought what. Most lines look normal. One line is from a shop that nobody in the family knows, for a small amount.

Now the family has questions. Who bought this? Is somebody’s copy of the card lost? One family member moved out last month: does that person still have a copy? Nobody can answer from the statement, because the statement records the card, not the person. And the small amount is not the real worry. The worry is that the card works in places that nobody chose.

A company that uses AI can be in the same position. Its “card” is a provider key, and its “statement” is the provider’s bill.

The problem: a script that nobody knew about

Section titled “The problem: a script that nobody knew about”

This is a constructed practice case. Larkfield, its people and the provider Fernway are made up for the course.

Larkfield sells home and garden products and runs a help desk. A few months ago, Lee, an analyst on the service team, wanted to save an hour every Monday. Lee wrote a small script, weekly_digest.py. It reads the week’s customer messages, sends them to a model, and prints a summary for the Monday meeting. The script runs on a build machine, build-runner-3, every Monday at 06:00.

Lee used Fernway, a provider that Lee already had an account with, and a key from that personal account. These two lines of the script show it:

apps/digest/weekly_digest.py (lines 19-20)
FERNWAY_URL = os.environ.get("FERNWAY_URL", "http://127.0.0.1:8320/fernway/v1")
FERNWAY_KEY = "sim-fw-personal-7Q2K9XW4" # my own account, for now

In the course, the address is the provider simulator on your computer, and the key is a practice key. In the story, they are a real provider and a real personal key.

Lee left Larkfield on 31 August. The script did not leave. It kept running every Monday, with Lee’s key, from a machine that nobody connected with AI. Each run sent 40 customer messages to Fernway, a provider that Larkfield has no contract with. So nobody had agreed how Fernway may store or use those messages.

The first sign was a line in the expenses:

evidence/fernway-expense-2026-09.csv
month,provider,key_id,deployment,requests,tokens_in,tokens_out,cost_usd
2026-09,fernway,fw-7Q2K,fernway-swift,1,2764,152,0.001349

One request, 2,764 tokens in, 152 out, US$0.0013. The amount is almost nothing. The data is the problem: customer messages, with their complaints and order numbers, went to a company that Larkfield never chose.

Larkfield even has an AI policy, approved by management. The script broke at least four of its twelve rules. The rules ask for an inventory of all AI, approved providers only, customer data only under a contract, and no keys in code. The policy is a document. Nothing in Larkfield’s systems read it, so nothing stopped the script.

The digest is not the only AI at Larkfield. Earlier courses built several AI systems for the company, and each team now runs its own. Together, the course calls them Larkfield’s AI estate. The name means every AI system that the company runs, like the buildings of an estate that one owner looks after.

App Owner From Model it calls Key
Ticket triage Amira (service team) the MLOps course chat-small, and embed-small for search az-key-1, the service team’s shared key
Support workflow Amira a pilot of the agents course workflow chat-strong az-key-1, the same shared key
Document assistant Mei (data team) the capstone course chat-small az-key-2, the data team’s key
Release judge Priya (evaluation) the evaluation course chat-strong az-key-3, Priya’s personal key
Support MCP server the platform team the MCP course none: it gives agents tools none
Weekly digest Lee, who left nobody registered it fernway-swift fw-7Q2K, Lee’s personal key

In September, the support workflow ran as a pilot: an early version of the workflow that the agents course finishes in October.

Five of these systems call models: they are the five agents of this lesson’s title. Kwame, Larkfield’s solution architect, knew about three of them when he started, and about the MCP server. He did not know about the release judge or the digest. Priya’s judge is a normal, useful tool: it checks the support replies before each release. But it calls the provider from Priya’s laptop, with a key that she made for herself.

Three columns. Apps and where they run: ticket triage on triage-svc-01 and the support workflow on workflow-01, both Amira's; the document assistant on docs-01, Mei's; the release judge on priya-laptop, Priya's; and weekly_digest.py on build-runner-3, owner Lee, who left, drawn in red. Keys: triage and the workflow both point to az-key-1, the service team's shared key; the document assistant to az-key-2, the data team's key; the judge to az-key-3, Priya's personal key; the digest to fw-7Q2K, a personal account, in red. Providers: the three Azure keys point to Azure (East US 2) with chat-small, chat-strong and embed-small, where Larkfield has a contract; fw-7Q2K points to Fernway, fernway-swift, no contract, in red. Below: the support MCP server makes no model calls and has no key. Both providers feed the provider's bill: one line per key and model, no app name, no owner, no purpose.
Larkfield's AI estate before any platform control: five paths to two providers, four keys in the apps' code, and a bill that names only keys.
Text description of the diagram

On the left are the apps and the machines they run on. Ticket triage runs on triage-svc-01 and the support workflow on workflow-01; Amira owns both. The document assistant runs on docs-01 and belongs to Mei. The release judge runs on priya-laptop. The weekly digest runs on build-runner-3, and its owner, Lee, has left; it is drawn in red.

In the middle are the keys. Triage and the workflow share az-key-1, the service team’s key. The document assistant uses az-key-2, the data team’s key. The judge uses az-key-3, Priya’s personal key. The digest uses fw-7Q2K, a key from a personal account, in red.

On the right are the providers. The three Azure keys reach Azure in the East US 2 region, where Larkfield has a contract. fw-7Q2K reaches Fernway, where Larkfield has no contract. The support MCP server makes no model calls and has no key. Both providers send a bill with one line per key and model. The bill has no app name, no owner and no purpose.

Each app made sense when its team built it. The problem is the sum: five paths to two providers, four keys written in code, and no single place that sees a call.

The provider’s bill is the only record that covers every app. So let us try to answer three simple questions from it. These are the bill lines of the replayed month that the course uses in Module 6. They cover the first days of September, when every app still called the provider with its own key. The traffic is replayed from real recordings; the bill itself is simulated.

Key Requests Cost
az-key-1 12,442 US$6.48
az-key-2 324 US$0.05
az-key-3 202 US$0.83
fw-7Q2K 1 US$0.0013

Question 1: what did the support workflow cost? The workflow uses az-key-1. But triage uses az-key-1 too. The bill splits each key by model, and in this estate only the workflow uses chat-strong. So you can guess: the chat-strong line of az-key-1 is the workflow. That guess breaks on the day the triage service also calls chat-strong. The bill cannot tell two apps on one key apart.

Question 2: whose is az-key-3? The bill names an account, not a person. Somebody must search the provider’s console, find that Priya made the key, and then ask her what calls it. The answer comes from a conversation, not from a record.

Question 3: which app sent customer data to a provider without a contract? The bill shows a key at Fernway. It does not show the app, the machine, the data or the owner. Larkfield learns that something happened, after it happened.

So the bill answers “how much, per key”. Every question that governance asks is about apps, owners and data, and the bill knows none of them.

It is tempting to look at the biggest number first. The chart shows why that is the wrong place to look.

The replayed month's bill for the days before each app moved behind the gateway (replayed traffic, simulated bill). The smallest line is the most serious one.
Text description of the chart

Four bars, one per key. az-key-1, the service team’s shared key for triage and the workflow, is the highest at US$6.48 for 12,442 requests. az-key-3, Priya’s personal key, is US$0.83 for 202 requests. az-key-2, the data team’s key, is US$0.05. fw-7Q2K, the personal Fernway key, is US$0.0013 for 1 request: the smallest bar by far, and the only one that sent customer messages to a provider without a contract.

The Fernway line is about 4,800 times smaller than the shared key’s line. But it is the one that breaks the company’s own rules. A budget alert would never fire on it. Risk depends on what data goes where, and who decided, not on the amount. So the controls in this course start with knowing what runs, and only then look at money.

The course uses the prices of its recordings, per million tokens in and out: chat-small US$0.10 and US$0.50, chat-strong US$2 and US$10. Fernway’s price is made up. Prices change: check the provider’s current price page.

Kwame becomes the owner of Larkfield’s AI platform. His first task is to write down what the company cannot answer. Sam, a support agent who is new to all of this, asks the simplest question first: “Can’t we just ask every team?” Kwame tried. The teams told him about four systems. The other two came from a bill and a key that nobody had mentioned.

Question What Larkfield has today Why it is not enough
What AI do we run? Kwame’s list, from asking teams It missed the judge and the digest. A list that people keep by hand is out of date the day someone writes a new script.
Who owns each system? Each team’s memory The digest’s owner left. Nobody knew that it was still running.
Which models and providers does each one call? Each app’s code Five codebases to read, and one of them nobody knew to read.
What data goes to which provider? Nothing Customer messages went to Fernway, and no record says so.
What does each app cost? The bill, per key Two apps share az-key-1. A personal key has its own account.
Who holds which key? The provider’s console Keys are in code and on a laptop. When Lee left, nobody stopped the key.
Which systems act, and which only suggest? The agents course’s design notes The support workflow writes refunds, after a person approves. Nothing outside the app records that it can.
Can we stop one system now? Delete its key Deleting az-key-1 stops the triage service and the workflow together.
Did a model change behind an app? Each app’s settings Nothing records which model version answered each call.

Every row has the same cause. There is no single place where every AI call passes, so there is no place that can check a call, record it, or stop it. Each team built a good app. Nobody built the platform between the apps and the providers.

Tomás, an AI engineer on the platform team, has a quick fix: “Let’s block Fernway on the firewall.” Kwame agrees that it helps for Fernway. But the next script will use another provider, or another personal key at Azure. Blocking one name answers one row of the table. The course answers all of them.

Here is how the course answers each row. You build each part on your computer, in Larkfield’s project.

  • Module 2 gives every app a capability to ask for, such as “fast-small” or “strong-reasoning”, instead of a model name. The platform decides which model serves it, and records which model answered.
  • Module 3 registers every agent before it may call anything: owner, purpose, risk, data, tools, budget, review date. Each agent gets its own identity, so no key is shared and no key is personal.
  • Module 4 puts one model gateway between the apps and the providers. It checks, limits and records every model call, and the only provider key is the gateway’s.
  • Module 5 puts a tool gateway in front of the tools and writes the approval rules as policy. Its switches stop one agent, or all writes.
  • Module 6 turns the records into evidence: cost per agent and team, and the unregistered AI found in bills, network logs and code.
  • Module 7 compares the course’s gateway with products, and plans how to roll it out in a real organisation.

Not always. One team with one app and one provider does not need a gateway, a registry and policy files. The app’s own code and the provider’s console are enough. Microsoft’s architecture guide for AI gateways says the same: a gateway solves “specific scenarios that might not be present in every workload”.

The platform pays when several teams or agents share models, providers, keys and budgets, which is where Larkfield is now. It has a cost. A gateway is a new shared service that must stay up, and every check in it adds a little time to every call. Registration is work for every team. And a control that is easy to go around is worth little, as this course shows more than once. Each module says what its control costs and what it cannot do.

You run Larkfield’s estate on your computer in the last lesson of this module. Download it now, and look at the two files of this lesson.

  1. Open a terminal and move to your projects folder. If it does not exist, make it first with mkdir ~/projects.

    Terminal window
    cd ~/projects
  2. If you have Git, clone the labs repository. If you cloned it for an earlier course, run git -C nextia-learning-labs pull instead.

    Terminal window
    git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.git

    Without Git, download the ZIP file of the repository. Unzip it in ~/projects. Its folder is nextia-learning-labs-main: use that name in the next steps.

  3. Copy the snapshot start into a new folder, ai-estate. Then copy Larkfield’s AI policy into it, from the snapshot end-of-m01:

    Terminal window
    cp -R nextia-learning-labs/C32/snapshots/start ai-estate
    mkdir ai-estate/docs
    cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/ai-policy.md ai-estate/docs/
  4. Open the folder ai-estate in VS Code, or in another editor.

  5. Open apps/digest/weekly_digest.py. Find the two lines that this lesson showed: lines 19 and 20.

  6. Open docs/ai-policy.md. Read its twelve rules. You sort them in the next lesson.

The project’s README.md says what each folder holds. You do not need Python yet. The snapshot start is the estate before governance, and the snapshot end-of-m01 is the project at the end of this module. If your project breaks later, the snapshots README shows how to start again from a snapshot.

Common mistake

The course folder is missing

Symptom: the copy stops with a message like cp: nextia-learning-labs/C32/snapshots/start: No such file or directory. On Windows, Copy-Item says that it cannot find the path.

Cause: your clone of the labs repository is older than this course, or you are in the wrong folder.

Diagnose: run ls nextia-learning-labs (Windows: dir nextia-learning-labs). If there is no C32, the clone is old. If the command fails, you are not in ~/projects.

Recover: run git -C nextia-learning-labs pull, or download the ZIP file again. Then do step 3 again. If ai-estate already exists from a failed try, delete it first.

The table above has nine questions. Add one more question that Larkfield cannot answer today, from your own work or from the estate’s diagram. For your question, write two lines: what Larkfield has today, and why it is not enough. Keep it for the next lesson, where you decide which control point could answer it.

Some answers other learners gave
  • “Which customer data did the digest send, and can Fernway delete it?” Today: nothing but the expense line. Not enough: no record of the request’s content, and no contract that obliges Fernway to delete anything.
  • “When the triage service fails, does a person see the ticket?” Today: the MLOps course’s design. Not enough: nothing outside the app records what happened to each call.
  • “Is any team’s AI spending close to its budget?” Today: the monthly bill. Not enough: it arrives after the money is spent, and only per key.

Check your understanding

The Fernway line on the bill is US$0.0013, the smallest line of all. Why is it still the most serious one?

Show the answer

Because the risk is the data and the decision, not the amount. The line shows that customer messages went to Fernway, a provider that Larkfield has no contract with. So nobody agreed how Fernway may keep or use them. It came from a personal key in code, in a script that nobody registered, and whose owner had left. A budget alert would never notice it.

Check your understanding

Kwame must stop the support workflow at once, for example because it made a wrong refund. With today's estate, what can he do, and what else stops?

Show the answer

The only tool is the key. He can delete or rotate az-key-1, the service team’s shared key. That also stops the ticket triage service, which uses the same key. He cannot stop one app alone, and he cannot stop only the workflow’s writes while it still answers. Per-agent identity and kill switches, later in the course, fix this.

Check your understanding

Which question can the provider's bill answer about Larkfield's AI, and name two that it cannot answer. Why not?

Show the answer

The bill answers how many requests and tokens each key used, per model, and what they cost. It cannot answer which app made a call, who owns it, what data it sent, or whether a model version changed. The bill records keys, not apps. Two apps can share a key, and one person can have a key that nobody else knows about.

Recap

  • Larkfield’s AI estate is six systems built by different teams: ticket triage, the support workflow, the document assistant, the release judge, the support MCP server and the weekly digest.
  • Each app calls the provider its own way, with a shared key, a team key or a personal key in code. The constructed digest script sent customer messages to Fernway, a provider without a contract.
  • The provider’s bill records keys, not apps. It cannot say who owns a call, what data it sent, or which app shares a key.
  • Cost is not risk: the most serious line on the bill was the smallest.
  • The cause of every unanswered question is the same: there is no single place where every AI call passes.

Your project now: ~/projects/ai-estate, a copy of the snapshot start, with docs/ai-policy.md. You can now say what Larkfield does not know. But Larkfield already has a policy that forbids what the digest did, and it changed nothing. Next, you sort the policy’s rules into those that code can enforce and those that only a person can check.

Next: Rules in documents, rules in code