Rules in documents, rules in code
- Lesson 2
- 30 min
- Module 1: Why agents need platform controls
Where we are
In the last lesson, we saw that Larkfield cannot say what AI it runs, who owns it or what it costs. Larkfield has an AI policy that forbids what the digest script did, but a document cannot stop a script from calling a provider. In this lesson, we sort the policy's rules into those that code can enforce and those that only a person can check, and we find where each check sits, the way a building has signs on its walls and badge readers on its doors.
Lesson 2 of 21 · Governing AI Agents: Gateways, Registries and ControlsRules in documents, rules in code
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
For each rule of an AI policy, decide who can make it true: the platform’s code, a person in a review, or nobody. Then find the place where each check belongs. A rule that only lives in a document stops nothing.
Before you startWhat to know and to have ready
What you need to know
- Larkfield’s AI estate and the digest script, from Five agents and nobody knows.
- How the support workflow asks a person to approve a refund before it writes, from AI Agents and Workflow Orchestration. This lesson only names that step; you need no code from it.
What you need on your computer
- The project
~/projects/ai-estatewithdocs/ai-policy.md, from the last lesson. You need an editor, and no Python yet.
Signs and doors
Section titled “Signs and doors”Think of an office building. On the wall by the entrance, a sign lists the house rules. “Only staff in the server room.” “Visitors sign in at the desk.” “Be friendly to visitors.” “No food in the meeting rooms.”
Some of these rules have a door that enforces them. The server room has a badge reader. A person without the right badge cannot open it, whatever the sign says. The badge reader also keeps a log: who opened the door, and when.
Other rules have no door, but somebody checks them later. The cleaner sees the pizza boxes in a meeting room the next morning. The rule was broken, but there is evidence, and somebody can act on it.
And some rules have neither. Nobody can check “be friendly” with a door or a log. The building can only ask for it, and hope.
Notice one more thing about the badge reader. It sees who opens the door. It does not see what the person does inside the room. A door is a good place for a check, but every door sees only what passes through it.
The problem: a policy that nothing reads
Section titled “The problem: a policy that nothing reads”Larkfield’s AI policy has twelve rules. Open docs/ai-policy.md. These four are the rules that the digest script broke:
1. Every AI system that Larkfield builds, runs or buys is in the AI inventory, with an owner who works at Larkfield.2. An AI system may call only models and providers that the platform team has approved.3. Customer data goes only to providers with a contract that covers it. Card numbers and bank details never go to a model.4. No one shares a provider key, and no key is written in code, in a chat or in a ticket.Management approved the policy, and it was published for every team. But on Monday at 06:00, the script did not ask the policy. It opened a connection to Fernway, sent the messages and printed a summary. Nothing between the script and the provider knew that the policy existed.
The policy is like the sign on the wall. Larkfield’s estate has no doors.
The idea: three kinds of rule, and control points
Section titled “The idea: three kinds of rule, and control points”Sort every rule by what can make it true:
- Enforce in code. A program checks every action at the moment it happens, and refuses the actions that break the rule. Like the badge reader.
- Check in review. A person looks at evidence later and acts on what they find. Like the cleaner and the pizza boxes. The rule can be broken, but not quietly.
- Cannot check. No program and no record can show whether the rule is kept. The rule is only a request. That does not make it useless, but you should know that it is only a request.
Many rules are partly enforceable: code enforces one part, and a review checks the rest.
A rule can be enforced in code only where the action passes through code that you run. That place is a control point. Every action of one kind must pass it, and there, code can see the action and refuse it. Security engineers call it a policy enforcement point. A control point needs two things. Every action must pass through it, and it must see what the rule needs. A badge reader on a door that has a second, open entrance enforces nothing.
The picture shows the questions to ask for each rule, in order.
Text description of the diagram
Start with a rule of the AI policy. First question: does the action pass a point that Larkfield’s platform runs? If it does, ask the second question: can the code at that point see everything that the rule needs? If yes, enforce it in code: the point refuses the call. If it sees only a part, enforce that part in code and check the rest in a review. If it sees nothing useful, or if no point exists, ask the last question: does any record show it later? If yes, check it in a review, where a person reads the evidence. If no, nobody can check it: the rule is only a request.
A worked example: five rules sorted
Section titled “A worked example: five rules sorted”Here are five rules, sorted with the three questions. The course’s project has this worked example in docs/rules-worksheet.md. First, one rule in full, with the answer to each question.
Rule 2: “An AI system may call only models and providers that the platform team has approved.”
- Does the action pass a point that the platform runs? Not today: every app calls the provider straight. After Module 4, every model call passes the model gateway. So the answer becomes yes, if nothing can go around it.
- Can the code there see what the rule needs? Yes. The gateway sees which model the request asks for, and it holds the list of approved models and providers (the catalog of Module 2). It can refuse anything else.
- What about the way around? The digest script did not ask anybody’s gateway. So a second control is needed: a network egress rule that lets only the gateway connect to providers. Egress means traffic that leaves Larkfield’s network.
- What can the code still not see? A person who uses an AI website on a personal phone, outside Larkfield’s network.
Result: enforce in code, at the model gateway and the egress rule, with one gap that no code closes.
Now the same three questions for four more rules. Each row is the result for one rule.
| Rule | Enforce in code? | Control point | What the code cannot do |
|---|---|---|---|
| 2. Only approved models and providers | yes | model gateway (catalog), network egress rule | stop a person who uses an AI website on a personal phone |
| 3. Card numbers never go to a model | partly | model gateway (a guardrail that removes card numbers), registration (which data an agent may send) | find a card number written in words, or in a photo |
| 6. A person approves every change | yes | tool gateway (approval policy and approval grant) | judge whether the person read the proposal before approving |
| 9. A new model is tested on each system’s tasks | partly | the remap gate (each consumer’s evaluation, Module 2) | test a system that has no evaluation: no evidence, no change |
| 10. Answers are fair, polite and correct | no | review (sampled answers, evaluations) | decide “correct” for one answer at the gateway: it does not know the task |
Look at the pattern. Code can enforce a rule about something that a control point sees in one call. Examples are a model name, a number in the text or a write to an order. Code cannot enforce a rule about the meaning of the work. Is this answer correct for this customer? Did the person really read before approving? That needs the task, the data and a judgement, and no gateway has them.
Rule 10 is not useless because code cannot enforce it. It becomes a review with evidence: sampled answers, and the evaluations of the evaluation course. A document is still the right home for that kind of rule, together with who reviews it and how often.
The control points of the estate
Section titled “The control points of the estate”The course builds five control points. Each one sees something, and each one is blind to something. Knowing what a control point cannot see is as important as knowing what it does.
Text description of the diagram
An agent, here the support workflow, meets the control points in order. It gets a token, the registry gives its permissions, it makes a model call and a tool call, and audit records everything.
Identity sees which agent asks for a token, and when. It cannot see what the agent does with the token. The registry sees what each agent is allowed to do, and who owns it. It cannot see what the agent really does. The model gateway sees one model call: the request, the answer, the agent and the cost. It cannot see the task as a whole, the tool calls or the decisions. The tool gateway sees one tool call: the tool, its arguments and the approval. It cannot see why the model asked for it. Audit sees every change and the record of every call. It cannot see anything that did not pass a gateway.
Some AI is outside every control point. Examples are AI features inside bought tools, such as a help desk’s reply suggester, AI used outside Larkfield’s network, and a script that calls a provider straight.
In words, from first to last:
- Identity gives each agent its own name and short-lived credentials, so that every call says which agent made it. (Module 3.)
- The registry holds each agent’s record: owner, purpose, risk, data it may send, capabilities, tools, budget and review date. It is what the gateways ask: “is this agent allowed to do this?” (Module 3.)
- The model gateway is the one door for model calls. It checks identity, registration, capability, data and limits, routes the call to a model, and records it. (Modules 2 and 4.)
- The tool gateway is the one door for tool calls. It allows only the tools that the registration lists, keeps reads and writes apart, checks arguments, and asks for an approval before a write. (Module 5.)
- Audit is the record: every change to the registry, every call through both gateways. It is the evidence that a review reads. (Module 6.)
The most important limit is in the middle. The model gateway sees model calls, not the agent’s work. A refund is a tool call, not a model call. Only the tool gateway and the app’s own traces see the order of the steps, the approvals and the result of a run. And an AI feature inside a bought tool never passes any of these points. It enters the inventory through the buying process, as a check in review.
Before and after
Section titled “Before and after”Put the control points into Larkfield’s estate, and the picture of the last lesson changes.
Text description of the diagram
In the before panel, every app calls a provider with its own key. Triage and the workflow use az-key-1, the document assistant az-key-2 and the release judge az-key-3, all straight to Azure. The digest uses fw-7Q2K and calls Fernway, which has no contract, drawn in red. There are five paths and four keys, and no check that they share.
In the after panel, the four apps each have their own identity and call the model gateway. Only the gateway holds a provider key, and it calls Azure. The workflow also calls the tool gateway, which calls the MCP servers for tickets and orders. The registry and identity sit behind both gateways. The digest is not registered: its path to Fernway is crossed out, because the gateway refuses it and the network blocks it. There is one door for each kind of call, and the checks are in one place.
Three things change. Each app has its own identity, so a call names its app. Only the gateway holds a provider key, so a key in an app’s code is worth nothing. And the network lets only the gateway reach a provider, so a script cannot go around the door.
The last two matter as much as the gateway itself. Registration is enforced only if there is no way around it. Even then, some use slips through, such as a new provider that the egress rule does not know yet, or AI outside the network. So Module 6 also looks for unregistered AI in bills, network logs and code. Prevention and discovery work together.
“We have a gateway, so we are governed”
Section titled ““We have a gateway, so we are governed””Tomás likes the after picture. “Once every call goes through the gateway, we are governed.” Kwame points at the worked example. Rule 10, correct answers, is not enforced by any door. Rule 6 is enforced at the tool gateway, not the model gateway, and even there only the approval is checked, not the person’s judgement. Rule 9 needs an evaluation for every system, which people must write.
A gateway is a place where checks can run. It is not the checks themselves, the rules behind them or the people who review the evidence. A governed estate needs all of them. It needs rules in policy files, control points that enforce them, records, and people who read the records.
Sam asks: “So why not put every rule in code?” Because a rule in code has costs. It must be exact: “approved providers” becomes a list that somebody keeps up to date. It must be maintained, like any code. And a wrong rule in code refuses real work for every agent at once, as Module 4 shows with a bad configuration change. Put a rule in code when a control point sees what it needs in one call. Also, a wrong decision must be worse than a refused call. Leave the rest to a review, with good evidence.
Guided practice: open the worksheet
Section titled “Guided practice: open the worksheet”-
Copy the two files of this lesson into your project, from the snapshot
end-of-m01. In~/projects:Terminal window cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/rules-worksheet.md ai-estate/docs/cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/control-points.md ai-estate/docs/Terminal window Copy-Item nextia-learning-labs\C32\snapshots\end-of-m01\docs\rules-worksheet.md ai-estate\docs\Copy-Item nextia-learning-labs\C32\snapshots\end-of-m01\docs\control-points.md ai-estate\docs\Terminal window cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/rules-worksheet.md ai-estate/docs/cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/control-points.md ai-estate/docs/ -
Open
docs/control-points.md. It is the table of the five control points, with what each one holds, sees and cannot see. -
Open
docs/rules-worksheet.md. It has the five sorted rules of the worked example. Its last line says: “The other rules are for you to sort.” -
For rule 3 in the worksheet, find its control points in
control-points.md. Read what they cannot see. Check that the worksheet’s last column says the same.
Your turn: sort five rules
Section titled “Your turn: sort five rules”Sort these five rules of Larkfield’s policy. For each one, add a row to the table in docs/rules-worksheet.md. Write “enforce in code” (yes or partly), “check in review” or “cannot check”. Then name the control point, and what the code cannot do.
- Every AI system is in the AI inventory, with an owner who works at Larkfield. (Rule 1.)
- No one shares a provider key, and no key is written in code, in a chat or in a ticket. (Rule 4.)
- Each team stays within its monthly AI budget; the platform team is told before a team spends more. (Rule 5.)
- Each AI system can be stopped within five minutes, by its owner or by the platform team. (Rule 8.)
- We keep no customer message longer than we need it, also not in the AI’s logs. (Rule 11.)
Use the three questions of the flow for each rule. Do not look at the answer before you have five rows.
The course’s answer
| Rule | Enforce in code? | Control point | What the code cannot do |
|---|---|---|---|
| 1. In the inventory, with an owner who works here | yes, for model and tool calls | registry; both gateways refuse an agent without an active registration; the registry refuses an owner who has left | see AI inside a bought tool, or a script that calls a provider straight: find those in review (bills, egress logs, code) |
| 4. No shared or personal keys, no key in code | partly | identity (one identity per agent, short-lived tokens); only the gateway holds a provider key; the egress rule | stop a person who pastes a key into a chat; a scan of the code and the bill find old keys in review |
| 5. Each team within its budget, warned before | yes | model gateway (a budget per agent and per team, a warning before the limit) | count money spent outside the gateway, for example in a bought tool |
| 8. Stoppable within five minutes | yes | kill switches in both gateways; revoking credentials at identity | stop something that does not pass the gateways; check with a practised drill that the switch really works |
| 11. No customer message kept longer than needed | partly | model gateway: its usage record keeps no prompt or answer text | see the app’s own logs or what the provider keeps: check those in review, with the contract |
Two common differences. If you wrote “check in review” for rule 1, you are partly right: the registry enforces it for every call through the gateways, and review finds the rest. If you wrote “yes” for rule 11, look again at what the gateway cannot see: the provider and the app keep their own copies.
Two more rules, if you want to continue: rule 7 asks for reviews every three months for systems that act. The registry enforces it partly: it knows each review date and lists overdue reviews. The quality of a review is a person’s work. Rule 12 (people are told when they talk to an AI) cannot be checked at any control point: the gateway sees the request, not the screen the customer reads. It is a check in review of the app.
Common mistake
A rule marked 'enforce in code' because a tool exists
Symptom: your table says “yes” for a rule, but you cannot say which call the control point refuses. For example: “rule 11, yes, the gateway does not keep prompts”.
Cause: the control point enforces one part of the rule, and the table hides the rest. The gateway keeps no prompt text, but the app writes its own logs and the provider keeps the request.
Diagnose: for each “yes”, write one sentence: “The control point refuses this action because it sees this.” Then read the control point’s “cannot see” column in control-points.md.
Recover: if the sentence does not work, change “yes” to “partly”, and add a review for the part that no code sees.
Knowledge check
Section titled “Knowledge check”Check your understanding
A control point must meet two conditions before it can enforce a rule. What are they? Use the digest script as an example.
Show the answer
Every action of that kind must pass through it, and it must see what the rule needs. A model gateway sees the provider and model of each call, so it can enforce “only approved providers”. But the digest script called Fernway straight, so the gateway alone does not meet the first condition. The network egress rule makes it true: only the gateway may reach a provider.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás says: 'Every call goes through the gateway now, so the rule that answers are correct is enforced.' What is wrong, and how should Larkfield check that rule instead?
Show the answer
The gateway sees one model call, but not the task, the customer’s order or the policy that decides what correct means here. So it cannot decide whether an answer is correct. The rule becomes a check in review: sampled answers and evaluations of each system, read by people. The gateway helps by recording which model answered each call.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A rule in a document stops nothing by itself. Sort each rule: enforce in code, check in review, or cannot check. Many rules are partly enforceable.
- Code enforces a rule only at a control point: a place that every action of one kind passes, where code sees what the rule needs.
- The five control points are identity, the registry, the model gateway, the tool gateway and audit. Each one is blind to something: the model gateway sees model calls, not the agent’s work.
- A gateway is a place for checks, not governance by itself. Registration is enforced only when there is no way around the gateway: one provider key, held by the gateway, and an egress rule.
- Rules about meaning (correct, fair, read before approving) stay as reviews with evidence.
Your project now: docs/ai-policy.md, docs/rules-worksheet.md with your five new rows, and docs/control-points.md. You can now say where each rule of the policy could be checked. But so far, the control points are boxes on a picture. Next, you start Larkfield’s estate on your computer, send requests from each app, and read what the provider records.