Skip to content
Français

Rules in documents, rules in code

  • Lesson 2
  • 30 min
  • Module 1: Why agents need platform controls

Where we are

In the last lesson, we saw that Larkfield cannot say what AI it runs, who owns it or what it costs. Larkfield has an AI policy that forbids what the digest script did, but a document cannot stop a script from calling a provider. In this lesson, we sort the policy's rules into those that code can enforce and those that only a person can check, and we find where each check sits, the way a building has signs on its walls and badge readers on its doors.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

For each rule of an AI policy, decide who can make it true: the platform’s code, a person in a review, or nobody. Then find the place where each check belongs. A rule that only lives in a document stops nothing.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need on your computer

  • The project ~/projects/ai-estate with docs/ai-policy.md, from the last lesson. You need an editor, and no Python yet.

Think of an office building. On the wall by the entrance, a sign lists the house rules. “Only staff in the server room.” “Visitors sign in at the desk.” “Be friendly to visitors.” “No food in the meeting rooms.”

Some of these rules have a door that enforces them. The server room has a badge reader. A person without the right badge cannot open it, whatever the sign says. The badge reader also keeps a log: who opened the door, and when.

Other rules have no door, but somebody checks them later. The cleaner sees the pizza boxes in a meeting room the next morning. The rule was broken, but there is evidence, and somebody can act on it.

And some rules have neither. Nobody can check “be friendly” with a door or a log. The building can only ask for it, and hope.

Notice one more thing about the badge reader. It sees who opens the door. It does not see what the person does inside the room. A door is a good place for a check, but every door sees only what passes through it.

Larkfield’s AI policy has twelve rules. Open docs/ai-policy.md. These four are the rules that the digest script broke:

docs/ai-policy.md (rules 1 to 4)
1. Every AI system that Larkfield builds, runs or buys is in the AI inventory, with an owner who works at Larkfield.
2. An AI system may call only models and providers that the platform team has approved.
3. Customer data goes only to providers with a contract that covers it. Card numbers and bank details never go to a model.
4. No one shares a provider key, and no key is written in code, in a chat or in a ticket.

Management approved the policy, and it was published for every team. But on Monday at 06:00, the script did not ask the policy. It opened a connection to Fernway, sent the messages and printed a summary. Nothing between the script and the provider knew that the policy existed.

The policy is like the sign on the wall. Larkfield’s estate has no doors.

The idea: three kinds of rule, and control points

Section titled “The idea: three kinds of rule, and control points”

Sort every rule by what can make it true:

  • Enforce in code. A program checks every action at the moment it happens, and refuses the actions that break the rule. Like the badge reader.
  • Check in review. A person looks at evidence later and acts on what they find. Like the cleaner and the pizza boxes. The rule can be broken, but not quietly.
  • Cannot check. No program and no record can show whether the rule is kept. The rule is only a request. That does not make it useless, but you should know that it is only a request.

Many rules are partly enforceable: code enforces one part, and a review checks the rest.

A rule can be enforced in code only where the action passes through code that you run. That place is a control point. Every action of one kind must pass it, and there, code can see the action and refuse it. Security engineers call it a policy enforcement point. A control point needs two things. Every action must pass through it, and it must see what the rule needs. A badge reader on a door that has a second, open entrance enforces nothing.

The picture shows the questions to ask for each rule, in order.

A decision flow. Start: a rule of the AI policy. Question 1: does the action pass a point that Larkfield's platform runs? If yes, question 2: can the code at that point see everything the rule needs? Yes: enforce in code, the point refuses the call. Only a part: enforce the part in code, check the rest in review. No, or if question 1 was no: does any record show it later? Yes: check in review, a person reads the evidence. No: cannot check, the rule is only a request.
How to sort a rule: first find a control point, then ask what it can see, then look for a record.
Text description of the diagram

Start with a rule of the AI policy. First question: does the action pass a point that Larkfield’s platform runs? If it does, ask the second question: can the code at that point see everything that the rule needs? If yes, enforce it in code: the point refuses the call. If it sees only a part, enforce that part in code and check the rest in a review. If it sees nothing useful, or if no point exists, ask the last question: does any record show it later? If yes, check it in a review, where a person reads the evidence. If no, nobody can check it: the rule is only a request.

Here are five rules, sorted with the three questions. The course’s project has this worked example in docs/rules-worksheet.md. First, one rule in full, with the answer to each question.

Rule 2: “An AI system may call only models and providers that the platform team has approved.”

  1. Does the action pass a point that the platform runs? Not today: every app calls the provider straight. After Module 4, every model call passes the model gateway. So the answer becomes yes, if nothing can go around it.
  2. Can the code there see what the rule needs? Yes. The gateway sees which model the request asks for, and it holds the list of approved models and providers (the catalog of Module 2). It can refuse anything else.
  3. What about the way around? The digest script did not ask anybody’s gateway. So a second control is needed: a network egress rule that lets only the gateway connect to providers. Egress means traffic that leaves Larkfield’s network.
  4. What can the code still not see? A person who uses an AI website on a personal phone, outside Larkfield’s network.

Result: enforce in code, at the model gateway and the egress rule, with one gap that no code closes.

Now the same three questions for four more rules. Each row is the result for one rule.

Rule Enforce in code? Control point What the code cannot do
2. Only approved models and providers yes model gateway (catalog), network egress rule stop a person who uses an AI website on a personal phone
3. Card numbers never go to a model partly model gateway (a guardrail that removes card numbers), registration (which data an agent may send) find a card number written in words, or in a photo
6. A person approves every change yes tool gateway (approval policy and approval grant) judge whether the person read the proposal before approving
9. A new model is tested on each system’s tasks partly the remap gate (each consumer’s evaluation, Module 2) test a system that has no evaluation: no evidence, no change
10. Answers are fair, polite and correct no review (sampled answers, evaluations) decide “correct” for one answer at the gateway: it does not know the task

Look at the pattern. Code can enforce a rule about something that a control point sees in one call. Examples are a model name, a number in the text or a write to an order. Code cannot enforce a rule about the meaning of the work. Is this answer correct for this customer? Did the person really read before approving? That needs the task, the data and a judgement, and no gateway has them.

Rule 10 is not useless because code cannot enforce it. It becomes a review with evidence: sampled answers, and the evaluations of the evaluation course. A document is still the right home for that kind of rule, together with who reviews it and how often.

The course builds five control points. Each one sees something, and each one is blind to something. Knowing what a control point cannot see is as important as knowing what it does.

An agent, the support workflow, on the left, with an arrow along a line: token, allowed?, model call, tool call, record. Above the line, five control points from left to right, each with what it sees and what it cannot see. Identity sees which agent asks for a token, and when; it cannot see what the agent does with the token. Registry sees what each agent is allowed to do, and its owner; it cannot see what the agent really does. Model gateway sees one model call: request, answer, agent, cost; it cannot see the task as a whole, tool calls or decisions. Tool gateway sees one tool call: tool, arguments, approval; it cannot see why the model asked for it. Audit sees every change and every call's record; it cannot see anything that did not pass a gateway. Below, outside every control point: AI inside bought tools, such as a help desk's reply suggester, AI used outside Larkfield's network, and a script that calls a provider straight.
The five control points, in the order an agent's work meets them, with what each one sees and cannot see.
Text description of the diagram

An agent, here the support workflow, meets the control points in order. It gets a token, the registry gives its permissions, it makes a model call and a tool call, and audit records everything.

Identity sees which agent asks for a token, and when. It cannot see what the agent does with the token. The registry sees what each agent is allowed to do, and who owns it. It cannot see what the agent really does. The model gateway sees one model call: the request, the answer, the agent and the cost. It cannot see the task as a whole, the tool calls or the decisions. The tool gateway sees one tool call: the tool, its arguments and the approval. It cannot see why the model asked for it. Audit sees every change and the record of every call. It cannot see anything that did not pass a gateway.

Some AI is outside every control point. Examples are AI features inside bought tools, such as a help desk’s reply suggester, AI used outside Larkfield’s network, and a script that calls a provider straight.

In words, from first to last:

  • Identity gives each agent its own name and short-lived credentials, so that every call says which agent made it. (Module 3.)
  • The registry holds each agent’s record: owner, purpose, risk, data it may send, capabilities, tools, budget and review date. It is what the gateways ask: “is this agent allowed to do this?” (Module 3.)
  • The model gateway is the one door for model calls. It checks identity, registration, capability, data and limits, routes the call to a model, and records it. (Modules 2 and 4.)
  • The tool gateway is the one door for tool calls. It allows only the tools that the registration lists, keeps reads and writes apart, checks arguments, and asks for an approval before a write. (Module 5.)
  • Audit is the record: every change to the registry, every call through both gateways. It is the evidence that a review reads. (Module 6.)

The most important limit is in the middle. The model gateway sees model calls, not the agent’s work. A refund is a tool call, not a model call. Only the tool gateway and the app’s own traces see the order of the steps, the approvals and the result of a run. And an AI feature inside a bought tool never passes any of these points. It enters the inventory through the buying process, as a check in review.

Put the control points into Larkfield’s estate, and the picture of the last lesson changes.

Two panels. Before: each app calls a provider with its own key. Triage and workflow with az-key-1, documents with az-key-2 and judge with az-key-3 all point straight to Azure; the digest with fw-7Q2K points to Fernway, no contract, in red. 5 paths, 4 keys, no common check. After: one model gateway, one tool gateway. Triage, workflow, documents and judge, each with its own identity, point to the model gateway, which holds the only provider key and points to Azure. The workflow also points to the tool gateway, which points to the MCP servers for tickets and orders. Registry and identity sit behind both gateways, with dashed links. The digest, not registered, has a red dashed arrow to Fernway with a cross: blocked, no registration, no egress. 1 door per kind of call, checks in one place.
Before: five paths and four keys, with no common check. After: one door for model calls and one for tool calls, with the registry behind both.
Text description of the diagram

In the before panel, every app calls a provider with its own key. Triage and the workflow use az-key-1, the document assistant az-key-2 and the release judge az-key-3, all straight to Azure. The digest uses fw-7Q2K and calls Fernway, which has no contract, drawn in red. There are five paths and four keys, and no check that they share.

In the after panel, the four apps each have their own identity and call the model gateway. Only the gateway holds a provider key, and it calls Azure. The workflow also calls the tool gateway, which calls the MCP servers for tickets and orders. The registry and identity sit behind both gateways. The digest is not registered: its path to Fernway is crossed out, because the gateway refuses it and the network blocks it. There is one door for each kind of call, and the checks are in one place.

Three things change. Each app has its own identity, so a call names its app. Only the gateway holds a provider key, so a key in an app’s code is worth nothing. And the network lets only the gateway reach a provider, so a script cannot go around the door.

The last two matter as much as the gateway itself. Registration is enforced only if there is no way around it. Even then, some use slips through, such as a new provider that the egress rule does not know yet, or AI outside the network. So Module 6 also looks for unregistered AI in bills, network logs and code. Prevention and discovery work together.

“We have a gateway, so we are governed”

Section titled ““We have a gateway, so we are governed””

Tomás likes the after picture. “Once every call goes through the gateway, we are governed.” Kwame points at the worked example. Rule 10, correct answers, is not enforced by any door. Rule 6 is enforced at the tool gateway, not the model gateway, and even there only the approval is checked, not the person’s judgement. Rule 9 needs an evaluation for every system, which people must write.

A gateway is a place where checks can run. It is not the checks themselves, the rules behind them or the people who review the evidence. A governed estate needs all of them. It needs rules in policy files, control points that enforce them, records, and people who read the records.

Sam asks: “So why not put every rule in code?” Because a rule in code has costs. It must be exact: “approved providers” becomes a list that somebody keeps up to date. It must be maintained, like any code. And a wrong rule in code refuses real work for every agent at once, as Module 4 shows with a bad configuration change. Put a rule in code when a control point sees what it needs in one call. Also, a wrong decision must be worse than a refused call. Leave the rest to a review, with good evidence.

  1. Copy the two files of this lesson into your project, from the snapshot end-of-m01. In ~/projects:

    Terminal window
    cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/rules-worksheet.md ai-estate/docs/
    cp nextia-learning-labs/C32/snapshots/end-of-m01/docs/control-points.md ai-estate/docs/
  2. Open docs/control-points.md. It is the table of the five control points, with what each one holds, sees and cannot see.

  3. Open docs/rules-worksheet.md. It has the five sorted rules of the worked example. Its last line says: “The other rules are for you to sort.”

  4. For rule 3 in the worksheet, find its control points in control-points.md. Read what they cannot see. Check that the worksheet’s last column says the same.

Sort these five rules of Larkfield’s policy. For each one, add a row to the table in docs/rules-worksheet.md. Write “enforce in code” (yes or partly), “check in review” or “cannot check”. Then name the control point, and what the code cannot do.

  1. Every AI system is in the AI inventory, with an owner who works at Larkfield. (Rule 1.)
  2. No one shares a provider key, and no key is written in code, in a chat or in a ticket. (Rule 4.)
  3. Each team stays within its monthly AI budget; the platform team is told before a team spends more. (Rule 5.)
  4. Each AI system can be stopped within five minutes, by its owner or by the platform team. (Rule 8.)
  5. We keep no customer message longer than we need it, also not in the AI’s logs. (Rule 11.)

Use the three questions of the flow for each rule. Do not look at the answer before you have five rows.

The course’s answer
Rule Enforce in code? Control point What the code cannot do
1. In the inventory, with an owner who works here yes, for model and tool calls registry; both gateways refuse an agent without an active registration; the registry refuses an owner who has left see AI inside a bought tool, or a script that calls a provider straight: find those in review (bills, egress logs, code)
4. No shared or personal keys, no key in code partly identity (one identity per agent, short-lived tokens); only the gateway holds a provider key; the egress rule stop a person who pastes a key into a chat; a scan of the code and the bill find old keys in review
5. Each team within its budget, warned before yes model gateway (a budget per agent and per team, a warning before the limit) count money spent outside the gateway, for example in a bought tool
8. Stoppable within five minutes yes kill switches in both gateways; revoking credentials at identity stop something that does not pass the gateways; check with a practised drill that the switch really works
11. No customer message kept longer than needed partly model gateway: its usage record keeps no prompt or answer text see the app’s own logs or what the provider keeps: check those in review, with the contract

Two common differences. If you wrote “check in review” for rule 1, you are partly right: the registry enforces it for every call through the gateways, and review finds the rest. If you wrote “yes” for rule 11, look again at what the gateway cannot see: the provider and the app keep their own copies.

Two more rules, if you want to continue: rule 7 asks for reviews every three months for systems that act. The registry enforces it partly: it knows each review date and lists overdue reviews. The quality of a review is a person’s work. Rule 12 (people are told when they talk to an AI) cannot be checked at any control point: the gateway sees the request, not the screen the customer reads. It is a check in review of the app.

Common mistake

A rule marked 'enforce in code' because a tool exists

Symptom: your table says “yes” for a rule, but you cannot say which call the control point refuses. For example: “rule 11, yes, the gateway does not keep prompts”.

Cause: the control point enforces one part of the rule, and the table hides the rest. The gateway keeps no prompt text, but the app writes its own logs and the provider keeps the request.

Diagnose: for each “yes”, write one sentence: “The control point refuses this action because it sees this.” Then read the control point’s “cannot see” column in control-points.md.

Recover: if the sentence does not work, change “yes” to “partly”, and add a review for the part that no code sees.

Check your understanding

A control point must meet two conditions before it can enforce a rule. What are they? Use the digest script as an example.

Show the answer

Every action of that kind must pass through it, and it must see what the rule needs. A model gateway sees the provider and model of each call, so it can enforce “only approved providers”. But the digest script called Fernway straight, so the gateway alone does not meet the first condition. The network egress rule makes it true: only the gateway may reach a provider.

Check your understanding

Tomás says: 'Every call goes through the gateway now, so the rule that answers are correct is enforced.' What is wrong, and how should Larkfield check that rule instead?

Show the answer

The gateway sees one model call, but not the task, the customer’s order or the policy that decides what correct means here. So it cannot decide whether an answer is correct. The rule becomes a check in review: sampled answers and evaluations of each system, read by people. The gateway helps by recording which model answered each call.

Recap

  • A rule in a document stops nothing by itself. Sort each rule: enforce in code, check in review, or cannot check. Many rules are partly enforceable.
  • Code enforces a rule only at a control point: a place that every action of one kind passes, where code sees what the rule needs.
  • The five control points are identity, the registry, the model gateway, the tool gateway and audit. Each one is blind to something: the model gateway sees model calls, not the agent’s work.
  • A gateway is a place for checks, not governance by itself. Registration is enforced only when there is no way around the gateway: one provider key, held by the gateway, and an egress rule.
  • Rules about meaning (correct, fair, read before approving) stay as reviews with evidence.

Your project now: docs/ai-policy.md, docs/rules-worksheet.md with your five new rows, and docs/control-points.md. You can now say where each rule of the policy could be checked. But so far, the control points are boxes on a picture. Next, you start Larkfield’s estate on your computer, send requests from each app, and read what the provider records.

Next: Meet Larkfield's AI estate