One door for model calls
- Lesson 10
- 40 min
- Module 4: The model gateway
Where we are
In Module 3, we registered every app, gave each one its own identity, and built a first gateway that refuses a call without a valid registration. But that gateway only says yes or no: it records nothing, limits nothing and does not look at the data. In this lesson, we build the whole path of the model gateway, with eleven checks and one usage record for every call. We start in a company's post room.
Lesson 10 of 21 · Governing AI Agents: Gateways, Registries and ControlsOne door for model calls
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
Put the whole model gateway in front of Larkfield’s apps. It is one door that checks every call in a fixed order, sends it to a model and records it. The apps keep the API they already use and change one setting.
Before you startWhat to know and to have ready
What you need to know
- What a registration holds and why an agent must be registered, from What a registration holds.
- How each agent gets its own token, and how the first gateway refuses a call without one, from One identity per agent.
- What a capability is, and how the catalog maps it to a model, from A capability catalog.
What you need on your computer
- The project
~/projects/ai-estateas Module 3 left it: the same as the snapshotend-of-m03, with its.venv, every app registered and the credentials of the five apps issued. If you start here, follow the README ofend-of-m03: “Start the estate”. - The labs repository in
~/projects/nextia-learning-labs, for the snapshotend-of-m04. - Two terminals, with the virtual environment active in both. Ports 8320 to 8322 must be free.
Tested on macOS (Apple silicon) with Python 3.12.13. The Windows (PowerShell) and Linux commands are written for you, but the course did not run them.
A post room
Section titled “A post room”Think of a company where every letter leaves through one post room. A clerk checks that the sender works there. She checks that the sender’s department may send this kind of letter, and that a letter with a bank card inside does not leave. She weighs it, picks a courier and writes one line in a ledger: who sent it, which courier, what it cost.
The staff do not change how they write letters. They only drop them in a new tray. And at the end of the month, the ledger says what each department sent and spent. The courier’s bill says only one thing: “the company sent 4,000 letters”.
The problem: a door that remembers nothing
Section titled “The problem: a door that remembers nothing”The first gateway of Module 3 is a door with a guard, but no ledger. Start the estate and send two apps through it, as you did in that module:
python -m estate uppython -m apps.replay ticket-triage --units 5 --via gatewaypython -m apps.replay support-workflow --units 2 --via gatewaypython -m provsim billThe bill, simulated from your replayed requests:
month,provider,key_id,deployment,requests,tokens_in,tokens_out,cost_usd2026-10,azure,az-key-9,chat-small,5,9594,2025,0.0019722026-10,azure,az-key-9,chat-strong,6,14776,738,0.0369322026-10,azure,az-key-9,embed-small,5,424,0,0.000008Every line now says az-key-9: the gateway’s own key. Before the gateway, the bill at least split the calls by key. Now it cannot say which app made a call at all. Ask the estate instead:
python -m estate usagepython -m estate: error: argument command: invalid choice: 'usage' (choose from init, up, down, status, serve, reset, catalog, remap, check, register, list, show, suspend, resume, retire, drain, review, review-due, events, credentials, token, config)There is no record. The first gateway also has no limit, so an agent in a loop could call a model all night. And it never looks at the data in a request. Stop the estate before you change it: python -m estate down.
The idea: one door with a fixed path
Section titled “The idea: one door with a fixed path”An AI gateway is a service that every model call of an organisation passes through. It is the post room: it checks the caller, decides where the call goes, and records it. This course calls it the model gateway, because Module 5 adds a second one for tools.
The gateway runs the same checks on every request, always in the same order. The first check that fails answers, and the request goes no further. Every request, answered or refused, gets one row in the usage store: the gateway’s ledger.
Text description of the diagram
The app sends a request with a capability, its token and its data. Check 1 asks whether the replica has its configuration. Check 2 checks the token, and check 3 the registration. Check 4 asks whether the registration lists the capability and whether the capability’s contract has what the request needs, such as tools. Check 5 checks the data class. Check 6 asks whether a kill switch stops the call. Check 7 asks whether every control can run, and what to do if one cannot. Check 8 compares the agent’s use with its limits. Check 9 looks for secrets and card numbers in the text. Check 10 sends the request to the mapping’s provider, or to a fallback. The answer carries headers that name the model. Check 11 writes one usage row, for a refusal too.
The order is not random. Each check needs what the checks before it found out. The gateway cannot look up a registration before it knows who is calling. It cannot apply an agent’s limits before it knows which agent it is. The cheap checks come first, and the one that costs money, the provider call, comes last. So a refused request costs a fraction of a millisecond and nothing on the bill.
This lesson builds the whole path. You use checks 1 to 4 and 10 to 11 now. The limits are the next lesson, the data classes and guardrails the one after, and the failure mode the last lesson of this module.
Keep the interface that the apps already use
Section titled “Keep the interface that the apps already use”The gateway speaks the same API as the provider: OpenAI-compatible chat completions and embeddings. So an app does not change its code. It changes three values. The base URL is the gateway’s, not the provider’s. The credential is its own token, not a provider key. The model name is a capability, such as strong-reasoning, not a deployment.
The support workflow of the agents course shows this. Its model calls go through one adapter class. This lesson adds a small file, resolver/platform.py, with an adapter that points the same OpenAI client at the gateway with the workflow’s token. One environment variable, RESOLVER_PROVIDER=gateway, chooses it. Nothing else in the workflow changes.
Add the lesson’s files
Section titled “Add the lesson’s files”The whole gateway arrives at once: its code for every check, the usage store and the gateway’s settings. The next lessons switch parts of it on and explain them.
-
Copy this lesson’s files from the snapshot
end-of-m04. In~/projects/ai-estate:Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m04cp $S/gateway/app.py $S/gateway/guardrails.py $S/gateway/telemetry.py gateway/cp $S/estate/usage.py $S/estate/switches.py estate/cp $S/estate/cli/usage.py estate/cli/cp $S/config/gateway.yaml config/cp $S/resolver/platform.py $S/resolver/config.py $S/resolver/__main__.py resolver/cp $S/apps/client.py apps/cp $S/tests/test_usage.py tests/Terminal window $S = "..\nextia-learning-labs\C32\snapshots\end-of-m04"Copy-Item $S\gateway\app.py, $S\gateway\guardrails.py, $S\gateway\telemetry.py gateway\Copy-Item $S\estate\usage.py, $S\estate\switches.py estate\Copy-Item $S\estate\cli\usage.py estate\cli\Copy-Item $S\config\gateway.yaml config\Copy-Item $S\resolver\platform.py, $S\resolver\config.py, $S\resolver\__main__.py resolver\Copy-Item $S\apps\client.py apps\Copy-Item $S\tests\test_usage.py tests\Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m04cp $S/gateway/app.py $S/gateway/guardrails.py $S/gateway/telemetry.py gateway/cp $S/estate/usage.py $S/estate/switches.py estate/cp $S/estate/cli/usage.py estate/cli/cp $S/config/gateway.yaml config/cp $S/resolver/platform.py $S/resolver/config.py $S/resolver/__main__.py resolver/cp $S/apps/client.py apps/cp $S/tests/test_usage.py tests/The new
apps/client.pygives each replay its own run IDs, so that a replay of the same tickets is a new run. -
Run the tests:
Terminal window python -m pytest -qThe last line says
137 passed, 1 deselected. The two new tests check the usage reports and the counters. -
Open
config/gateway.yamlin your editor. It holds the gateway’s settings. They include the failure mode of each risk tier, the budget warning, two guardrails,prompts: keep: neverand the rules for a staged change. The next lessons explain each part. -
Publish the configuration, so that the gateway gets the new settings:
Terminal window python -m estate config publish --note "the gateway's settings"Published version 2.Told 0 of 1 gateways to refresh now.The estate is stopped, so no gateway was there to tell. It reads the new version when it starts.
-
Start the estate:
Terminal window python -m estate upprovsim up (process 70161) http://127.0.0.1:8320control up (process 70163) http://127.0.0.1:8321gateway up (process 70165) http://127.0.0.1:8322Your process numbers differ. The apps’ address is the same as in Module 3:
http://127.0.0.1:8322/v1.
Worked example: one call through the whole path
Section titled “Worked example: one call through the whole path”Send one triage ticket through the gateway, and print each request:
python -m apps.replay ticket-triage --units 1 --via gateway --verboseThe output, replayed (the MLOps course’s triage request and this course’s embedding):
200 fast-small -> chat-small (gpt-6-luna-2026-09-22)200 embed -> embed-small (text-embedding-3-small)ticket-triage: 2 requests through the gateway | status {200: 2}answered by: chat-small (gpt-6-luna-2026-09-22) x1, embed-small (text-embedding-3-small) x1tokens: 1,954 in, 375 out | cost: US$0.0004The app asked for two capabilities, fast-small and embed. The gateway chose the model for each, and its answer header x-gateway-model named the model that answered. Now read the gateway’s record of the last call:
python -m estate usage --last 1ts (the time of the call)request_id 274f4fc12975486dreplica aconfig_version v2.r15agent_id ticket-triageteam servicecapability embedprovider azuredeployment embed-smallmodel_version text-embedding-3-smalldata_class personaltokens_in 62tokens_out 0cost_usd 1.24e-06gateway_ms 0.35upstream_ms 0.93recorded_ms 329.0status 200outcome okrun_id ticket-triage:T-64856:r1:6f6b3dfallback 0degraded 0Your request ID and the last part of the run ID differ, and the course leaves the time out. Read it as the post room’s ledger line:
- Who:
agent_idandteamcome from the token and the registration, not from anything that the app says about itself. - What for: the
capabilitythat the app asked for, and thedata_classof the request. The triage service sent noX-Data-Classheader, so the gateway took the most sensitive class of its registration,personal. - What answered: the
provider, thedeploymentand the exactmodel_version. When Module 2’s remap moves a capability to another model, this field shows it on the first call. - What it cost: the tokens, and
cost_usdat the catalog’s prices: 62 tokens × US$0.02 per million = US$0.00000124. - How long:
gateway_msis the gateway’s own work (0.35 ms),upstream_msthe provider’s answer time. The simulator answers at once;recorded_msis the real model’s recorded time, 329 ms. - Which configuration:
config_version v2.r15is the published version 2, with revision 15 of the registry. Module 3 explained the revisions; the last lesson of this module uses the version.
The row has no prompt and no answer text. prompts: keep: never in config/gateway.yaml is the rule: the gateway sees every request, so it is the worst place to keep their text.
A refused call is recorded too
Section titled “A refused call is recorded too”Ask for a capability that the triage service’s registration does not list. Get the service’s token first:
TOKEN=$(python -m estate token ticket-triage)curl -s -i -X POST http://127.0.0.1:8322/v1/chat/completions \ -H 'Content-Type: application/json' -H "Authorization: Bearer $TOKEN" \ -d '{"model": "strong-reasoning", "messages": [{"role": "user", "content": "Hello"}]}'$TOKEN = python -m estate token ticket-triagecurl.exe -s -i -X POST http://127.0.0.1:8322/v1/chat/completions ` -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" ` -d '{\"model\": \"strong-reasoning\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}'TOKEN=$(python -m estate token ticket-triage)curl -s -i -X POST http://127.0.0.1:8322/v1/chat/completions \ -H 'Content-Type: application/json' -H "Authorization: Bearer $TOKEN" \ -d '{"model": "strong-reasoning", "messages": [{"role": "user", "content": "Hello"}]}'HTTP/1.1 403 Forbiddenserver: uvicornx-gateway-request-id: 3ea457c0fbb643ccx-gateway-replica: ax-gateway-config: v2.r15content-length: 154content-type: application/json
{"error":{"code":"capability_not_allowed","message":"ticket-triage's registration does not list strong-reasoning.","type":"gateway","check":"capability"}}The token is valid for 10 minutes; in a new terminal, get it again. The answer names the check (capability) and says what to change: the registration, not the gateway. Run python -m estate usage --last 1 again. The row now has outcome refused, check_name capability, the same message, and no provider: the request never left the gateway. Refusals are evidence too. In Module 6, they show which agents try to do what their registration does not allow.
Explore
Where does this request stop?
Predict first: The document assistant sends a request with the header X-Data-Class: personal. Does it pass? If not, which check stops it?
- Result
- 200 chat-small
- Checks passed
- 10 / 10
- Replica
- a, b or c
200: passes every check, and chat-small (gpt-6-luna-2026-09-22) answers.
- passes
- passes, degraded or with a fallback
- stops the request
- not reached
| # | Check | What the gateway found |
|---|---|---|
| 1 | Configuration | The replica has a configuration snapshot (it polls the control plane every 2 s). |
| 2 | Identity | A valid access token for the model gateway (sub: doc-assistant). |
| 3 | Registration | doc-assistant is registered and active (low risk). |
| 4 | Capability | fast-small is in the registration and in the catalog; the contract has what the request needs. |
| 5 | Data class | No X-Data-Class header: the gateway takes internal, the most sensitive class of the registration. |
| 6 | Kill switches | No kill switch stops this agent or this capability. |
| 7 | Failure mode | Every control can run. |
| 8 | Limits | Under its limits: 60 requests a minute, its daily tokens, US$3 a month, and its team's budget. |
| 9 | Guardrails | No card number and no secret in the text. |
| 10 | Route | azure/chat-small answers: chat-small (gpt-6-luna-2026-09-22). |
| 11 | Record | One usage row: agent, team, capability, model, tokens, cost, time, outcome. No prompt text. |
Computed from the gateway’s rules; the outcomes match the measured runs.
Text description of the explorer
Choose one of five agents: the triage service, the support workflow, the document assistant, the release judge, or the weekly digest, which is not registered. Then choose what it sends: its usual request, a request for strong-reasoning, a request with tools, personal data, payment data, or a request with a key in the prompt. A table shows the 11 checks in order. Each check passes, stops the request with its code and message, or is not reached. For example, the document assistant with personal data stops at check 5, the data class: “doc-assistant is not registered for personal data”. The triage service asking for strong-reasoning stops at check 4: its registration does not list it. The workflow with tools also stops at check 4, because the contract of strong-reasoning has no tools. The weekly digest stops at check 2: it has no token, only a personal key, and the gateway answers 401 invalid_token. Payment data stops every agent at check 5. A key in the prompt stops at check 9, the guardrails.
Every app through the whole path
Section titled “Every app through the whole path”-
Send the four apps’ requests through the gateway:
Terminal window python -m apps.replay ticket-triage --units 5 --via gatewaypython -m apps.replay support-workflow --units 2 --via gatewaypython -m apps.replay doc-assistant --units 3 --via gatewaypython -m apps.replay release-judge --units 3 --via gatewayEach one ends with
status {200: …}and the same tokens and costs as in Module 1. The answers are the same recordings. Only the door changed. -
Read the usage per agent:
Terminal window python -m estate usageagent calls ok refused failed tokens in tokens out US$------------------------------------------------------------------------------------------------------support-workflow 6 6 0 0 14,776 738 0.0369release-judge 3 3 0 0 4,559 309 0.0122ticket-triage 13 12 1 0 11,972 2,400 0.0024doc-assistant 3 3 0 0 2,385 681 0.0006total 25 24 1 0 33,692 4,128 0.0521The triage line counts the worked example too: one ticket (2 calls), the refused call, and the five tickets (10 calls).
-
Read it per model, and per outcome:
Terminal window python -m estate usage --by modelpython -m estate usage --by outcome--by modelnames each model version that answered, and a line-for the refused call, which no model answered.--by outcomeshowsrefused:capability 1.
This is the question that Module 1 could not answer from the bill: which app used what, and what it cost. The answer now comes from the gateway’s own records, per agent and per team, and does not depend on keys.
The workflow changes one setting
Section titled “The workflow changes one setting”Run the support workflow’s real agent on ticket T-90103, with its model calls through the gateway:
RESOLVER_PROVIDER=gateway python -m resolver run T-90103 --variant agent_structured --model strong-reasoning$env:RESOLVER_PROVIDER = "gateway"python -m resolver run T-90103 --variant agent_structured --model strong-reasoningRESOLVER_PROVIDER=gateway python -m resolver run T-90103 --variant agent_structured --model strong-reasoningThe first lines, replayed (the agents course’s structured agent on chat-strong):
Run T-90103-agent_structured-strong-reasoning-1 | single agent with one JSON step per call (for models without tool calling) | strong-reasoningOutcome: resolve (rule W5) | 2 model call(s), 1 tool call(s) | 4647 tokens in, 266 out | US$0.01195The agent proposes a reship and a free return label for a cracked planter, and waits for a person. Approve it as the agents course did, with the same setting: RESOLVER_PROVIDER=gateway python -m resolver approve T-90103-agent_structured-strong-reasoning-1 --as agent (in PowerShell, the variable is still set). The two changes are made in the practice database. Run python -m estate usage again: support-workflow now has 2 more calls and US$0.0120 more. The workflow’s tool calls still run inside the app. The gateway did not see them, which is the next point.
What the door does not see
Section titled “What the door does not see”Tomás, an AI engineer, says what many teams think: “We have a gateway now, so we are governed.” The usage store shows why that is not true:
- It sees model calls, not the agent’s work. The workflow’s
get_ordercall and the two writes that you approved never passed the model gateway. Module 5 puts a tool gateway in front of them. - It sees only what goes through it. The document assistant embeds its documents on its own CPU, with a local model: those embeddings never reach the gateway. An app that keeps a provider key can still go around the door, as the failure case below shows.
- It sees text, not meaning. It can find a card number in a prompt. It cannot know whether a refund is right for this ticket. The guardrails lesson of this module returns to this.
What the door costs
Section titled “What the door costs”A door on every call adds time. The course team measured it on a laptop with 271 recorded triage requests, sent three times each. They went straight to the simulator, through one gateway replica, and through the front door of the last lesson of this module. The simulator answered at once (speed 0), so only the door’s own time is left.
Text description of the chart
Three bars, the median time of one request in milliseconds. Straight to the simulator: 0.61 ms. Through one gateway replica: 1.82 ms. Through the front door and a replica: 2.66 ms. So the gateway adds about 1.2 ms, and 2.0 ms with the front door. The real model took 2,922 ms at the median for the same requests.
The gateway adds about 1.2 ms at the median, and 2.0 ms with the front door (p95: 3.06 ms in all). The model itself took 2,922 ms for the same requests, more than a thousand times longer. The replica’s own work, the usage row included, was 0.52 ms. These are times on one laptop, not a capacity: a real gateway runs on other machines, with a network between them. But the order of size holds. For an app that waits seconds for a model, a gateway’s milliseconds do not decide the design.
What does decide it:
- You do not need a gateway for one app with one provider. Microsoft’s gateway guide says that a gateway serves “specific scenarios that might not be present in every workload”. It pays when several teams and agents share models, providers and budgets, as at Larkfield.
- It becomes a shared dependency. If the door fails, every app fails, unless it is built for that. The last lesson of this module is about that.
- It holds every provider key and sees every request. So it needs strict access, pinned dependencies and no prompt text in its records.
Common mistake
The usage store does not show an app's calls
Symptom: you send the document assistant’s question, and the replay works:
doc-assistant: 1 requests straight to the provider | status {200: 1}answered by: chat-small (gpt-6-luna-2026-09-22) x1tokens: 779 in, 439 out | cost: US$0.0003But python -m estate usage shows no new row for doc-assistant.
Cause: the app did not use the gateway. The first line says it: “straight to the provider”. The replay ran without --via gateway, so the app used its old provider key, sim-az-data-team-0002. In a real estate, this is an app that someone forgot to move, or a copy that still holds the old key.
Diagnose: the egress log shows the call. Its last line names the machine and the key:
python -m provsim egress...,docs-01,larkfield-ai.openai.azure.example,azure,az-key-2,200,3472docs-01 connected to the provider with az-key-2, not through the gateway.
Recover: send the app through the gateway (--via gateway; for the workflow, RESOLVER_PROVIDER=gateway). The real fix is that the old key stops working and the network lets only the gateway reach the provider. Find unregistered AI finds such calls and switches on that network rule. The rollout lesson of Module 7 plans when the old keys are revoked.
Your turn: predict which check stops each request
Section titled “Your turn: predict which check stops each request”Each request below breaks one rule. Before you send it, write down the check that stops it and the code that you expect. Use the request path diagram. Then send them, with the tokens of the right apps.
-
Get the tokens of the document assistant and the workflow:
Terminal window DOC=$(python -m estate token doc-assistant)WF=$(python -m estate token support-workflow)Terminal window $DOC = python -m estate token doc-assistant$WF = python -m estate token support-workflowTerminal window DOC=$(python -m estate token doc-assistant)WF=$(python -m estate token support-workflow) -
The document assistant sends personal data:
Terminal window curl -s -X POST http://127.0.0.1:8322/v1/chat/completions -H 'Content-Type: application/json' \-H "Authorization: Bearer $DOC" -H 'X-Data-Class: personal' \-d '{"model": "fast-small", "messages": [{"role": "user", "content": "Hello"}]}' -
The workflow asks strong-reasoning for a tool call:
Terminal window curl -s -X POST http://127.0.0.1:8322/v1/chat/completions -H 'Content-Type: application/json' \-H "Authorization: Bearer $WF" \-d '{"model": "strong-reasoning", "messages": [{"role": "user", "content": "Hello"}], "tools": [{"type": "function", "function": {"name": "get_order", "parameters": {"type": "object"}}}]}' -
The triage service asks for
vision, with its token from the worked example in$TOKEN:Terminal window curl -s -X POST http://127.0.0.1:8322/v1/chat/completions -H 'Content-Type: application/json' \-H "Authorization: Bearer $TOKEN" \-d '{"model": "vision", "messages": [{"role": "user", "content": "Hello"}]}' -
The weekly digest tries the gateway with its Fernway key:
Terminal window curl -s -X POST http://127.0.0.1:8322/v1/chat/completions -H 'Content-Type: application/json' \-H 'Authorization: Bearer sim-fw-personal-7Q2K9XW4' \-d '{"model": "fast-small", "messages": [{"role": "user", "content": "Hello"}]}' -
Count the refusals per check with
python -m estate usage --by outcome.
On Windows, write the bodies with escaped quotes, as in the worked example (not tested).
Check your predictions
{"error":{"code":"data_class_not_allowed","message":"doc-assistant is not registered for personal data.","type":"gateway","check":"data_class"}}{"error":{"code":"feature_not_in_contract","message":"strong-reasoning does not offer tool calling (its contract has no tools).","type":"gateway","check":"capability"}}{"error":{"code":"capability_not_allowed","message":"ticket-triage's registration does not list vision.","type":"gateway","check":"capability"}}{"error":{"code":"invalid_token","message":"The token is not a JWT.","type":"gateway","check":"identity"}}The document assistant stops at check 5: its registration allows only internal data. The workflow stops at check 4, before any provider sees the request. Module 2’s contract for strong-reasoning has no tools, because chat-strong refuses tools through this API. vision is in the catalog, but the triage registration does not list it. So check 4 refuses it before it finds that nothing serves vision yet. The digest stops at check 2: a provider key is not a token. The registry will not give the digest a token while its owner has left. --by outcome shows each refusal under its check, and none of them cost anything.
Knowledge check
Section titled “Knowledge check”Check your understanding
Why does the gateway check the token and the registration before the limits and the guardrails, and the provider last?
Show the answer
Each check needs what the earlier ones found out. The limits and the allowed data classes belong to an agent. So the gateway must first know which agent is calling (the token) and whether it may call (the registration). The cheap checks come first, so a refused request costs a fraction of a millisecond and nothing on the bill. The provider call is the only step that costs money, so it comes last, when every check has passed.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
After the move, the provider's bill shows every call on az-key-9. How does Larkfield now know what the triage service spent, and why can't a misconfigured app hide its calls in that number?
Show the answer
From the usage store. The gateway writes one row per request, with the agent and team, the model that answered and the cost. python -m estate usage adds them up per agent. An app cannot write another agent’s name in its rows, because the gateway takes the name from the signed token, not from the request. But the store only sees calls that pass the gateway. An app that still uses an old provider key is not in it, and the egress log or the bill must find it.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Practical check: send one question of the document assistant through the gateway, then show its usage row. Which fields tell you who called, which model answered and what it cost?
Show the answer
python -m apps.replay doc-assistant –units 1 –via gateway, then python -m estate usage –last 1. Who: agent_id doc-assistant and team data. Which model: provider azure, deployment chat-small and model_version gpt-6-luna-2026-09-22. What it cost: tokens_in, tokens_out and cost_usd, at the catalog’s prices. The data class is internal, the most sensitive class of its registration, because the request sent no X-Data-Class header. The row has no prompt text.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- An AI gateway is one door for every model call. Larkfield’s model gateway runs 11 checks in a fixed order, and the first check that fails answers with its own code.
- Every request, answered or refused, gets one usage row: who called, what for, which model answered, the tokens, the cost and the time. No prompt text is kept.
- Apps keep the OpenAI-compatible API and change three values: the base URL, their credential and the model name. The workflow changes one setting,
RESOLVER_PROVIDER=gateway. - The gateway added about 1.2 ms at the median on the course team’s laptop, against 2.9 s for the model. Its real costs are elsewhere: a shared dependency, and the keys it holds.
- The gateway sees model calls only, and only the calls that pass through it.
Your project now: the whole gateway in gateway/, the usage store in estate/usage.py, config/gateway.yaml, the workflow’s gateway adapter in resolver/platform.py, and 137 tests that pass. You can now see who called which model and what it cost. But nothing stops an agent that calls a model in a loop: the record only shows the cost afterwards. Next, you set the limits that stop it.