Map capabilities to models
- Lesson 5
- 40 min
- Module 2: Ask for capabilities, not models
Where we are
In the last lesson, we gave each capability a contract: features, a quality floor, a latency, data classes and a price band. But a contract does not say which model keeps the promise, where it runs, or what happens when that provider is down. In this lesson, we map each capability to a model and a provider, add a fallback and data rules, record the model that answered, and let a consumer pin an exact model when it must. Back to the car rental.
Lesson 5 of 21 · Governing AI Agents: Gateways, Registries and ControlsMap capabilities to models
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
Decide which model and provider serve each capability, which provider takes over when the first one fails, and which data each provider may receive. Then record, on every call, the model that really answered, and let one consumer keep an exact model until a stated date.
Before you startWhat to know and to have ready
What you need to know
- Capabilities and their contracts, from A capability catalog.
- Data classes:
public,internal,personalandpayment, from the same lesson. - HTTP status codes:
200is an answer,4xxmeans the request is wrong,5xxmeans the server failed,429means “too many requests”.
What you need on your computer
- Your project after the last lesson, with
config/and theestate catalogcommand. - The estate running (
python -m estate up) in your terminal, with the virtual environment active.
Tested on macOS (Apple silicon) with Python 3.12.13; the Windows and Linux commands are not tested. The model answers are replayed. The outage is simulated, and so is the second Azure region: it answers with the same recordings as the first.
Back at the car rental
Section titled “Back at the car rental”You booked “mid-size, five seats, automatic, with a tow bar”. The company now has to keep that promise. Its plan has four parts.
First, which car: at the airport branch, mid-size means a Hyundai with a tow bar. Second, a backup: when the airport branch has no car left, the branch in town gives you the same model. Third, rules about where a car may go: some cars may not cross the border. A customer who drives abroad gets only the cars that may. Fourth, a record: the contract says which car you got, with its plate. If a light breaks on the trip, everyone knows which car it was.
One customer is special. She drives with hand controls, and the company fitted them to one car. She asks to keep that exact car until her contract ends, even if the company changes its mid-size cars. The company agrees, writes the end date down, and calls her a month before it.
The problem: the contract does not route a call
Section titled “The problem: the contract does not route a call”Kwame’s catalog says what fast-small promises. It does not say where a request for fast-small goes. If nobody decides that in one place, each app decides for itself, and they decide differently.
Larkfield saw what that means in Module 1. The triage service and the workflow each held the same shared key and called East US 2 directly. If that region fails, each app needs its own retry code and its own idea of a second region. And nothing records which model answered. The bill names a key and a deployment, never the app or the model version. Only the app printed gpt-6-luna-2026-09-22, and only on the screen.
Data rules have the same problem. Mei knows which contract allows customer data, but that knowledge sits in her head and in a document. Nothing stops an app from sending a customer’s message to a provider without a contract. That is exactly what the weekly digest did, with Fernway.
The idea: a mapping with fallbacks, data rules and a record
Section titled “The idea: a mapping with fallbacks, data rules and a record”A model mapping says, for one capability, which places can serve it and in which order:
- a primary: the provider and deployment that serve it today, such as
azure / chat-small; - fallbacks: the next places to try, in order, such as the same deployment in a second region;
- settings that the model needs.
paramsgo with every request.params_if_toolsgo only with requests that have tools, for examplereasoning_effort: nonefor chat-small.
The data rules sit with the providers. Each provider in config/providers.yaml lists the data classes that its contract allows. Routing keeps only the providers that may receive the request’s data class. A provider that is not in the file is never called.
Two rules decide when to use a fallback. Fall back on 429, on 5xx and on a time-out: the provider is busy or broken, and another one may answer. Never fall back on another 4xx. A 4xx says that the request itself is wrong: a missing setting, a bad key, a field the model refuses. The next provider would refuse it too. Or worse, it would accept a request that broke a rule.
Finally, every answer names the model that produced it. In the OpenAI-compatible API, the answer’s model field holds the exact version, such as gpt-6-luna-2026-09-22. The platform records it with the provider and the deployment. So “which model answered ticket T-90103?” has an answer later.
Text description of the diagram
A request asks for a capability, for example fast-small, with the data class personal. If the capability has no mapping, as vision today, nothing serves it. Otherwise the targets are the primary and then the fallbacks, but only at providers whose contract allows the data class. If the app pins a deployment, only that deployment is kept, at each of those providers. For each target in turn, the platform puts the deployment’s name in the request. It adds the settings the model needs, such as reasoning_effort none for a request with tools. If the provider answers 200, the platform records the model that answered: provider, deployment and model version. If it answers 429 or 5xx, or does not answer in time, the platform tries the next target. When none is left, no provider answered. If it answers any other 4xx, the request itself is wrong, and the platform stops without a fallback.
A tiny worked example: route one request by hand
Section titled “A tiny worked example: route one request by hand”In Module 3, the gateway does this for every call. Here you do it for one call, with about 35 lines of Python, so that every step is visible.
-
Copy this lesson’s test file from the snapshot
end-of-m02, then run the tests:Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m02cp $S/tests/test_catalog.py tests/python -m pytest -qTerminal window $S = "..\nextia-learning-labs\C32\snapshots\end-of-m02"Copy-Item $S\tests\test_catalog.py tests\python -m pytest -qTerminal window S=../nextia-learning-labs/C32/snapshots/end-of-m02cp $S/tests/test_catalog.py tests/python -m pytest -qThe last line says
100 passed, 1 deselected. The 9 new tests check the contracts, the mappings, the data rules and the pins. -
Print the card of
fast-small:Terminal window python -m estate catalog show fast-smallThe last two lines are the mapping:
serves today: azure/chat-small (gpt-6-luna-2026-09-22) + with tools {'reasoning_effort': 'none'}fallback: azure-canada/chat-small -
Create the file
try_route.pyin the project folder, with this content. It is a practice file, not part of the project; you delete it at the end of the lesson.try_route.py """Route one recorded request by capability, by hand (a practice file: not part of the project)."""import argparseimport httpxfrom apps.replay import load_streamfrom estate import catalogSTREAMS = {"ticket-triage": "ticket-triage", "release-judge": "release-judge__chat-strong"}ap = argparse.ArgumentParser()ap.add_argument("app", choices=STREAMS)ap.add_argument("capability")ap.add_argument("--data-class", default="personal")ap.add_argument("--catalog", help="another catalog file")ap.add_argument("--pin", help="a deployment that this app pins")a = ap.parse_args()# The app's first recorded request. It now asks for a capability, not a model.body = dict(load_stream(STREAMS[a.app])[0]["body"], model=a.capability)cat = catalog.load_catalog(a.catalog)pin = {"deployment": a.pin} if a.pin else Nonetargets = catalog.targets(cat, a.capability, a.data_class, pin)print("targets:", [t.label() for t in targets] or "none")for t in targets:sent = t.body_for(body)r = httpx.post(f"http://127.0.0.1:8320/{t.provider}/v1/chat/completions", json=sent,headers={"Authorization": "Bearer sim-az-service-team-0001"})print(f" {t.label()} -> HTTP {r.status_code}")if r.status_code == 200:print(f"answered by: {t.label()} ({r.json()['model']})")breakif r.status_code != 429 and r.status_code < 500:print(" the request itself was refused: no fallback")breakelse:print("no provider answered")catalog.targets()is the routing of the diagram: the mapping, filtered by the data rules and the pin.body_for()puts the deployment’s name in the request and adds the settings. -
Route the triage service’s first recorded request, which now asks for
fast-small:Terminal window python try_route.py ticket-triage fast-smalltargets: ['azure/chat-small', 'azure-canada/chat-small']azure/chat-small -> HTTP 200answered by: azure/chat-small (gpt-6-luna-2026-09-22)
Read the three lines. targets: the primary and the fallback, both allowed for personal data. azure/chat-small -> HTTP 200: the platform replaced fast-small with the deployment name chat-small. The triage request has no tools, so it added no setting. The changed body is exactly the body that the evaluation course recorded, so the simulator found its answer. answered by: the model version comes from the answer itself, not from the catalog. If Azure changed the model behind chat-small, this line would show it.
Fallbacks and data rules, on purpose
Section titled “Fallbacks and data rules, on purpose”Now make the primary fail. The simulator can stop a provider on purpose, so you can practise an outage.
-
Stop the simulated East US 2 region:
Terminal window python -m provsim control azure downControls now: [{"provider": "azure", "deployment": null, "mode": "down", "seconds": 5.0, "rate": 0.5, "requests_left": null}] (simulated) -
Route the same request again:
Terminal window python try_route.py ticket-triage fast-smalltargets: ['azure/chat-small', 'azure-canada/chat-small']azure/chat-small -> HTTP 503azure-canada/chat-small -> HTTP 200answered by: azure-canada/chat-small (gpt-6-luna-2026-09-22) -
Start the region again:
Terminal window python -m provsim control azure normal -
Route the request with the data class
payment:Terminal window python try_route.py ticket-triage fast-small --data-class paymenttargets: noneno provider answered
The 503 sent the call to the fallback, and the same model version answered from the second region. So the record says azure-canada, not azure. The model is the same; the place is not, and for personal data the place can matter. payment found no target at all, because no provider in providers.yaml lists it. Card numbers never reach a model, and nobody had to remember that rule.
Here the app chose its own data class with --data-class. A real app could choose public for a customer’s message. In Module 4, the gateway uses the registration’s most sensitive class when a request does not say. It also refuses a class that the registration does not allow.
Larkfield: Kwame’s mapping and Mei’s data rules
Section titled “Larkfield: Kwame’s mapping and Mei’s data rules”Kwame owns the mapping. Mei owns the data rules. They are in two files, so that each owner changes only their own file.
| Provider | Region | Contract | Data classes it may receive |
|---|---|---|---|
azure |
East US 2 | Larkfield’s Azure subscription, with a data processing agreement (assumption: it covers personal data) | public, internal, personal |
azure-canada |
Canada East (simulated second region) | the same subscription | public, internal, personal |
local |
Larkfield’s own hardware (gemma3:4b) | none needed: the data does not leave Larkfield | public, internal, personal |
| Fernway | – | none | not listed: never called |
Every capability’s primary is in azure, and every fallback is the same deployment in azure-canada. That choice is important. A fallback to a different model would be a remap that happens without warning, during an outage, when nobody watches. The second region gives the same model, so the answers stay the same. Microsoft’s guidance on a gateway in front of several deployments describes this pattern of several regions behind one gateway.
Sam asks: “Why is local in the file at all? No capability maps to it.” Kwame answers: “Because it is the cheapest place. Someone will propose it. Its data rule is already decided: data on our own hardware stays at Larkfield.” The last lesson of this module tests that proposal.
Common mistake
A mapping without the setting that tools need
Symptom: after a tidy-up of catalog.yaml, python -m estate catalog check prints this line and exits with code 1:
fast-small -> azure/chat-small: tools on chat-small needs reasoning_effort: none in the mappingThe diff shows what changed:
< primary: {provider: azure, deployment: chat-small, params_if_tools: {reasoning_effort: none}}---> primary: {provider: azure, deployment: chat-small}Cause: someone removed params_if_tools because “the apps do not send it anyway”. That is the point: the apps do not send it, so the mapping must add it. models.yaml says that chat-small needs reasoning_effort: none for tools, and the contract of fast-small promises tools. Without the setting, every tool call through fast-small would get the HTTP 400 that you saw in the last lesson.
Diagnose: the line names the capability, the target, the feature and the setting. Compare the mapping with the needs: line of the deployment in models.yaml.
Recover: put params_if_tools: {reasoning_effort: none} back on every target of fast-small that uses chat-small, the fallback too. Run catalog check again. The tests catch this mistake as well: test_a_mapping_must_add_the_setting_that_a_feature_needs.
Pin an exact model
Section titled “Pin an exact model”Priya owns the release judge. It decides whether a new version of the support replies may ship. Its evidence is 99 of 101 replies judged the same way as the code checks, measured on chat-strong, model version gpt-6.1-sol-2026-09-29. If the model behind strong-reasoning changes, her evidence is about a model that no longer answers.
So the judge may pin the model: keep one deployment and one model version for one capability, until a retirement date (retire_by). In the registration that you write in Module 3, a pin looks like this. The date is an example date, not one that a provider published:
pins: strong-reasoning: {deployment: chat-strong, model_version: gpt-6.1-sol-2026-09-29, retire_by: 2027-03-31}A pin has three parts that work together. The routing keeps the pinned deployment, also after a remap. The version check refuses an answer from another model version, with pinned_model_changed, which the gateway does in Module 4. And the reminder: the review list in Module 3 names every pin that retires within 60 days, so this one appears from 2027-01-30. Before the date, Priya runs her evaluation on the new model. Providers retire model versions on their own dates; check your provider’s model retirement page.
Text description of the diagram
After a remap, the mapping of strong-reasoning points at azure chat-small. The support workflow has no pin, so it now gets chat-small, and the model gpt-6-luna-2026-09-22 answers. The release judge pins chat-strong, version gpt-6.1-sol-2026-09-29, until 2027-03-31, so it still gets chat-strong. Below, a time line: from 2027-01-30, sixty days before the date, the review list names the pin. On 2027-03-31 the pin ends, and Priya runs her evaluation on the new model before that.
Your turn: a practice remap and a pin
Section titled “Your turn: a practice remap and a pin”Change one condition: the mapping of strong-reasoning. Work on a copy of the catalog. Predict first: after you move strong-reasoning to chat-small, which model answers the release judge’s first request without a pin, and with --pin chat-strong? And does catalog check accept the copy?
-
Copy the catalog:
cp config/catalog.yaml my-catalog.yaml(Windows:Copy-Item config\catalog.yaml my-catalog.yaml). -
In
my-catalog.yaml, understrong-reasoning:andmapping:, changedeployment: chat-strongtodeployment: chat-smallin two places: the primary and the fallback. -
Check the copy:
python -m estate catalog check --file my-catalog.yaml. -
Route the judge’s request without a pin, then with it:
Terminal window python try_route.py release-judge strong-reasoning --catalog my-catalog.yamlpython try_route.py release-judge strong-reasoning --catalog my-catalog.yaml --pin chat-strong -
Delete the practice files:
rm my-catalog.yaml try_route.py(Windows:Remove-Item my-catalog.yaml, try_route.py).
Check your prediction
The catalog is consistent: every mapping keeps its contract.targets: ['azure/chat-small', 'azure-canada/chat-small'] azure/chat-small -> HTTP 200answered by: azure/chat-small (gpt-6-luna-2026-09-22)targets: ['azure/chat-strong', 'azure-canada/chat-strong'] azure/chat-strong -> HTTP 200answered by: azure/chat-strong (gpt-6.1-sol-2026-09-29)Without the pin, chat-small answers. The evaluation course recorded the judge’s requests on both models, so the simulator has this answer. With the pin, chat-strong still answers, in both regions. And the check accepts the copy: chat-small has every feature of the contract, and it is far below the price band. The check knows features, data and prices. It knows nothing about quality. Whether chat-small is good enough for the judge and the workflow is a question that only their evaluations can answer.
When to use each part, and what it costs
Section titled “When to use each part, and what it costs”A fallback in a second region pays when an outage of one region would stop important work, like the 822 triage tickets a day. It costs a second deployment and its quota, and the data is processed in another place. Check that the contract allows that place for each data class.
A fallback to a different model is a remap in disguise. Use one only if every consumer’s evaluation passed on that model, exactly as for a planned remap. Otherwise, a ticket that goes to a person during an outage is the safer answer.
Data rules belong with the providers’ contracts, in one file that Mei owns. They cost almost nothing to check on each call. The hard part is keeping the file true when a contract changes.
A pin is an exception, so it has an end date. It costs the platform a second model to keep available. It also delays the consumer’s move to the new model. Pin only when the consumer’s evidence depends on that exact model, as the judge’s does. Never pin “just in case”.
The record of the model that answered costs one field per call. It is what lets you find out later which calls a model change touched.
Knowledge check
Section titled “Knowledge check”Check your understanding
East US 2 answers 503 to a triage request. A minute later, it answers 400 to a workflow request with a setting the model refuses. What should the platform do in each case, and why?
Show the answer
For the 503, try the next target: the same chat-small deployment in the second region. The provider failed, and another place can answer the same request. For the 400, stop and give the error to the app. A 4xx says the request itself is wrong; the second region has the same model and would refuse it too. Retrying it there only doubles the failed calls. And a different provider might accept a request that broke a rule.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Kwame wants gemma3:4b on Larkfield's own hardware as a second fallback for strong-reasoning, 'so that an outage of both regions never stops the workflow'. What is wrong with that plan?
Show the answer
A fallback to another model is a remap that happens during an outage, without a gate and when nobody watches. gemma3:4b is not the model whose evidence the workflow and the judge have; its answers differ. It also cannot call tools, which matters for any consumer that needs them. Use it as a fallback only if every consumer’s evaluation passes on it. Otherwise, when both regions are down, the safer answer is to send the work to a person.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Why does a pin need a retirement date and a reminder, and not only the deployment name?
Show the answer
A pin is an exception to the mapping, and providers retire model versions. Without a date, the pin stays until the old model disappears, and then the consumer fails on the day the provider removes it. With retire_by, everyone knows when the exception ends. The reminder 60 days before gives the owner time to run the evaluation on the new model and move. Or the owner asks for a new pin, with new evidence.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A model mapping gives each capability a primary, ordered fallbacks and the settings each model needs (
params,params_if_tools). - Data rules: each provider lists the data classes its contract allows; routing skips the others, and
paymentreaches no model. - Fall back on 429, 5xx and time-outs; never on another 4xx. Larkfield’s fallbacks are the same model in a second region, so a fallback is never a silent remap.
- Record the model that answered from the answer itself:
azure-canada/chat-small (gpt-6-luna-2026-09-22). - A pin keeps one model for one consumer until
retire_by, with a reminder 60 days before.
Your project now has the catalog’s tests: 100 passed. You can route a call and keep a pin. But your practice remap passed catalog check without one question about quality, and a real remap changes every consumer of the capability at once. Next, you treat a remap as a release, with each consumer’s own evaluation as the gate.
Next: Change a mapping safely