Skip to content
Français

Map capabilities to models

  • Lesson 5
  • 40 min
  • Module 2: Ask for capabilities, not models

Where we are

In the last lesson, we gave each capability a contract: features, a quality floor, a latency, data classes and a price band. But a contract does not say which model keeps the promise, where it runs, or what happens when that provider is down. In this lesson, we map each capability to a model and a provider, add a fallback and data rules, record the model that answered, and let a consumer pin an exact model when it must. Back to the car rental.

What you will be able to do

Decide which model and provider serve each capability, which provider takes over when the first one fails, and which data each provider may receive. Then record, on every call, the model that really answered, and let one consumer keep an exact model until a stated date.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

  • Capabilities and their contracts, from A capability catalog.
  • Data classes: public, internal, personal and payment, from the same lesson.
  • HTTP status codes: 200 is an answer, 4xx means the request is wrong, 5xx means the server failed, 429 means “too many requests”.

What you need on your computer

  • Your project after the last lesson, with config/ and the estate catalog command.
  • The estate running (python -m estate up) in your terminal, with the virtual environment active.

Tested on macOS (Apple silicon) with Python 3.12.13; the Windows and Linux commands are not tested. The model answers are replayed. The outage is simulated, and so is the second Azure region: it answers with the same recordings as the first.

You booked “mid-size, five seats, automatic, with a tow bar”. The company now has to keep that promise. Its plan has four parts.

First, which car: at the airport branch, mid-size means a Hyundai with a tow bar. Second, a backup: when the airport branch has no car left, the branch in town gives you the same model. Third, rules about where a car may go: some cars may not cross the border. A customer who drives abroad gets only the cars that may. Fourth, a record: the contract says which car you got, with its plate. If a light breaks on the trip, everyone knows which car it was.

One customer is special. She drives with hand controls, and the company fitted them to one car. She asks to keep that exact car until her contract ends, even if the company changes its mid-size cars. The company agrees, writes the end date down, and calls her a month before it.

The problem: the contract does not route a call

Section titled “The problem: the contract does not route a call”

Kwame’s catalog says what fast-small promises. It does not say where a request for fast-small goes. If nobody decides that in one place, each app decides for itself, and they decide differently.

Larkfield saw what that means in Module 1. The triage service and the workflow each held the same shared key and called East US 2 directly. If that region fails, each app needs its own retry code and its own idea of a second region. And nothing records which model answered. The bill names a key and a deployment, never the app or the model version. Only the app printed gpt-6-luna-2026-09-22, and only on the screen.

Data rules have the same problem. Mei knows which contract allows customer data, but that knowledge sits in her head and in a document. Nothing stops an app from sending a customer’s message to a provider without a contract. That is exactly what the weekly digest did, with Fernway.

The idea: a mapping with fallbacks, data rules and a record

Section titled “The idea: a mapping with fallbacks, data rules and a record”

A model mapping says, for one capability, which places can serve it and in which order:

  • a primary: the provider and deployment that serve it today, such as azure / chat-small;
  • fallbacks: the next places to try, in order, such as the same deployment in a second region;
  • settings that the model needs. params go with every request. params_if_tools go only with requests that have tools, for example reasoning_effort: none for chat-small.

The data rules sit with the providers. Each provider in config/providers.yaml lists the data classes that its contract allows. Routing keeps only the providers that may receive the request’s data class. A provider that is not in the file is never called.

Two rules decide when to use a fallback. Fall back on 429, on 5xx and on a time-out: the provider is busy or broken, and another one may answer. Never fall back on another 4xx. A 4xx says that the request itself is wrong: a missing setting, a bad key, a field the model refuses. The next provider would refuse it too. Or worse, it would accept a request that broke a rule.

Finally, every answer names the model that produced it. In the OpenAI-compatible API, the answer’s model field holds the exact version, such as gpt-6-luna-2026-09-22. The platform records it with the provider and the deployment. So “which model answered ticket T-90103?” has an answer later.

A flow for one request. A request asks for a capability, fast-small, with the data class personal. Does the capability have a mapping? No, as for vision: nothing serves it, no call. Yes: the targets are the primary, then the fallbacks, but only providers that may receive the data class. Does the app pin a deployment? If yes, keep the pinned deployment at each of those providers. Then, for the next target, use its deployment name and add the settings it needs, with tools reasoning_effort none. The provider's answer: 200, record the model that answered, the provider, deployment and version. 429, 5xx or a time-out: is there another target? Yes, try it; no, no provider answered. Another 4xx: the request is wrong, stop, no fallback.
How the platform routes one request by capability. A fallback happens only when the provider failed, never when the request was wrong.
Text description of the diagram

A request asks for a capability, for example fast-small, with the data class personal. If the capability has no mapping, as vision today, nothing serves it. Otherwise the targets are the primary and then the fallbacks, but only at providers whose contract allows the data class. If the app pins a deployment, only that deployment is kept, at each of those providers. For each target in turn, the platform puts the deployment’s name in the request. It adds the settings the model needs, such as reasoning_effort none for a request with tools. If the provider answers 200, the platform records the model that answered: provider, deployment and model version. If it answers 429 or 5xx, or does not answer in time, the platform tries the next target. When none is left, no provider answered. If it answers any other 4xx, the request itself is wrong, and the platform stops without a fallback.

A tiny worked example: route one request by hand

Section titled “A tiny worked example: route one request by hand”

In Module 3, the gateway does this for every call. Here you do it for one call, with about 35 lines of Python, so that every step is visible.

  1. Copy this lesson’s test file from the snapshot end-of-m02, then run the tests:

    Terminal window
    S=../nextia-learning-labs/C32/snapshots/end-of-m02
    cp $S/tests/test_catalog.py tests/
    python -m pytest -q

    The last line says 100 passed, 1 deselected. The 9 new tests check the contracts, the mappings, the data rules and the pins.

  2. Print the card of fast-small:

    Terminal window
    python -m estate catalog show fast-small

    The last two lines are the mapping:

    serves today: azure/chat-small (gpt-6-luna-2026-09-22) + with tools {'reasoning_effort': 'none'}
    fallback: azure-canada/chat-small
  3. Create the file try_route.py in the project folder, with this content. It is a practice file, not part of the project; you delete it at the end of the lesson.

    try_route.py
    """Route one recorded request by capability, by hand (a practice file: not part of the project)."""
    import argparse
    import httpx
    from apps.replay import load_stream
    from estate import catalog
    STREAMS = {"ticket-triage": "ticket-triage", "release-judge": "release-judge__chat-strong"}
    ap = argparse.ArgumentParser()
    ap.add_argument("app", choices=STREAMS)
    ap.add_argument("capability")
    ap.add_argument("--data-class", default="personal")
    ap.add_argument("--catalog", help="another catalog file")
    ap.add_argument("--pin", help="a deployment that this app pins")
    a = ap.parse_args()
    # The app's first recorded request. It now asks for a capability, not a model.
    body = dict(load_stream(STREAMS[a.app])[0]["body"], model=a.capability)
    cat = catalog.load_catalog(a.catalog)
    pin = {"deployment": a.pin} if a.pin else None
    targets = catalog.targets(cat, a.capability, a.data_class, pin)
    print("targets:", [t.label() for t in targets] or "none")
    for t in targets:
    sent = t.body_for(body)
    r = httpx.post(f"http://127.0.0.1:8320/{t.provider}/v1/chat/completions", json=sent,
    headers={"Authorization": "Bearer sim-az-service-team-0001"})
    print(f" {t.label()} -> HTTP {r.status_code}")
    if r.status_code == 200:
    print(f"answered by: {t.label()} ({r.json()['model']})")
    break
    if r.status_code != 429 and r.status_code < 500:
    print(" the request itself was refused: no fallback")
    break
    else:
    print("no provider answered")

    catalog.targets() is the routing of the diagram: the mapping, filtered by the data rules and the pin. body_for() puts the deployment’s name in the request and adds the settings.

  4. Route the triage service’s first recorded request, which now asks for fast-small:

    Terminal window
    python try_route.py ticket-triage fast-small
    targets: ['azure/chat-small', 'azure-canada/chat-small']
    azure/chat-small -> HTTP 200
    answered by: azure/chat-small (gpt-6-luna-2026-09-22)

Read the three lines. targets: the primary and the fallback, both allowed for personal data. azure/chat-small -> HTTP 200: the platform replaced fast-small with the deployment name chat-small. The triage request has no tools, so it added no setting. The changed body is exactly the body that the evaluation course recorded, so the simulator found its answer. answered by: the model version comes from the answer itself, not from the catalog. If Azure changed the model behind chat-small, this line would show it.

Now make the primary fail. The simulator can stop a provider on purpose, so you can practise an outage.

  1. Stop the simulated East US 2 region:

    Terminal window
    python -m provsim control azure down
    Controls now: [{"provider": "azure", "deployment": null, "mode": "down", "seconds": 5.0, "rate": 0.5, "requests_left": null}] (simulated)
  2. Route the same request again:

    Terminal window
    python try_route.py ticket-triage fast-small
    targets: ['azure/chat-small', 'azure-canada/chat-small']
    azure/chat-small -> HTTP 503
    azure-canada/chat-small -> HTTP 200
    answered by: azure-canada/chat-small (gpt-6-luna-2026-09-22)
  3. Start the region again:

    Terminal window
    python -m provsim control azure normal
  4. Route the request with the data class payment:

    Terminal window
    python try_route.py ticket-triage fast-small --data-class payment
    targets: none
    no provider answered

The 503 sent the call to the fallback, and the same model version answered from the second region. So the record says azure-canada, not azure. The model is the same; the place is not, and for personal data the place can matter. payment found no target at all, because no provider in providers.yaml lists it. Card numbers never reach a model, and nobody had to remember that rule.

Here the app chose its own data class with --data-class. A real app could choose public for a customer’s message. In Module 4, the gateway uses the registration’s most sensitive class when a request does not say. It also refuses a class that the registration does not allow.

Larkfield: Kwame’s mapping and Mei’s data rules

Section titled “Larkfield: Kwame’s mapping and Mei’s data rules”

Kwame owns the mapping. Mei owns the data rules. They are in two files, so that each owner changes only their own file.

Provider Region Contract Data classes it may receive
azure East US 2 Larkfield’s Azure subscription, with a data processing agreement (assumption: it covers personal data) public, internal, personal
azure-canada Canada East (simulated second region) the same subscription public, internal, personal
local Larkfield’s own hardware (gemma3:4b) none needed: the data does not leave Larkfield public, internal, personal
Fernway – none not listed: never called

Every capability’s primary is in azure, and every fallback is the same deployment in azure-canada. That choice is important. A fallback to a different model would be a remap that happens without warning, during an outage, when nobody watches. The second region gives the same model, so the answers stay the same. Microsoft’s guidance on a gateway in front of several deployments describes this pattern of several regions behind one gateway.

Sam asks: “Why is local in the file at all? No capability maps to it.” Kwame answers: “Because it is the cheapest place. Someone will propose it. Its data rule is already decided: data on our own hardware stays at Larkfield.” The last lesson of this module tests that proposal.

Common mistake

A mapping without the setting that tools need

Symptom: after a tidy-up of catalog.yaml, python -m estate catalog check prints this line and exits with code 1:

fast-small -> azure/chat-small: tools on chat-small needs reasoning_effort: none in the mapping

The diff shows what changed:

< primary: {provider: azure, deployment: chat-small, params_if_tools: {reasoning_effort: none}}
---
> primary: {provider: azure, deployment: chat-small}

Cause: someone removed params_if_tools because “the apps do not send it anyway”. That is the point: the apps do not send it, so the mapping must add it. models.yaml says that chat-small needs reasoning_effort: none for tools, and the contract of fast-small promises tools. Without the setting, every tool call through fast-small would get the HTTP 400 that you saw in the last lesson.

Diagnose: the line names the capability, the target, the feature and the setting. Compare the mapping with the needs: line of the deployment in models.yaml.

Recover: put params_if_tools: {reasoning_effort: none} back on every target of fast-small that uses chat-small, the fallback too. Run catalog check again. The tests catch this mistake as well: test_a_mapping_must_add_the_setting_that_a_feature_needs.

Priya owns the release judge. It decides whether a new version of the support replies may ship. Its evidence is 99 of 101 replies judged the same way as the code checks, measured on chat-strong, model version gpt-6.1-sol-2026-09-29. If the model behind strong-reasoning changes, her evidence is about a model that no longer answers.

So the judge may pin the model: keep one deployment and one model version for one capability, until a retirement date (retire_by). In the registration that you write in Module 3, a pin looks like this. The date is an example date, not one that a provider published:

pins:
strong-reasoning: {deployment: chat-strong, model_version: gpt-6.1-sol-2026-09-29, retire_by: 2027-03-31}

A pin has three parts that work together. The routing keeps the pinned deployment, also after a remap. The version check refuses an answer from another model version, with pinned_model_changed, which the gateway does in Module 4. And the reminder: the review list in Module 3 names every pin that retires within 60 days, so this one appears from 2027-01-30. Before the date, Priya runs her evaluation on the new model. Providers retire model versions on their own dates; check your provider’s model retirement page.

Two apps ask for strong-reasoning after a remap moved its mapping to azure chat-small. The support workflow has no pin: the new mapping sends it to chat-small, and gpt-6-luna-2026-09-22 answers. The release judge pins chat-strong, model version gpt-6.1-sol-2026-09-29, with retire_by 2027-03-31: the pin sends it to chat-strong, and gpt-6.1-sol-2026-09-29 answers. A time line below: from 2027-01-30, 60 days before, the review list names the pin; on 2027-03-31 the pin ends, and Priya re-runs her evaluation first.
A pin after a remap: one capability, two consumers, two models, and a date when the exception ends.
Text description of the diagram

After a remap, the mapping of strong-reasoning points at azure chat-small. The support workflow has no pin, so it now gets chat-small, and the model gpt-6-luna-2026-09-22 answers. The release judge pins chat-strong, version gpt-6.1-sol-2026-09-29, until 2027-03-31, so it still gets chat-strong. Below, a time line: from 2027-01-30, sixty days before the date, the review list names the pin. On 2027-03-31 the pin ends, and Priya runs her evaluation on the new model before that.

Change one condition: the mapping of strong-reasoning. Work on a copy of the catalog. Predict first: after you move strong-reasoning to chat-small, which model answers the release judge’s first request without a pin, and with --pin chat-strong? And does catalog check accept the copy?

  1. Copy the catalog: cp config/catalog.yaml my-catalog.yaml (Windows: Copy-Item config\catalog.yaml my-catalog.yaml).

  2. In my-catalog.yaml, under strong-reasoning: and mapping:, change deployment: chat-strong to deployment: chat-small in two places: the primary and the fallback.

  3. Check the copy: python -m estate catalog check --file my-catalog.yaml.

  4. Route the judge’s request without a pin, then with it:

    Terminal window
    python try_route.py release-judge strong-reasoning --catalog my-catalog.yaml
    python try_route.py release-judge strong-reasoning --catalog my-catalog.yaml --pin chat-strong
  5. Delete the practice files: rm my-catalog.yaml try_route.py (Windows: Remove-Item my-catalog.yaml, try_route.py).

Check your prediction
The catalog is consistent: every mapping keeps its contract.
targets: ['azure/chat-small', 'azure-canada/chat-small']
azure/chat-small -> HTTP 200
answered by: azure/chat-small (gpt-6-luna-2026-09-22)
targets: ['azure/chat-strong', 'azure-canada/chat-strong']
azure/chat-strong -> HTTP 200
answered by: azure/chat-strong (gpt-6.1-sol-2026-09-29)

Without the pin, chat-small answers. The evaluation course recorded the judge’s requests on both models, so the simulator has this answer. With the pin, chat-strong still answers, in both regions. And the check accepts the copy: chat-small has every feature of the contract, and it is far below the price band. The check knows features, data and prices. It knows nothing about quality. Whether chat-small is good enough for the judge and the workflow is a question that only their evaluations can answer.

A fallback in a second region pays when an outage of one region would stop important work, like the 822 triage tickets a day. It costs a second deployment and its quota, and the data is processed in another place. Check that the contract allows that place for each data class.

A fallback to a different model is a remap in disguise. Use one only if every consumer’s evaluation passed on that model, exactly as for a planned remap. Otherwise, a ticket that goes to a person during an outage is the safer answer.

Data rules belong with the providers’ contracts, in one file that Mei owns. They cost almost nothing to check on each call. The hard part is keeping the file true when a contract changes.

A pin is an exception, so it has an end date. It costs the platform a second model to keep available. It also delays the consumer’s move to the new model. Pin only when the consumer’s evidence depends on that exact model, as the judge’s does. Never pin “just in case”.

The record of the model that answered costs one field per call. It is what lets you find out later which calls a model change touched.

Check your understanding

East US 2 answers 503 to a triage request. A minute later, it answers 400 to a workflow request with a setting the model refuses. What should the platform do in each case, and why?

Show the answer

For the 503, try the next target: the same chat-small deployment in the second region. The provider failed, and another place can answer the same request. For the 400, stop and give the error to the app. A 4xx says the request itself is wrong; the second region has the same model and would refuse it too. Retrying it there only doubles the failed calls. And a different provider might accept a request that broke a rule.

Check your understanding

Kwame wants gemma3:4b on Larkfield's own hardware as a second fallback for strong-reasoning, 'so that an outage of both regions never stops the workflow'. What is wrong with that plan?

Show the answer

A fallback to another model is a remap that happens during an outage, without a gate and when nobody watches. gemma3:4b is not the model whose evidence the workflow and the judge have; its answers differ. It also cannot call tools, which matters for any consumer that needs them. Use it as a fallback only if every consumer’s evaluation passes on it. Otherwise, when both regions are down, the safer answer is to send the work to a person.

Check your understanding

Why does a pin need a retirement date and a reminder, and not only the deployment name?

Show the answer

A pin is an exception to the mapping, and providers retire model versions. Without a date, the pin stays until the old model disappears, and then the consumer fails on the day the provider removes it. With retire_by, everyone knows when the exception ends. The reminder 60 days before gives the owner time to run the evaluation on the new model and move. Or the owner asks for a new pin, with new evidence.

Recap

  • A model mapping gives each capability a primary, ordered fallbacks and the settings each model needs (params, params_if_tools).
  • Data rules: each provider lists the data classes its contract allows; routing skips the others, and payment reaches no model.
  • Fall back on 429, 5xx and time-outs; never on another 4xx. Larkfield’s fallbacks are the same model in a second region, so a fallback is never a silent remap.
  • Record the model that answered from the answer itself: azure-canada/chat-small (gpt-6-luna-2026-09-22).
  • A pin keeps one model for one consumer until retire_by, with a reminder 60 days before.

Your project now has the catalog’s tests: 100 passed. You can route a call and keep a pin. But your practice remap passed catalog check without one question about quality, and a real remap changes every consumer of the capability at once. Next, you treat a remap as a release, with each consumer’s own evaluation as the gate.

Next: Change a mapping safely