When the gateway fails
- Lesson 13
- 45 min
- Module 4: The model gateway
Where we are
In the last three lessons, we put every model call behind one door that checks identity, limits and data. So Tomás asks the fair question: if the gateway stops, does every agent stop with it? And one bad rule reached two apps at once. In this lesson, we stop a replica, the control plane and a provider during a replayed hour, and we see which design choices decide what still works. We start at a bank's cash machines.
Lesson 13 of 21 · Governing AI Agents: Gateways, Registries and ControlsWhen the gateway fails
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
Build the gateway so that it is not a single point of failure. Use several replicas behind one address, a cached configuration, a failure mode per risk tier, provider fallbacks and staged changes. Then measure what still works when each part stops.
Before you startWhat to know and to have ready
What you need to know
- The gateway’s path, limits and guardrails, from the three earlier lessons of this module, starting with One door for model calls.
- What a risk tier is, from What a registration holds. The workflow is high risk, triage and the judge are medium, the document assistant is low.
What you need on your computer
- The project as the last lesson left it (159 tests pass), with the guardrails back to the two real rules.
- Two terminals in
~/projects/ai-estate, with the virtual environment active in both. Ports 8320 to 8325 must be free.
Tested on macOS (Apple silicon) with Python 3.12.13. The Windows and Linux commands are written, not run.
A bank’s cash machines
Section titled “A bank’s cash machines”One bank has many cash machines in a city. If one machine breaks, you walk to the next one: nobody says the bank is down. Each machine also keeps working for a while when its line to the head office drops. It pays out small amounts from what it already knows. But it refuses a large transfer, because that needs a check that only the head office can do.
When the bank changes the machines’ software, it does not change every machine at the same minute. It tries one branch first. If the new software fails there, only that branch’s customers notice, and the bank goes back.
One bank, one way to get cash, but not one machine. That is the whole idea of this lesson.
The problem: Tomás is right, so far
Section titled “The problem: Tomás is right, so far”Tomás says: “A gateway is a single point of failure, so do not use one.” With the gateway that you built so far, he is right. Stop it, and send one ticket:
python -m estate down gatewaypython -m apps.replay ticket-triage --units 1 --via gatewayThe replay ends with a long traceback, whose last line is:
httpx.ConnectError: [Errno 61] Connection refusedOn Linux the number is 111; on Windows, 10061 (not tested). One process stopped, and every app lost its models. In the last lesson, one bad line in config/gateway.yaml stopped two apps at once. That is the blast radius of a change: how many requests it reaches before someone stops it.
But without a gateway, each app needs its own retries, fallback region, key and limits. Five teams build five versions, and the weekly digest builds none. Their failures are smaller, but each is different, and nobody sees them together. So the question is how to build the gateway so that one failure does not stop everything. Stop the estate now: python -m estate down.
The idea: one logical door, not one machine
Section titled “The idea: one logical door, not one machine”The gateway is one logical door: every app knows one address, and one set of rules applies. Behind that address, it is not one machine. Microsoft’s guide to gateways in front of several model deployments says the same: “deploy multiple instances of the gateway”. Larkfield’s gateway uses five design choices:
Text description of the diagram
The apps know one address. Behind it, a front door sends each request to one of three replicas, a, b and c. It skips a replica that fails its health check. Each replica keeps its own copy of the configuration. The control plane publishes the configuration: each replica polls it every 2 seconds, and a change is also pushed to them. All replicas write to one usage store, so they share one count per agent and team. They call the provider in East US 2, and fall back to a second region.
- Several stateless replicas behind a front door. The counters are in the shared usage store, and the rules come from the control plane. So any replica can serve any request, and one replica can stop.
- Configuration cached at each replica. A replica polls the control plane every 2 seconds and keeps the last good copy, in memory and on disk. If the control plane stops, the replica goes on with its copy, up to a limit,
max_stale_seconds. - A failure mode per risk tier. When a control cannot run, fail open means: serve the call without that control, marked as degraded. Fail closed means: refuse the call. Larkfield chose by risk tier.
- Provider fallback. When a provider fails, the gateway tries the next provider of the mapping.
- Staged configuration changes. A new version goes to one replica first, the canary, and goes back if its error rate is too high.
Worked example: how old may a copy be?
Section titled “Worked example: how old may a copy be?”The control plane stops at 12:00:00. Replica a’s last good poll was at 11:59:59. It keeps polling every 2 seconds, and each poll fails. The age of its copy decides what happens:
| Time | Age of the copy | State | Ticket triage (medium) | Support workflow (high) |
|---|---|---|---|---|
| 12:00:03 | 4 s | fresh (at most 2 polls + 1 s) | served | served |
| 12:00:30 | 31 s | stale (under 60 s) | served, as usual | served, as usual |
| 12:01:01 | 62 s | expired (over 60 s) | served, marked degraded (fail open) | refused: 503 control_unavailable (fail closed) |
For the first minute, a control-plane outage is invisible to the apps. After the limit, each tier gets its own answer. Sixty seconds is a practice value. A real estate weighs two risks: an old copy that misses a suspension, against a failure mode that starts too soon.
What fail open costs
Section titled “What fail open costs”Fail open is not free, and the course measured each cost on the real code:
| Risk tier | Mode | While the control cannot run | What fail open means here |
|---|---|---|---|
| low (document assistant) | fail open | calls pass, marked x-gateway-degraded |
each replica counts from the last total it read plus its own calls: with three replicas, up to three times what was left of the budget. The guardrails still run (they are code in the replica). A new suspension does not arrive |
| medium (ticket triage, release judge) | fail open | the same | the same |
| high (support workflow, tool servers) | fail closed | 503 control_unavailable |
not used: no write without every control. The workflow hands its tasks to people |
This is docs/failure-modes.md in the project. The same modes apply when the usage store is down. A registration may set its own failure mode.
Add the lesson’s files
Section titled “Add the lesson’s files”-
Copy the front door, the failure-mode table, the staged configuration commands, the traffic tool, the tests, and the new README:
Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m04cp $S/gateway/front.py gateway/cp $S/docs/failure-modes.md docs/cp $S/estate/cli/config.py estate/cli/cp $S/apps/traffic.py apps/cp $S/tests/test_front.py $S/tests/test_config_cache.py $S/tests/test_rollout.py $S/tests/test_services.py tests/cp $S/README.md $S/.env.example .Terminal window $S = "..\nextia-learning-labs\C32\snapshots\end-of-m04"Copy-Item $S\gateway\front.py gateway\Copy-Item $S\docs\failure-modes.md docs\Copy-Item $S\estate\cli\config.py estate\cli\Copy-Item $S\apps\traffic.py apps\Copy-Item $S\tests\test_front.py, $S\tests\test_config_cache.py, $S\tests\test_rollout.py, $S\tests\test_services.py tests\Copy-Item -Force $S\README.md, $S\.env.example .Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m04cp $S/gateway/front.py gateway/cp $S/docs/failure-modes.md docs/cp $S/estate/cli/config.py estate/cli/cp $S/apps/traffic.py apps/cp $S/tests/test_front.py $S/tests/test_config_cache.py $S/tests/test_rollout.py $S/tests/test_services.py tests/cp $S/README.md $S/.env.example . -
Run the tests:
python -m pytest -q. The last line says172 passed, 1 deselected. Your project is now the same as the snapshotend-of-m04. -
Start the estate:
Terminal window python -m estate upprovsim up (process 71680) http://127.0.0.1:8320control up (process 71681) http://127.0.0.1:8321replica-a up (process 71682) http://127.0.0.1:8323replica-b up (process 71683) http://127.0.0.1:8324replica-c up (process 71684) http://127.0.0.1:8325front up (process 71685) http://127.0.0.1:8322/frontThe estate now starts three replicas and a front door, because the project has
gateway/front.py. The apps’ address, port 8322, did not change: the front door took it. -
Send six tickets, and see which replicas served them:
Terminal window python -m apps.replay ticket-triage --units 6 --via gatewaypython -m estate usage --by replicaThe replay ends with
status {200: 12}. In--by replica, replicas b and c now have about 4 calls each, and replica a has the rest. Earlier lessons had only replica a. -
Ask one replica how fresh its copy of the configuration is:
Terminal window curl -s http://127.0.0.1:8323/ready{"replica":"a","config":"v6.r19","freshness":"fresh","age_s":1.0,"last_error":"","usage_store":"up","pending_rows":0}Your version numbers differ if you published more often.
Guided practice: stop a replica and the control plane during an hour
Section titled “Guided practice: stop a replica and the control plane during an hour”The tool apps/traffic.py sends a compressed hour of Larkfield’s traffic through the gateway. The triage service, the workflow and the document assistant send their real recorded requests, about 2 a second. Every 10 seconds it prints what each app got back. You stop parts of the estate in the second terminal, at the times below.
-
In terminal 1, start the traffic:
Terminal window python -m apps.traffic --seconds 150 -
At about 20 seconds, in terminal 2, stop replica b:
Terminal window python -m estate down replica-b -
At about 45 seconds, stop the control plane:
Terminal window python -m estate down control -
At about 60 seconds, ask replica a how old its copy is, with
curl -s http://127.0.0.1:8323/ready. Then send one document question, which needs a new token:python -m apps.replay doc-assistant --units 1 --via gateway. -
At about 110 seconds, ask replica a again.
-
At about 140 seconds, start the control plane again:
Terminal window python -m estate up control -
When the traffic ends, start replica b again with
python -m estate up.
The course’s run. Terminal 1, the traffic (replayed requests; the hour is compressed, so read the order of events, not the rate):
Traffic through the gateway for 150 s, about 2 requests a second (Ctrl+C stops) 10 s | triage ok 20 | workflow - | documents - | replicas a 7, b 7, c 6 20 s | triage ok 16 | workflow ok 4 | documents - | replicas a 7, b 6, c 7 30 s | triage ok 16 | workflow ok 3 | documents - | replicas a 9, c 10 40 s | triage ok 19 | workflow ok 1 | documents - | replicas a 10, c 10 50 s | triage ok 17 | workflow ok 1 | documents ok 1 | replicas a 10, c 9 60 s | triage ok 18 | workflow - | documents ok 2 | replicas a 10, c 10 70 s | triage ok 16 | workflow - | documents ok 3 | replicas a 9, c 10 80 s | triage ok 16 | workflow - | documents ok 4 | replicas a 10, c 10 90 s | triage ok 14 | workflow ok 3 | documents ok 2 | replicas a 10, c 9 100 s | triage ok 20 | workflow - | documents - | replicas a 10, c 10 110 s | triage degraded 9, ok 8 | workflow - | documents degraded 2 | replicas a 9, c 10 120 s | triage degraded 16 | workflow control_unavailable 2 | documents degraded 2 | replicas a 10, c 10 130 s | triage degraded 17 | workflow control_unavailable 1 | documents degraded 1 | replicas a 10, c 9 140 s | triage degraded 20 | workflow - | documents - | replicas a 10, c 10 150 s | triage degraded 5, ok 12 | workflow - | documents ok 2 | replicas a 9, c 10Total: triage degraded 67, ok 192 | workflow control_unavailable 3, ok 12 | documents degraded 5, ok 14Terminal 2, the two answers of replica a, at about 61 and 111 seconds:
{"replica":"a","config":"v6.r19","freshness":"stale","age_s":16.4,"last_error":"ConnectError: All connection attempts failed","usage_store":"up","pending_rows":0}{"replica":"a","config":"v6.r19","freshness":"expired","age_s":66.5,"last_error":"ConnectError: All connection attempts failed","usage_store":"up","pending_rows":6}Read it phase by phase:
- Replica b stops (20 s). Only a and c answer from then on, and no call failed: the front door skips a replica that fails its health check.
- The control plane stops (45 s). For 60 seconds, every app is served, the workflow too, from the cached copies. Replica a’s copy is
stale, 16.4 s old. - The copy expires (about 105 s). Triage and the document assistant fail open: served, marked
degraded. The workflow fails closed:control_unavailable, and its tasks go to people. Replica a keeps 6 usage rows that wait for the store (pending_rows). - A new token (60 s). The document question of step 4 ended with
httpx.ConnectError: [Errno 61] Connection refused: a new token comes from the control plane. Apps that already had a token went on. - The control plane is back (140 s). One poll later, the copies are fresh again.
The course team also ran this with a crash instead of a clean stop. It killed replica b with SIGKILL, and the simulator answered at the recorded speed of the real models, so that calls were in flight. The chart shows that run, one point per 10 seconds:
- answered
- answered, degraded (fail open)
- refused or failed
Text description of the chart
Three lines over 160 seconds, calls per 10 seconds. The answered line stays near 20 from 0 to 100 seconds, through the replica’s crash at 20 seconds and the control plane’s stop at 45 seconds. Between 100 and 140 seconds it falls to 0, and the degraded line rises to about 20: the triage service and the document assistant failed open. After 140 seconds, the answered line is back near 20. The refused-or-failed line stays at 0 to 2: 2 calls in flight on replica b when it crashed (502 replica_failed, not sent again), 5 workflow calls refused when the configuration was too old (fail closed), and 2 recorded content-filter refusals of the provider. Replicas: a 175, b 14, c 174 calls.
Two calls failed, and only those: they were on replica b when it crashed. The front door does not send such a request again, because it may already have reached the provider. A model call is not a write, but it costs money, and a second answer may differ. So the app gets 502 replica_failed and decides. A refused connection is different: the request never reached the replica, so the front door sends it to the next one.
Explore
What still works when one part stops?
Predict first: The control plane has been down for 90 seconds. Does the support workflow's request pass? And the document assistant's?
- Result
- 503 control_unavailable
- Checks passed
- 6 / 10
- Replica
- a, b or c
Stopped at check 7 of 11: Failure mode. 503 control_unavailable: A control cannot run (the configuration is 90 s old (limit 60 s)): high risk fails closed. The workflow hands the task to a person. Measured: after max_stale_seconds, triage 70 and documents 1 passed degraded (fail open); the workflow’s 5 calls were refused (fail closed).
- passes
- passes, degraded or with a fallback
- stops the request
- not reached
| # | Check | What the gateway found |
|---|---|---|
| 1 | Configuration | The control plane has not answered for 90 s: the replica uses its cached snapshot. |
| 2 | Identity | A valid access token for the model gateway (sub: support-workflow). |
| 3 | Registration | support-workflow is registered and active (high risk). |
| 4 | Capability | strong-reasoning is in the registration and in the catalog; the contract has what the request needs. |
| 5 | Data class | No X-Data-Class header: the gateway takes personal, the most sensitive class of the registration. |
| 6 | Kill switches | No kill switch stops this agent or this capability. |
| 7 | Failure mode | 503 control_unavailable: A control cannot run (the configuration is 90 s old (limit 60 s)): high risk fails closed. |
| 8 | Limits | Not reached: an earlier check answered. |
| 9 | Guardrails | Not reached: an earlier check answered. |
| 10 | Route | Not reached: an earlier check answered. |
| 11 | Record | One usage row for the refusal too: outcome refused, check failure_mode. |
Computed from the gateway’s rules; the outcomes match the measured runs.
Text description of the explorer
Choose an agent and a request, as in the first lesson of this module, and then what is down: nothing, replica b, the control plane for 30 or 90 seconds, the usage store, the East US 2 region, both regions, a kill switch on the capability, the monthly budget, or the rate limit. The 11 checks light up in order. With the control plane down for 90 seconds, the workflow stops at check 7, the failure mode: 503 control_unavailable, because high risk fails closed. The document assistant and the triage service pass, marked degraded, because they fail open. With the control plane down for 30 seconds, everyone passes: the cached copy is still within its limit. With East US 2 down, every agent passes through the second region. With both regions down, every agent stops at check 10, the route, with 503 no_provider_available, and each app does its own fallback: the triage service sends the ticket to a person, the workflow hands the task to a person, the document assistant says “try again later”.
A provider outage is not a gateway failure
Section titled “A provider outage is not a gateway failure”When the provider fails, the gateway tries the mapping’s fallbacks, which each app would otherwise build itself. It falls back on 429, 5xx errors and timeouts, not on a 4xx answer: another provider would refuse the same bad request. After 3 failures in a row, a circuit breaker skips a provider for 10 seconds.
-
Take down the East US 2 region of the simulator, and send three tickets:
Terminal window python -m provsim control azure downpython -m apps.replay ticket-triage --units 3 --via gatewaypython -m estate usage --last 1The replay ends with
status {200: 6}, and the usage row saysprovider azure-canadaandfallback 1. The second region is simulated: the same recordings answer, as they would from the same models in another region. -
Take down every provider, and send three tickets and one workflow task:
Terminal window python -m provsim control '*' downpython -m apps.replay ticket-triage --units 3 --via gatewayThe replay ends with
status {503: 6}andno_provider_available x6. The usage row of such a call saysoutcome failed,check_name routeandNo provider answered for embed: last error 503 azure-canada is unavailable (a simulated outage).Run the workflow on T-90104 withRESOLVER_PROVIDER=gateway, as in the second lesson of this module. It ends withStopped: provider_error. No proposal: the ticket goes to a person. -
Bring the providers back, and send three tickets at once:
Terminal window python -m provsim control '*' normalpython -m apps.replay ticket-triage --units 3 --via gatewayIn the course’s run, all 6 calls still got
no_provider_available. The breaker skips each provider for 10 seconds after its last failure. Wait 10 seconds and replay again:status {200: 6}. If 10 seconds have already passed when you replay, the calls pass at once.
The course found a real bug here. In the replayed month, the breaker first counted real seconds while the replay moved through hours. It refused 1,280 calls after the outage had ended. A breaker on the wrong clock is a real class of bug. The fix was the estate’s clock everywhere.
In the replayed month, East US 2 was down from 10:00 to 11:00 on 17 September, a simulated outage. The second region served 123 answers, and no call failed. From 11:00 to 11:30, both regions were down. Then 35 triage calls, 2 workflow calls and 3 document questions had no provider. The triage service sent its tickets to a person’s queue, the workflow handed its tasks to people, and the document assistant said “try again later”.
Staged changes: the blast radius of a bad rule
Section titled “Staged changes: the blast radius of a bad rule”The last lesson’s bad rule reached every app at once. Now send it to the canary replica only, and let the watch decide:
-
Add the same
long-numbersline toconfig/gateway.yaml, as in the last lesson. Publish it to the canary only:Terminal window python -m estate config publish --stage canary --note "a new guardrail: long numbers"python -m estate config statusPublished version 7 (canary replicas first).Told 3 of 3 gateway replicas to refresh now.stable version 6 | candidate 7 on the canary since …replica a runs v7.r20 last seen …replica b runs v6.r20 last seen …replica c runs v6.r20 last seen …The course leaves the times out. The status also lists the last five versions. Only replica a runs version 7.
-
Send ten tickets:
python -m apps.replay ticket-triage --units 10 --via gateway. Only the calls that the front door sends to replica a can be blocked. The course’s run ended withstatus {200: 16, 400: 4}: 4 blocked of 20, against 12 of 20 in the last lesson. -
Let the watch decide:
Terminal window python -m estate config watch{"candidate": 7,"canary": ["a"],"seen": 6,"errors": 4,"error_rate": 0.6667,"state": "rolled back to v6"}The watch counts the canary’s errors with the candidate version. It promotes the version after 40 requests with at most 5% errors. It rolls it back as soon as the errors pass 5% of 40, here after 6 requests.
-
Remove the line from
config/gateway.yaml, and publish again:python -m estate config publish --note "the two guardrails, after the canary".
The course measured the same bad rule in five ways, on two replayed hours of Larkfield’s traffic. The rule is simulated, the requests are replayed, and the rule went out at 09:30. “A person” means that someone reads an alert and rolls back after 15 minutes, an assumption.
Text description of the chart
Five bars. A, all replicas at once, a person rolls back after 15 minutes: 18 requests blocked (triage 14, workflow 4). B, all at once, an automatic rollback: 5. C, staged on one replica of three, the automatic watch: 4. D, staged, a person after 15 minutes: 6. E, no shared gateway, the same rule in the triage service’s own code, a person after 15 minutes: 14, all of them triage.
A shared gateway makes a bad change reach every app: 18 against 14. But staging and an automatic watch make it smaller than one app’s own mistake: 4. Without a gateway, there is no canary for five apps’ code, and each team finds its own bugs alone.
Protect the door that holds every key
Section titled “Protect the door that holds every key”The gateway holds the provider keys of the whole estate, and it sees every request. It is the most valuable thing to attack:
- Only the gateway reads the keys in
state/secrets.json. The apps hold only their own credentials. In Find unregistered AI, an egress rule blocks the other network paths to the provider. The rollout lesson of Module 7 plans when the apps’ old provider keys are revoked. - Strict access to the door itself. Here,
/admin/refreshand/admin/faultsanswer anyone on your computer. In a real estate, only the control plane may call them, with its own credential. - Pinned and checked dependencies.
requirements.txtpins every version. A change to the gateway’s code goes through review and the canary. - No prompt text:
prompts: keep: never.
Then the gateway is a critical shared dependency, like the identity provider or DNS, and it is run like one. It has many replicas, a cached copy of what it needs, a known answer for every failure, and changes in stages.
Common mistake
A replica restarts while the control plane is down
Symptom: you restart replica a (python -m estate down replica-a, then python -m estate up replica-a) while the control plane is stopped. At once, the workflow’s calls to it are refused:
{"error":{"code":"control_unavailable","message":"A control cannot run (the configuration came from disk, and the store has not answered since): high risk fails closed.","type":"gateway","check":"failure_mode"}}The document assistant’s calls pass, with x-gateway-degraded: fail open (low risk): the configuration came from disk, and the store has not answered since.
Cause: a replica that starts without the control plane starts from its copy on disk, state/gateway/a-config.json. It cannot know how old that copy is, so it counts it as expired at once: high risk fails closed, low and medium fail open. The other replicas still have their copies in memory, and serve as before for up to 60 seconds.
Diagnose: curl -s http://127.0.0.1:8323/ready says "freshness":"expired" and "age_s":null, and last_error names the failed poll.
Recover: python -m estate up control. Two seconds later, /ready says "freshness":"fresh". Do not restart replicas while the control plane is down, unless you must.
Your turn: a failure mode for a provider outage
Section titled “Your turn: a failure mode for a provider outage”Both regions are down for half an hour. This is not a failure of the gateway’s controls, so config/gateway.yaml does not decide it: each app’s owner decides what the app does with 503 no_provider_available.
- Read
docs/failure-modes.mdand the two registrations,registrations/ticket-triage.yamlandregistrations/support-workflow.yaml: their risk tiers and their autonomy. - For the triage service and for the refund workflow, write down your choice. May the app use another model that Larkfield has, such as the local gemma3:4b? Or does it hand its work to people?
- Write one reason for each, from the course’s evidence.
- Compare with the knowledge check below.
Knowledge check
Section titled “Knowledge check”Check your understanding
Tomás says: 'A shared gateway is a single point of failure, so do not use one.' When is he right, and what makes him wrong for Larkfield's gateway? Use the course's measurements.
Show the answer
He is right for a gateway built as one process: when it stopped, every app got “Connection refused”. He is wrong for replicas behind one address, with a cached configuration and a failure mode per tier. In the measured run, replica b crashed and only its 2 calls in flight failed. Without the control plane, every call passed for 60 seconds; then low and medium risk passed degraded, and only the workflow’s 5 calls were refused, on purpose. A bad change still reaches every app (18 blocked, against 14 in one app), so changes go out in stages: 4 with the canary.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
During the store outage, the document assistant fails open. What exactly does it lose, and why is that acceptable for it but not for the workflow?
Show the answer
Its calls pass with the last configuration, marked degraded. Each replica counts its budget from the last total it read, plus its own calls. So with three replicas, it could spend up to three times what was left. A suspension made meanwhile does not arrive. The guardrails still run, because they are code in the replica. That is acceptable for a low-risk agent that reads internal documents and changes nothing. The workflow proposes refunds and reships: a write without every control is not allowed, so it fails closed and hands its tasks to people.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Completion check: choose and justify the failure mode of the triage service and of the refund workflow during a provider outage, when the second region is down too.
Show the answer
First, both use the fallback to the second region. It runs the same models, so their evaluations still hold: 123 answers on 17 September, and no call failed. When no region is left, the triage service fails open in the business sense. It is medium risk and only suggests a team, so the ticket goes to a person’s queue without AI, and only speed is lost (35 tickets that day). The refund workflow is high risk: it proposes refunds and reships. So it fails closed: no proposal, and the task goes to a person. Neither should move to a model without evidence for its task: on the agents course’s tasks, gemma3:4b passed 28 of 70, and the workflow’s minimum is 0.95.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- One logical door is not one machine: stateless replicas behind one address, a cached configuration at each, and a shared usage store.
- Without the control plane, every call passed for 60 s; then low and medium risk failed open (degraded) and high risk failed closed. Fail open costs budgets counted per replica and missed suspensions; the guardrails still run.
- A provider outage goes to the fallbacks. When nothing is left, each app’s owner decides: a person’s queue for triage, no proposal for the workflow.
- A bad change through a shared gateway: 18 blocked all at once, 4 with the canary, 14 in one app alone.
- The gateway holds every key: protect its access, pin its dependencies, keep no prompt text.
Your project now: the same as the snapshot end-of-m04, with a front door, three replicas, staged configuration and 172 tests that pass. Every model call is now checked, limited and recorded, and the gateway survives its own failures. But the agents do more than call models: the workflow’s refunds and reships are tool calls, and the model gateway never sees them. Next, in Module 5, you put a gateway in front of the tools.
Next: A gateway for tools