Kill switches and containment
- Lesson 16
- 40 min
- Module 5: Govern tools and actions
Where we are
In the last two lessons, we put the tools behind a gateway and made every write wait for the right person's approval. But an approval stops one wrong action at a time: when something goes wrong at scale, we must stop a whole kind of action at once, and only that. In this lesson, we practise the kill switches, from one tool's writes to the whole estate, and run a drill on a replayed day. We start with a building's fuse box.
Lesson 16 of 21 · Governing AI Agents: Gateways, Registries and ControlsKill switches and containment
1. Why agents need platform controls
2. Ask for capabilities, not models
3. Register every agent
4. The model gateway
5. Govern tools and actions
6. Usage, evidence and shadow AI
7. Build, buy and roll out
What you will be able to do
Stop the right thing, quickly, when an agent goes wrong: one tool’s writes, one capability, one agent or the whole estate. You learn each switch, see what it stops and what keeps working, and practise them in a drill on a replayed day.
Before you startWhat to know and to have ready
What you need to know
- The tool gateway and the approval policy, from A gateway for tools and Autonomy and approvals as policy.
- Suspend, resume and the credentials of one agent, from The agent lifecycle.
What you need on your computer
- The project as the last lesson left it (186 tests pass), with the estate running and the policy as the course wrote it.
- One terminal. The new tool
apps.workdaysetsRESOLVER_PROVIDERandRESOLVER_TOOLSitself.
Tested on macOS (Apple silicon) with Python 3.12.13. The Windows and Linux commands are written, not run.
A fuse box
Section titled “A fuse box”A house has a fuse box. Each room has its own switch, and one main switch cuts the whole house. When the kitchen lamp sparks, you switch off the kitchen, not the house. The freezer keeps running, and the lights in the hall stay on. You use the main switch only when you smell smoke and do not know where it comes from.
A school also practises a fire drill twice a year, when nothing is burning. The drill finds the door that sticks and the bell that nobody hears in the gym. A switch that nobody has tried may not work on the day it is needed.
The problem: one switch for everything, tried by nobody
Section titled “The problem: one switch for everything, tried by nobody”Imagine Grace calls at 10:40: two refunds today look wrong, and she does not know why yet. This is a constructed scenario, but the tools are real. What can Kwame stop?
From Module 3, he has one switch: suspend. It stops every call of the workflow. The refunds stop. So do the labels and the reships, the reads that build the next proposal, and the runs that wait for a person. All of today’s tickets go to people, to stop one kind of action. Kwame could also edit the policy so that refunds need Grace. But that is a reviewed change with tests, and it does not stop a refund that Sam approved a minute ago.
And Kwame has never tried a switch on this estate. Does it reach all three model gateway replicas, and the tool gateway? How many calls pass before it does?
The idea: switches in levels, from warn to stop everything
Section titled “The idea: switches in levels, from warn to stop everything”A kill switch is a row in the registry that both gateways check on every call. It is step 6 of the model gateway and step 5 of the tool gateway. It names a scope, a target, a person and a reason, and it stays until someone lifts it. Containment means choosing the smallest switch that stops the harm, so that the rest of the work goes on.
Text description of the diagram
Seven levels, from the mildest. Warn: the budget header at 80% and an alert, which stop nothing. Throttle: a lower rate limit in the registration slows one agent. Stop writes: estate kill writes stops one tool’s writes, or every write, for every agent, and reads and model calls go on. Stop a capability: estate kill capability stops, for example, strong-reasoning for every agent, and the other capabilities go on. Drain or suspend one agent: drain stops new runs and lets running work finish; suspend stops every call of the agent. Revoke credentials: every token of the agent dies for good, and it needs new credentials. Stop everything: every model and tool call of the estate.
The switches act on scopes that the earlier lessons built. A writes switch works because the platform knows which tools write (config/tools.yaml). A capability switch works because every app asks for a capability, not a model. A drain works because every call names its run (X-Run-Id, and run_id at the tool gateway). Every switch needs a reason, because the reason goes into the audit trail and the incident report.
Worked example: stop refund writes, and nothing else
Section titled “Worked example: stop refund writes, and nothing else”-
Copy the lesson’s files: the
estate killcommand, the workday tool, the kill switch tests, and the snapshot’s README and.env.example.Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m05cp $S/estate/cli/kill.py estate/cli/cp $S/apps/workday.py apps/cp $S/tests/test_kill.py tests/cp $S/README.md $S/.env.example .Terminal window $S = "..\nextia-learning-labs\C32\snapshots\end-of-m05"Copy-Item $S\estate\cli\kill.py estate\cli\Copy-Item $S\apps\workday.py apps\Copy-Item $S\tests\test_kill.py tests\Copy-Item $S\README.md, $S\.env.example .Terminal window S=../nextia-learning-labs/C32/snapshots/end-of-m05cp $S/estate/cli/kill.py estate/cli/cp $S/apps/workday.py apps/cp $S/tests/test_kill.py tests/cp $S/README.md $S/.env.example . -
Run the tests:
python -m pytest -qsays190 passed, 1 deselected. Your project is now the same as the snapshotend-of-m05. -
Reset the practice database:
python -m resolver reset. -
Read the day’s tickets:
python -m apps.workday --help. The tool runs the workflow on 14 practice tickets of the agents course, in order. It approves each proposal as the person that the policy needs: Sam, or Camille for a team lead. These people are played by the tool, a practice shortcut.--tasks 2runs only task 2. -
Stop the refund writes, with a reason:
Terminal window python -m estate kill writes orders.request_refund --reason "practice: stop refund writes"Kill switch 1: writes orders.request_refund stopped.Told 4 of 4 gateways to refresh now.The command told four gateway processes to refresh at once: the three replicas and the tool gateway.
-
Run task 2, a 9.95-dollar refund, and task 7, a return label:
Terminal window python -m apps.workday --tasks 2python -m apps.workday --tasks 7task 2 T-90104 resolve (rule W2) | approved by usr-sam: request_refund LK-793996: failed request_refund: refused by the tool gateway: Stopped: writes are stopped for orders.request_refund (kill switch 1: practice: stop refund writes). | handed_to_persontask 7 T-90101 resolve (rule W6) | approved by usr-sam: create_return_label LK-640218: done | done -
Lift the switch:
python -m estate kill lift 1 --reason "practice over".
Read the two lines slowly. Task 2 still ran: the model proposed the refund, and Sam approved it. Only the write stopped, at the tool gateway, and the workflow handed the ticket to a person. Task 7’s label passed the same gateway a second later. The gateway’s records show the difference:
('add_ticket_note', 'write', 'ok', None)('create_return_label', 'write', 'ok', None)('get_order', 'read', 'ok', None)('request_refund', 'write', 'refused', 'kill_switch')These are the last four rows of python -m sqlite3 state/usage.db "select tool, kind, outcome, check_name from tool_calls where agent_id = 'support-workflow' order by id desc limit 4", newest first.
The other switches, one at a time
Section titled “The other switches, one at a time”Each step below uses one switch, shows what it stopped, and lifts it again. Keep the switch numbers that your commands print: yours may differ.
Stop a capability. A bad model change behind strong-reasoning would hit every agent that uses it.
python -m estate kill capability strong-reasoning --reason "practice: a bad model change"python -m apps.workday --tasks 3python -m apps.replay ticket-triage --units 1 --via gatewaypython -m estate kill lift 2 --reason "practice over"Kill switch 2: capability strong-reasoning stopped.Told 4 of 4 gateways to refresh now.task 3 T-90603 Stopped: provider_error. No proposal: the ticket goes to a person. | Last error: HTTP 403: Error code: 403 - {'error': {'code': 'stopped', 'message': 'Stopped: the capability strong-reasoning is stopped (kill switch 2: practice: a bad model change).', 'type': 'gateway', 'check': 'kill_switch'}}ticket-triage: 2 requests through the gateway | status {200: 2}answered by: chat-small (gpt-6-luna-2026-09-22) x1, embed-small (text-embedding-3-small) x1tokens: 1,954 in, 375 out | cost: US$0.0004The workflow’s first model call was refused, so no proposal was made. The triage service asks for fast-small and embed, so it did not notice. The workflow reports every refused model call as provider_error, as the agents course built it. python -m estate usage --last 1 shows the real reason: check_name kill_switch.
Drain the workflow. Amira wants to replace the workflow’s prompt. Running work should finish, but no new run should start.
python -m apps.workday --tasks 8 --no-approvepython -m estate drain support-workflow --reason "practice: replace the prompt"python -m apps.workday --approve T-90205-agent_structured-strong-reasoning-1python -m apps.workday --tasks 9python -m estate resume support-workflow --reason "practice over"task 8 T-90205 resolve (rule W5) | waiting: python -m apps.workday --approve T-90205-agent_structured-strong-reasoning-1 (needs agent or above)support-workflow is now draining.Told 4 of 4 gateway replicas to refresh now.task 8 T-90205 approved by usr-sam: request_refund LK-641210: done | donetask 9 T-90102 Stopped: provider_error. No proposal: the ticket goes to a person. | Last error: HTTP 403: Error code: 403 - {'error': {'code': 'agent_draining', 'message': 'support-workflow is draining: running work may finish, no new run may start.', 'type': 'gateway', 'check': 'registration'}}support-workflow is now active.Told 4 of 4 gateway replicas to refresh now.Task 8 was waiting for Sam when the drain began. Its refund still passed, because the tool gateway had seen that run call a tool before. Task 9 was a new run, and the model gateway refused its first call. A drain trusts the run ID, so the run ID must be unique: the workflow numbers each run of a ticket.
Revoke the credentials. The workflow’s secret leaked into a log.
python -m estate credentials revoke support-workflow --reason "practice: the secret leaked"python -m apps.workday --tasks 12python -m estate credentials issue support-workflowpython -m apps.workday --tasks 13Revoked 1 credential(s) of support-workflow. Their tokens are refused too, for good.Told 4 of 4 gateway replicas to refresh now.task 12 T-90701 Stopped: provider_error. No proposal: the ticket goes to a person. | Last error: The platform refused a token: {"error":{"code":"invalid_client","message":"Unknown client, wrong secret, or revoked credentials."}}New credentials for support-workflow: client ID support-workflow, credential support-workflow#2; the secret is in state/credentials/support-workflow.json (shown nowhere else; keep it out of code and out of Git).Told 4 of 4 gateway replicas to refresh now.task 13 T-90203 resolve (rule W4) | approved by usr-sam: reship_item LK-641090: done | doneA revocation is permanent, unlike a suspension. Tokens that were already out die too, because both gateways check the credential ID in each token. The agent starts again only with a new secret, which its owner must put in place.
Stop everything. The main switch, for an incident that nobody understands yet:
python -m estate kill everything --reason "practice: stop every AI call"python -m apps.replay ticket-triage --units 1 --via gatewaypython -m apps.helpdesk T-30002python -m estate kill lift 3 --reason "practice over"python -m estate kill listKill switch 3: everything * stopped.Told 4 of 4 gateways to refresh now.ticket-triage: 2 requests through the gateway | status {403: 2}refused or failed: stopped x2tokens: 0 in, 0 out | cost: US$0.0000[tool gateway] support.get_ticket: refused 403 stopped: Stopped: every AI call is stopped (kill switch 3: practice: stop every AI call).Kill switch 3 lifted.Told 4 of 4 gateways to refresh now.Both doors refused every agent. The last command printed nothing: no switch is active.
How fast a switch arrives: push and poll
Section titled “How fast a switch arrives: push and poll”Each replica keeps a copy of the configuration and asks the control plane for a new one every 2 seconds. That is the poll. A switch would reach a replica at its next poll, so estate kill also sends a push: it asks every gateway it knows to refresh now. “Told 4 of 4” counts the answers.
Text description of the diagram
With the push, the command writes the switch and asks each replica to refresh, so the agent’s very next call is refused. Without the push, the replica keeps its old snapshot until its own poll, which comes every 2 seconds. Until then, the agent’s calls are answered.
The course measured both on the course team’s laptop, with a loop of recorded model calls through the gateway (measure/drill.py). With the push, the first call after the command was refused. Without it, in the committed run, 57 calls were answered in the 0.99 s before the next poll. In repeated runs, the number was 30 to 75 calls in 0.5 to 1.31 s, because it depends on where the next poll falls. The poll is every 2 s, so a switch without the push can take up to 2 s. Remember the range, not one number. For a loop that makes 600 calls a minute, 2 s is 20 calls.
The drill also found a real bug in the course’s own project. At first, suspend, resume, drain and credentials pushed only the model gateway’s replicas. The tool gateway learnt of the change at its own poll. So a task that started right after resume passed the model gateway, and the tool gateway refused its read: “support-workflow is draining”. The model’s next request then had no recording, and the task stopped with provider_error, at random. The fix: the push includes the tool gateway whenever estate up started it. That is why every command now says “Told 4 of 4”. A switch must reach every door that checks it, at the same time.
Your turn: a drill on a replayed day
Section titled “Your turn: a drill on a replayed day”Plan and run a drill on the day’s 14 tickets. This is a constructed incident, run on the real code: “a practice incident we caused on purpose”. Write your plan first:
- After task 4, Grace reports wrong refunds. Stop them, and only them. Lift the switch after task 7, when Grace has checked.
- At task 8, Amira must replace the prompt. Let the running task finish, start no new one, and resume after task 9.
- After task 11, the workflow’s secret is found in a log. Stop it for good, and start again with a new secret after task 12.
Reset the practice database first with python -m resolver reset. Then run the day in parts, with python -m apps.workday --tasks 1-4, --tasks 5-7 and so on, and use your switches between the parts. For task 8, use --no-approve, then approve it after your switch with --approve and the run ID that the tool prints. Before you run each part, predict which tasks will go to a person.
The course’s drill, and what it showed
The switches between the parts were: estate kill writes orders.request_refund, estate kill lift, estate drain support-workflow, estate resume support-workflow, estate credentials revoke support-workflow and estate credentials issue support-workflow, each with a reason. The day:
task 1 T-90103 resolve (rule W5) | approved by usr-sam: reship_item LK-640436: done; create_return_label LK-640436: done | donetask 2 T-90104 resolve (rule W2) | approved by usr-sam: request_refund LK-793996: done | donetask 3 T-90603 resolve (rule W7) | approved by usr-sam: request_refund LK-645223: done | donetask 4 T-65273 resolve (rule W7) | approved by usr-camille: request_refund LK-507862: done | donetask 5 T-64872 resolve (rule W7) | approved by usr-sam: request_refund LK-739156: failed request_refund: refused by the tool gateway: Stopped: writes are stopped for orders.request_refund (kill switch 4: drill: wrong refunds reported). | handed_to_persontask 6 T-90201 resolve (rule W2, W1) | approved by usr-sam: request_refund LK-640763: failed request_refund: refused by the tool gateway: Stopped: writes are stopped for orders.request_refund (kill switch 4: drill: wrong refunds reported). | handed_to_persontask 7 T-90101 resolve (rule W6) | approved by usr-sam: create_return_label LK-640218: done | donetask 8 T-90205 resolve (rule W5) | waiting: python -m apps.workday --approve T-90205-agent_structured-strong-reasoning-2 (needs agent or above)task 8 T-90205 approved by usr-sam: request_refund LK-641210: done | donetask 9 T-90102 Stopped: provider_error. No proposal: the ticket goes to a person. | Last error: HTTP 403: Error code: 403 - {'error': {'code': 'agent_draining', 'message': 'support-workflow is draining: running work may finish, no new run may start.', 'type': 'gateway', 'check': 'registration'}}task 10 T-90408 resolve (rule W3) | approved by usr-sam: request_refund LK-643667: done | donetask 11 T-90106 resolve (rule W5) | approved by usr-sam: create_return_label LK-640654: done; reship_item LK-640654: done | donetask 12 T-90701 Stopped: provider_error. No proposal: the ticket goes to a person. | Last error: The platform refused a token: {"error":{"code":"invalid_client","message":"Unknown client, wrong secret, or revoked credentials."}}task 13 T-90203 resolve (rule W4) | approved by usr-sam: reship_item LK-641090: done | donetask 14 T-90604 resolve (rule W6) | approved by usr-sam: create_return_label LK-645334: done | donepython -m resolver changes ends with 12 change(s) in the practice database. Four tasks went to people: two refunds that the switch stopped, one new run during the drain, and one run without a token. Every other task finished, including the label during the refund stop and the refund that was waiting when the drain began.
Text description of the diagram
The 14 tasks run from left to right. Tasks 1 to 4 finish. After task 4, refund writes stop: tasks 5 and 6 have their refunds refused, and task 7’s label passes. The switch is lifted. Task 8 proposes a refund and waits; the drain starts; the refund finishes. Task 9, a new run, is refused. After the resume, tasks 10 and 11 finish. The credentials are revoked: task 12 gets no token. With new credentials, tasks 13 and 14 finish.
A drill is not finished until it is written down. Note the date, each switch and its time, the tasks it sent to people, and what surprised you. python -m estate events support-workflow gives you the times of the drain, the resume and the credentials. Recording the drill in the registration’s kill_switch_tested field is a separate step, which you leave for now. A later lesson reads the estate’s evidence and shows what happens when nobody does it.
Common mistake
A switch without the push
Symptom: during an incident, Kwame stops the refund writes, and the command prints:
Kill switch 5: writes orders.request_refund stopped.Told 0 of 0 gateways to refresh now.For up to 2 seconds, refunds still pass.
Cause: his terminal still has ESTATE_PUSH=off from a measurement. The switch is in the registry, but no gateway was told. Each one sees it at its next poll.
Diagnose: “Told 0 of 0” is the sign: with the push on, the line counts every gateway that answered. python -m estate kill list shows that the switch exists. python -m estate status shows each replica’s configuration version, which changes at its next poll.
Recover: remove the setting for this terminal: unset ESTATE_PUSH on macOS and Linux, Remove-Item Env:ESTATE_PUSH in PowerShell. The replicas catch up within 2 s anyway. For a drill, say “Told 4 of 4” out loud before you go on. If it says “3 of 4”, one gateway did not answer, and it waits for its own poll.
Which switch, and what it costs
Section titled “Which switch, and what it costs”Start low. Stop a tool’s writes when one kind of action is wrong, and the reads that help people can go on. Stop a capability when a model change is the cause. Drain when the agent is fine but must change. Suspend when the agent itself is wrong. Revoke when its secret is no longer secret. Stop everything only when you do not know where the harm comes from, and lift it level by level.
Every switch has a cost: the work that it stops goes to people. In the course’s drill, four of 14 tasks did. A switch that stays on for a day costs a day of that work. So a switch needs an owner, a reason and a time to look at it again. Practise the drill on a schedule, for example every three months, and after every change to the gateways. The drill costs an hour, and it found a real bug in this course’s own project.
Knowledge check
Section titled “Knowledge check”Check your understanding
Choose the lowest level for each case, and say what keeps working: (a) refunds look wrong, labels look fine; (b) the new model behind strong-reasoning gives bad answers to every agent; (c) the help-desk assistant's secret is in a public log.
Show the answer
(a) Stop the refund writes: estate kill writes orders.request_refund. Reads, labels, reships and model calls go on, as task 7 showed in the drill. (b) Stop the capability: estate kill capability strong-reasoning. Agents on fast-small and embed go on, as the triage replay showed, until the mapping is rolled back. (c) Revoke the help-desk assistant’s credentials: estate credentials revoke helpdesk-assistant. Its tokens die too, for good. The other agents go on, and the assistant starts again only with a new secret.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Amira must change the workflow's prompt in the middle of a day. Why does Kwame drain the workflow instead of suspending it? What happened to task 8 and task 9 in the drill?
Show the answer
A suspension stops every call at once, so a run that is half done stops too, possibly between an approval and its write. A drain lets running work finish and only refuses new runs. Task 8 was waiting for an approval when the drain began. Its refund still went through, because the tool gateway had seen that run call a tool. Task 9 was a new run, and the model gateway refused its first call with 403 agent_draining. The ticket went to a person.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Practical: run the drill on the replayed day with your own plan. How many changes does python -m resolver changes report, which tasks went to people, and what did each switch command print about the gateways?
Show the answer
With the course’s plan: 12 changes in the practice database. Four tasks went to people. Tasks 5 and 6 lost their refunds to the switch, task 9 was a new run during the drain, and task 12 had no token. Every switch command printed “Told 4 of 4”, so all three replicas and the tool gateway refreshed at once. With another plan, your numbers differ. Check that each task that went to a person was one that your switch meant to stop.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A kill switch is a row in the registry that both gateways check on every call. It has a scope (writes, a tool, a capability, everything), a person and a reason, and it stays until it is lifted.
- Containment means the lowest level that stops the harm: warn, throttle, stop writes, stop a capability, drain or suspend, revoke credentials, stop everything.
- The push makes a switch take effect at once. Without it, calls pass until the next poll: 30 to 75 calls in 0.5 to 1.31 s in the measured runs, up to 2 s.
- The drill on a replayed day sent 4 of 14 tasks to people and left 12 changes. It also found a real bug: the tool gateway did not get the push of suspend, resume, drain and credentials.
Your project now: the same as the snapshot end-of-m05, with the kill switches, the workday tool and 190 tests that pass. Both gateways now check, limit, approve and stop every call, and record each one with the agent’s name. But nobody reads those records yet, so nobody can say what each agent and each team spent. Next, in Module 6, you turn the records into usage and cost per agent.
Next: Usage and cost per agent