Fallback and change planning

What you will be able to do

The model you chose will not stay the same. Its provider can be down for an hour, the model can be retired, and its replacement can break a prompt that worked. Plan for each of these before it happens, and keep the comparison ready to run again.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• Larkfield’s must-haves and weights, and why Haiku 5.5 ranked first, from Weighted selection.
• Hidden reasoning tokens, and why they count against the output limit, from Reasoning and multimodality.
• Temperature, from Predicting continuations.
• Who updates and retires a hosted model, from Hosted and self-hosted choices.

What you need

• A browser. There is no code to run in this lesson.

Hosted models change often: your own results will differ.

The spare tyre

A car carries a spare tyre. It is not as good as the real tyre: it is smaller, and you drive slowly on it. But when a tyre goes flat on the motorway, it takes you home. Two things make it useful. It is ready before the flat tyre, and it works: a spare with no air is no spare. Careful drivers check its pressure from time to time.

A car also changes slowly. The garage tells you months ahead that a part will no longer be made. You choose the replacement part before the old one breaks, and you check that it fits.

Larkfield’s assistant needs both: a spare for a sudden failure, and a plan for a change that is announced in advance.

Three ways the chosen model can change

Grace chose Claude Haiku 5.5. Amira lists what can happen to that choice after launch. Each item is real, with a date.

1. The provider is down

Every hosted service has outages. For minutes or hours, requests return errors or take too long. Larkfield cannot repair the provider’s service. During an outage, tickets keep arriving: Larkfield gets about 45 a day, about 2 an hour on average.

2. The model is retired

A provider retires a model when it stops serving it. Anthropic’s model deprecations page defines it plainly: “Retired: The model is no longer available for use. Requests to retired models will fail.”

Here is a real notice. “On September 30, 2026, Anthropic notified developers using Claude Sonnet 4.5 of its upcoming retirement on the Claude API.” The retirement date is November 30, 2026, and the recommended replacement is claude-sonnet-5-5. The page also promises “at least 60 days’ notice before model retirement for publicly released models”.

Preview models are different. Google released Gemini 3 Pro preview on November 18, 2025, and shut it down on March 9, 2026. That was 16 weeks from start to end. This is why Larkfield’s must-haves in the last lesson ask for a stable model.

Three real model ends, from the providers' deprecation pages. A stable model gets about two months of notice; a preview model can disappear in a few months.

Text description of the diagram

A timeline from November 2025 to December 2026. Gemini 3 Pro preview: one bar from its release on November 18, 2025 to its shutdown on March 9, 2026, 16 weeks. Claude Opus 4.1: one bar from the notice on June 5, 2026 to the retirement on August 5, 2026, 61 days. Claude Sonnet 4.5: one bar from the notice on September 30, 2026 to the retirement on November 30, 2026, 61 days, still running.

For Claude Haiku 5.5, the same page says “Not sooner than October 7, 2027”. That is not a promise that the model stays until then. It is the earliest date, and the notice will come at least 60 days before.

3. The same prompt breaks on a new model

The third change is the most surprising. You move to the replacement model, with the same prompt and the same settings, and it stops working. Two real examples from this course:

Temperature is rejected. For a support assistant, you want the same ticket to get the same labels each time. So older code often set temperature to 0 (temperature controls how random the choice of the next token is). On the current Claude models, the same request fails. Amira sent one to Claude Sonnet 5.5. The answer:

HTTP 400
{"type":"error","error":{"type":"invalid_request_error","message":"`temperature` is deprecated for this model."}}

The deprecations page says that temperature, top_p and top_k return a 400 error on Claude 4.7 and later models. Code that worked on an older Claude model fails on every ticket after the switch. The course’s runner leaves temperature out for Claude for this reason.

Hidden reasoning uses up the output limit. Gemini 3.5 Flash is a reasoning model: it spends tokens on hidden reasoning before it answers. Amira first ran it with an output limit of 600 tokens, which is plenty for a 150-token reply. This is the complete reply to ticket S01:

```json
{
  "language": "en",
  "category": "delivery",
  "needs

The reply stops in the middle of a word. The hidden reasoning had used almost all of the 600 tokens. On that run, all 24 replies were cut off and none was valid JSON. With a limit of 8,000, the same model used a median of 974 reasoning tokens per ticket and answered all 24. A setting that is safe for one model breaks another.

A new model can also change the cost without any change in price. Larkfield’s prompt is 796 tokens on Claude and 560 on Gemini, because each provider splits text into tokens in its own way. Anthropic’s pricing page says that Claude 4.7 and later models use a newer tokenizer that “produces approximately 30% more tokens for the same text”. So the same tickets can cost more after a switch.

The idea: a fallback, and a plan for change

A fallback is a second way to handle a request, prepared and tested in advance, that the app uses when the first way fails. Like the spare tyre, it may be a little worse. It must be ready and it must work.

The first question for a fallback is: which failure does it cover? A fallback that shares the failure is not a fallback for it.

• Claude Sonnet 5.5 or Opus 5.5 passed every must-have. But they come from the same provider as Haiku 5.5. If Anthropic’s service is down, they are down too. They are a good answer to “Haiku’s drafts are not good enough”, not to “the provider is down”.
• Gemini 3.5 Flash passed every must-have, and it comes from another provider. Its rubric score was 9.95, with 24 of 24 replies ready (Haiku: 9.84, 24 of 24). It covers an outage of Anthropic’s service, as long as it is called with an output limit of 8,000 tokens.
• No draft at all. The ticket goes to the agents’ queue, as it did before the assistant existed. This always works and costs nothing, because a person checks every reply anyway. It is the last step.

So Larkfield’s fallback has three steps, in order:

Larkfield's fallback path. Each step is used only when the step before it fails, and the agent can see which step made the draft.

Text description of the diagram

A new ticket goes to the primary model, Claude Haiku 5.5. If the reply is valid JSON, the agent gets a draft. If the call returns an error, takes too long, or the reply is not valid JSON, the ticket goes to the fallback model, Gemini 3.5 Flash, called with an 8,000-token limit, at another provider. If its reply is valid JSON, the agent gets a draft marked “fallback”. If the fallback also fails, there is no draft: the ticket goes to the agents’ queue, as before the assistant existed.

A small worked example: a three-hour outage

Suppose Anthropic’s service is down for 3 hours on a busy afternoon. About 12 tickets arrive (4 an hour).

Plan	Tickets with a draft	Extra cost
No fallback: the app shows an error	0 of 12	US$0
Fallback to the agents’ queue only	0 of 12, but no error: the old way of working	US$0
Gemini 3.5 Flash, then the queue	12 of 12 if Gemini works	12 × US$0.0109 = US$0.13

Gemini costs 40 times as much per ticket as Haiku, but only during the outage. A hosted fallback costs nothing while it waits, because you pay per token. You pay only for the tickets that it handles. A fallback on your own hardware is different: the machine costs money every month, used or not.

The fallback is not free in other ways. Larkfield needs a second account, a second contract with checked data terms, and code that knows two providers. Mark every fallback draft, so that Camille and the agents know it came from another model.

When to look again: repeated comparison

The 24 tickets, the must-haves, the weights and the recorded runs are the comparison. Keep them, because you will run the comparison again. On Haiku 5.5, one run of the 24 tickets costs about 24 × US$0.00027 = less than one cent, and takes about a minute. A run on all five hosted models cost less than US$1.

Run it again when one of these triggers happens:

• A retirement notice for the chosen model or the fallback. Run the 24 tickets on the recommended replacement in the first two weeks. Switch at least two weeks before the retirement date.
• A new model or a new version that you might use. Never switch on the provider’s word alone.
• A price change, or a change in how tokens are counted. Recompute the cost score.
• New kinds of tickets. If agents see failures that the 24 tickets do not cover, add tickets of that kind first, then run again.
• A fixed date, for example every three months, even if nothing else happened.

Before any switch, check the prompt and the settings, not only the scores:

1. Send the request with the exact settings of production: output limit, temperature (or none), and the “effort” or reasoning setting.
2. Check that all 24 replies are valid JSON and that no reply stopped at the limit.
3. Check the language of the 12 French and mixed tickets.
4. Recompute the cost per ticket with the new model’s own token counts.
5. Apply the same must-haves and weights. Write the result next to the old one.

When to use it, and what it costs

Plan a fallback for every assistant that people rely on during the day. Plan the change process for every hosted model, because every hosted model is retired one day.

Keep the plan in proportion. Larkfield’s assistant only drafts replies, and a person checks each one. So “no draft” is an acceptable last step, and one fallback model is enough. An assistant that answers customers directly, with no person in between, needs a stricter plan. For it, the human queue is not a quiet last step.

The costs are a second provider to manage, a little code, and a test run of the fallback from time to time. A fallback that you never test is the spare tyre with no air.

Guided practice: answer the real notice

Suppose Larkfield had launched in the summer with Claude Sonnet 4.5. On September 30, 2026, the notice arrives.

1. Write the retirement date and the recommended replacement from the notice.
2. Write the date by which you will have run the 24 tickets on the replacement.
3. Write the latest date on which you will switch production.
4. List the two settings that you check first, because of this lesson’s examples.
5. Write what happens to Larkfield’s tickets on December 1, 2026, if nobody acts.

Example answer

1. Retirement on November 30, 2026; replacement claude-sonnet-5-5.
2. October 14, 2026: two weeks after the notice.
3. November 16, 2026: two weeks before the retirement, so that a problem leaves time to fix it.
4. Temperature: Sonnet 5.5 returns a 400 error when it is set. The output limit: thinking counts inside the output tokens, so a small limit can cut the reply.
5. Every request to claude-sonnet-4-5-20250929 fails. With the fallback path, every ticket goes to Gemini 3.5 Flash, at about three times the cost per ticket of Sonnet 5.5. Without a fallback, no ticket gets a draft.

Your turn: Tomás’s perfect fallback

Tomás proposes Claude Opus 5.5 as the only fallback. “It had full marks on all 24 tickets, and it is 2nd on the leaderboard.”

Which of the three changes in this lesson does Opus cover, and which not? Where would you use it?

Example answer

Opus does not cover an outage of Anthropic’s service: it is down at the same time as Haiku. It does not cover a broken setting either, if the same code calls both models. It covers a different event: Haiku’s drafts turn out not to be good enough, or Haiku’s quality drops on a new version. For that, Opus is a good, tested second choice: it passed every must-have and had full marks. Its full marks do not make it better than Haiku on this test, because the quality scores tie. And it costs 35 times as much per ticket. So keep Opus as the quality alternative in the memo. Keep Gemini 3.5 Flash and the agents’ queue as the fallback for outages.

Common mistake

The fallback was never tested with its real settings

Symptom: during an outage, the app switches to the fallback model, and every draft fails. The logs show replies that stop in the middle of the JSON, such as "needs.

Cause: the fallback was called with the primary model’s settings, for example an output limit of 600 tokens. The fallback is a reasoning model. Its hidden reasoning used up the limit before it wrote the answer, as Gemini 3.5 Flash did on all 24 tickets at 600 tokens.

Fix: give the fallback its own settings (for Gemini 3.5 Flash, an 8,000-token limit), and run the 24 tickets on it exactly as production calls it. Repeat that run on a fixed date, and after every change to the prompt. In a real service, also send a small share of normal traffic to the fallback path from time to time. Then you know that it works before you need it.

Check your understanding

Check your understanding

Why is Claude Sonnet 5.5 not a fallback for an outage of Claude Haiku 5.5, although it passed every must-have?

Show the answer

Both models come from the same provider and the same service. When that service is down, both are down. A fallback for an outage must not share the failure, so it must come from another provider (Gemini 3.5 Flash) or not use a model at all (the agents’ queue). Sonnet is still a possible alternative for other reasons, for example if Haiku’s quality drops on a new version.

Check your understanding

Predict: Amira moves from an older Claude model to Claude Haiku 5.5. Her code sets temperature to 0 and the output limit to 600. What happens, and what should she check?

Show the answer

Every request fails with HTTP 400: “temperature is deprecated for this model.” She must remove temperature. Then she should check the output limit: Claude counts thinking inside the output tokens, so a small limit can cut replies. Then she runs the 24 tickets: all valid JSON, no reply stopped at the limit, the right language on the 12 French and mixed tickets, and the new cost per ticket.

Check your understanding

Name three triggers that should make Larkfield run the 24-ticket comparison again.

Show the answer

Any three of these: a retirement notice for the chosen model or the fallback; a new model or version that might replace it; a price change or a change in how tokens are counted; new kinds of tickets or failures that the 24 tickets do not cover; a fixed date, for example every three months.

Recap

• Three changes are certain to happen: outages, retirements, and new models that break an old prompt.
• A real notice: Claude Sonnet 4.5, announced 2026-09-30, retired 2026-11-30. After retirement, every request fails. Preview models can end within months.
• The same prompt can break on a new model: temperature returns a 400 error on current Claude models, and a 600-token limit cut every Gemini 3.5 Flash reply because of hidden reasoning.
• A fallback must not share the failure. Larkfield’s path: Haiku 5.5, then Gemini 3.5 Flash with its own settings, then the agents’ queue.
• Keep the 24 tickets, the rules and the runs. Run them again on each trigger, and check the settings before you switch.

Next: Communicate the recommendation