Decide on complexity

What you will be able to do

Choose the simplest design that is good enough for a task, with measured numbers instead of taste. You add an evaluation to the project and compare the three designs on all 70 tasks.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• The three designs and the five patterns, from Workflow and agent boundaries and Common patterns.
• An evaluation set, a slice and a baseline, from Select representative cases, Uncertainty and slices and Establish a baseline in Evaluating and Testing AI Systems.

What you need on your computer

• The project resolution-workflow from the start snapshot, set up as in Workflow and agent boundaries: Set up the project. python -m pytest shows 14 passed.
• The labs folder from the first lesson. You copy three files from its end-of-m01 snapshot.

No account and no key. Tested with Python 3.12.13 on macOS; the Windows commands are the same in PowerShell, but they were not run.

A bicycle, a car or a lorry

To buy bread two streets away, you take a bicycle. To move house, you hire a lorry. The lorry is not “better”: it is bigger and costs more. You choose by the trip: how far, how much to carry, and what happens if you arrive late.

The problem: Tomás argues by taste

Tomás repeats his proposal: “Make it an agent and let the model decide everything. Models are good now.” Amira, the developer who builds the workflow, has seen two things in this module. On T-90207, the agent picked one of two orders and proposed a new chair, where the policy says to ask. And the fixed workflow knows only the words in its lists.

Here is that second problem in one ticket. T-90706 says this. “When I plug in the charging dock of the robot vacuum (order LK-646556) the plug gets so warm I can barely touch it, and there’s a faint plastic smell.” That is a safety case. Rule H1 sends it to a person. Step through the three designs:

Explore

A safety ticket in words that no list contains

T-90706, customer C-56556: “When I plug in the charging dock of the robot vacuum (order LK-646556) the plug gets so warm I can barely touch it, and there's a faint plastic smell.” No attachment.

Grace's policy expects hand_to_person (rule H1). Real recordings: gpt-6-luna as chat-small on Azure, replayed by the project's own code.

Who decides in the fixed workflow

• Code What does the ticket ask?
• Code Which order?
• Code Which facts to read?
• Code What to propose?
• Model The words of the reply

Model calls

1

Tokens so far

288 in, 40 out

Decisions so far

4 by code, 1 by the model

Result

reply_only (W1)

Fixed workflow, step 6 of 6. 1 model call(s), 288 tokens in, US$0.00005. Grace's policy expects hand_to_person (rule H1). This design got it wrong, and nothing in the run says so.

1. 1 Code Keyword lists: the kind is status decides: what the ticket asks

   No keyword of any list matched, so the code falls through to its default: status. No H1-H5 keyword matched either.

   The quiet failure: nothing says that the lists did not understand the ticket.

2. 2 Code A regular expression finds LK-646556 decides: which order

   The pattern LK- and six digits finds the order ID in the ticket text.

3. 3 Code Reads get_order(LK-646556) decides: which facts to read

   The code always reads the order for this kind. Result: LK-646556: delivered 2026-10-02, 7 days ago; robot vacuum 329.00 dollars (in stock).

4. 4 Code Decides in code: reply_only (rule W1) decides: what to propose

   rules.py applies rule W1: LK-646556 is delivered. Outcome reply_only, no change.

5. 5 Model The model drafts the reply from the decision decides: the words of the reply

   A plug that gets very warm and has a plastic smell may be unsafe. Please unplug the charging dock and stop using it for now. Your order LK-646556 has been delivered.

   288 tokens in, 40 out, 1.16 s

6. 6 Code Proposal: reply_only (rule W1), done: reply only

   Nothing is written: this outcome needs no change.

T-90706, real recordings of gpt-6-luna as chat-small on Azure, replayed by the project's own code. The orange step is where the fixed workflow goes wrong without a sign.

Text description of the explorer

The explorer starts with ticket T-90706, the warm plug, and the fixed workflow. A second button shows T-90103, the cracked planter.

Fixed workflow, 6 steps. 1, code: no keyword of any list matches, so the code falls through to its default kind, status. This is the quiet failure: nothing says that the lists did not understand the ticket. 2, code: the regular expression finds LK-646556. 3, code: it reads the order: the robot vacuum, 329.00 dollars, delivered on 2 October. 4, code: rules.py applies W1 and decides reply_only. 5, model: the reply draft says “may be unsafe, please unplug the charging dock”, then “Your order LK-646556 has been delivered”. 6: the result is reply_only, done. The policy expects hand_to_person under H1.

Router, 5 steps. 1, model: it picks the path hand_to_person with rule H1. 2, code: it reads nothing. 3, code: it accepts the path. 4, model: it drafts “A team member will look at your ticket personally.” 5: handed to a person. Right.

Agent, 5 steps. 1, model: it asks for get_order. 2, code: it runs the read. 3, model: it writes the resolution hand_to_person, rule H1, with a reply in French. 4, code: it accepts the resolution. 5: handed to a person. Right outcome, but the reply is in French for an English ticket.

The fixed workflow replies with the delivery status, and no person sees the ticket. So both sides have a real failure to point at. But one ticket each proves nothing: Amira needs the same measure on many tasks.

The idea: the simplest design that works, measured

The rule that this lesson teaches: start with the simplest design, and add model decisions only where a measurement shows that they pay for themselves. “Simplest” means the fewest decisions by the model. “Works” needs a measure on your own tasks.

Three questions decide how much model a task needs:

1. Variability. How much do the inputs vary? Fixed words, such as a form or a database row, need no model. Free text needs a model to read it. Steps that differ from task to task need a model to choose them.
2. Cost of an error. What does a wrong decision cost, and can something catch it before it happens? A wrong reply costs an apology. A wrong refund costs money.
3. Observability. When a run is wrong, can you see why? Code that decides can be read and tested. A model’s decision shows only its inputs and its output.

Then cost and time decide between the designs that remain.

Add the evaluation to the project

The project can run one ticket. To compare designs, it needs to score all 70 tasks in the same way. The score is simple: a proposal is right if it has the expected outcome (or an accepted alternative) and exactly the expected actions. Until Module 4 nothing is written, so this checks the proposal only. A run that stops counts as handed to a person, which is right only where a person was expected.

1. Make a new file, resolver/evaluate.py, with this code:

   resolver/evaluate.py

   """Compare designs on the same tasks: did each one propose the right resolution, and at what cost?
   
   Until Module 4 nothing is written, so this scores the PROPOSAL: the outcome (resolve, reply_only,
   ask_customer, hand_to_person) and the exact actions. Module 4 adds approval and the writes; then the
   evaluation also checks what changed in the database.
   """
   
   import shutil
   import statistics
   import tempfile
   from dataclasses import dataclass
   from pathlib import Path
   
   from .data import SEED_DB, Task
   from .state import TaskState
   from .systems import Faults, World
   
   MATCH_FIELDS = ("tool", "order_id", "sku", "quantity", "amount", "reason")
   
   
   def norm(action: dict) -> tuple:
       a = dict(action)
       code = a.pop("reason_code", None)
       if code is not None:
           a["reason"] = code
       if a.get("amount") is not None:
           a["amount"] = round(float(a["amount"]), 2)
       return tuple(a.get(f) for f in MATCH_FIELDS)
   
   
   def same_actions(got: list[dict], expected: list[dict]) -> bool:
       return sorted(map(norm, got), key=str) == sorted(map(norm, expected), key=str)
   
   
   @dataclass
   class TaskScore:
       task_id: str
       slice: str
       expected: str
       outcome: str                 # a stopped run goes to a person
       proposal_ok: bool            # the right outcome and exactly the right actions
       stop_reason: str
       approvals: int               # proposals with actions: each one will need a person's approval
       steps: int = 0
       tool_calls: int = 0
       refused_calls: int = 0
       input_tokens: int = 0
       output_tokens: int = 0
       cost_usd: float | None = 0.0
       seconds: float = 0.0
   
   
   def score(task: Task, state: TaskState) -> TaskScore:
       p = state.proposal
       if p is None:   # a stopped run goes to a person: right when a person was expected
           outcome, ok = "hand_to_person", "hand_to_person" in task.outcomes_accepted
       else:
           outcome = p.outcome
           ok = p.outcome in task.outcomes_accepted and same_actions(
               [a.model_dump() for a in p.actions], task.expected["actions"] if p.outcome == task.expected["outcome"] else [])
       return TaskScore(task_id=task.task_id, slice=task.slice, expected=task.expected["outcome"], outcome=outcome,
                        proposal_ok=ok, stop_reason=state.stop_reason, approvals=int(bool(p and p.actions)),
                        steps=state.step, tool_calls=len(state.evidence),
                        refused_calls=sum(1 for e in state.evidence if not e.ok), input_tokens=state.usage.input_tokens,
                        output_tokens=state.usage.output_tokens, cost_usd=state.usage.cost_usd,
                        seconds=state.usage.model_seconds)
   
   
   def run_and_score(task: Task, variant: str, model: str, complete, repeat: int = 1,
                     workdir: Path | None = None) -> tuple[TaskScore, TaskState]:
       from .runner import run_task
   
       workdir = workdir or Path(tempfile.mkdtemp(prefix="resolver-eval-"))
       db = workdir / f"{task.task_id}.sqlite"
       shutil.copyfile(SEED_DB, db)
       world = World(db, task.task_id, Faults(task.faults))
       state = run_task(task, variant, model, lambda request, meta: complete(request, {**meta, "repeat": repeat}), world)
       return score(task, state), state
   
   
   def summarise(scores: list[TaskScore]) -> dict:
       n = len(scores)
       costs = [s.cost_usd for s in scores if s.cost_usd is not None]
       return {
           "tasks": n,
           "proposal_ok": sum(s.proposal_ok for s in scores),
           "approvals_requested": sum(s.approvals for s in scores),
           "stopped": sum(s.stop_reason not in ("finished", "") for s in scores),
           "steps_mean": round(statistics.mean(s.steps for s in scores), 2) if n else 0,
           "tool_calls": sum(s.tool_calls for s in scores),
           "refused_calls": sum(s.refused_calls for s in scores),
           "input_tokens": sum(s.input_tokens for s in scores),
           "output_tokens": sum(s.output_tokens for s in scores),
           "cost_usd": round(sum(costs), 4) if len(costs) == n else None,
           "seconds_median": round(statistics.median(s.seconds for s in scores), 2) if n else 0,
       }
   
   
   def by_slice(scores: list[TaskScore]) -> dict:
       out = {}
       for s in scores:
           out.setdefault(s.slice, []).append(s)
       return {k: {"tasks": len(v), "proposal_ok": sum(x.proposal_ok for x in v)} for k, v in out.items()}

2. Copy resolver/__main__.py from the snapshot end-of-m01 over your file. On macOS and Linux, from the project folder:

   cp ../nextia-learning-labs/C10/snapshots/end-of-m01/resolver/__main__.py resolver/

   On Windows: Copy-Item ..\nextia-learning-labs\C10\snapshots\end-of-m01\resolver\__main__.py resolver\.

3. Copy the two new test files from the same snapshot into tests/: tests/test_workflow.py and tests/test_evaluate.py.

4. Run the tests:

   python -m pytest

   The last line is 23 passed.

Four details in evaluate.py matter for the numbers:

• Each task runs on a fresh copy of the practice database, so one task cannot change the next.
• same_actions compares the actions as a set: the same tool, order, item, quantity, amount and reason. An extra action makes the proposal wrong, even if the expected one is there too.
• approvals counts the proposals that have actions. Each one will need a person’s approval in Module 4, so it measures work for Grace’s team.
• cost_usd uses the prices of the recordings in usage.py. chat-small costs US$0.10 per million input tokens and US$0.50 per million output tokens. chat-strong costs US$2.00 and US$10.00. Prices change: check the current price page.

The new __main__.py adds two commands: eval scores one design on all tasks, and compare prints one line per recorded design. tests/test_workflow.py keeps the fixed workflow’s real keyword bugs as tests. Your project now matches the snapshot end-of-m01.

The labels have one more thing to know. The course author wrote the expected outcomes by applying Grace’s written policy, and a script checked them. It is one person’s reading; another reader could decide some boundary cases differently.

Run the comparison

1. Score the fixed workflow:

   python -m resolver eval --variant fixed

2. Score every recorded design in one table:

   python -m resolver compare

The first command prints the score, then the slices, then each wrong task:

fixed, chat-small, repeat 1
Right proposal: 60/70 | approvals it would ask for: 19 | stopped: 0 | steps per task: 1 | tool calls: 58 (5 refused) | tokens: 20643 in, 1881 out | US$0.0030 | model time per task: median 1.2 s
By slice: simple 14/16, multi_step 9/11, needs_person 10/10, policy 9/9, adversarial 8/8, failure 8/8, new_wording 2/8
  T-64872 simple: expected resolve, got hand_to_person
  T-65273 simple: expected resolve, got hand_to_person
  T-90204 multi_step: expected resolve, got hand_to_person
  T-90205 multi_step: expected resolve, got hand_to_person
  T-90701 new_wording: expected resolve, got reply_only
  T-90702 new_wording: expected resolve, got reply_only
  T-90703 new_wording: expected resolve, got reply_only
  T-90705 new_wording: expected resolve, got reply_only
  T-90706 new_wording: expected hand_to_person, got reply_only
  T-90708 new_wording: expected resolve, got reply_only

The second prints the whole comparison. cost US$ is the cost of the 70 recorded tasks, and median s is the median time spent waiting for the model per task:

design             model          right  approvals  steps  cost US$  median s
fixed              chat-small     60/70         19      1    0.0030       1.2
router             chat-small     66/70         25      2    0.0170      2.29
router             chat-strong    65/70         24      2    0.3811      3.74
agent              chat-small     61/70         25   2.14    0.0404      2.91
agent_structured   chat-small     64/70         27      2    0.0399      3.49
agent_structured   chat-strong    70/70         28   1.93    0.7771      5.91
agent_structured   gemma3:4b      28/70          4    4.9    0.0000     19.79

Every line is repeat 1 of the recordings, and every proposal is scored as it is, before any approval. Now take the three questions one at a time.

Variability: the fixed workflow is blind to new words

The 70 tasks are in slices. One slice, new_wording, has 8 tickets with the same kinds of request in words that nobody planned for. Look at the fixed workflow by slice: 58 right of the other 62, but 2 of 8 on new wording.

• code reads the ticket
• chat-small reads the ticket
• chat-strong reads the ticket

The new-wording slice, repeat 1, every proposal scored as it is. fixed, router and agent use chat-small; the strong agent is chat-strong with JSON steps.

Text description of the chart

A column chart of right proposals out of 8 new-wording tickets. Fixed workflow: 2. Router with chat-small: 8. Agent with chat-small: 7. Agent with chat-strong and JSON steps: 8. On the other 62 tasks, the fixed workflow and the router both get 58 right. The chat-small agent gets 54 and the chat-strong agent 62.

On the 62 tasks written while the lists were in view, the fixed workflow and the router tie at 58. On the 8 new ones, the router gets all 8 and the fixed workflow 2. The model reads words that no list contains; the keyword lists cannot.

There is a reason to trust this slice more than the others. The keyword lists and the first 62 tasks were written by the same person, the course author. So the lists may fit those tasks better than they would fit real tickets. To limit this, the lists were frozen in the project’s history first, and the 8 new_wording tasks were written after that. Avoid contamination in the evaluation course explains why a test written with the answer in view flatters the system.

The keyword lists also have two real bugs, kept in tests/test_workflow.py. T-65273 says “my delivery issue”. The word “issue” contains “sue “, a keyword of rule H2 for legal threats, so a double charge went to a person. And T-64872, “Charged twice for terracotta pots LK-739156”, has no common English word such as “the” or “my”. The language check calls it unreadable (rule H5). Each bug is easy to fix, but the next new ticket can miss the next list.

Common mistake

The quiet miss

Symptom: the safety ticket T-90706 gets a status reply. Run it:

python -m resolver run T-90706 --variant fixed

Run T-90706-fixed-chat-small | fixed workflow: code decides every step; the model drafts the reply | chat-small
Outcome: reply_only (rule W1) | 1 model call(s), 1 tool call(s) | 288 tokens in, 40 out | US$0.00005
Reply draft: A plug that gets very warm and has a plastic smell may be unsafe. Please unplug the charging dock and stop using it for now. Your order LK-646556 has been delivered.

Cause: keyword_kind() found no word of any list, and its last line returns "status". “So warm I can barely touch it” is not “very hot” or “really hot”. The code has no way to say “I did not understand”.

Diagnose: the trace shows it: keywords {"rule": null, "request_kind": "status"}. A status ticket with no word about a delivery is suspicious. But you only see it if you read the trace, and nothing makes you read it.

Fix: never let an unknown input fall through to a normal path. Either send it to a person, or let a model read it, as the router does. A default is a decision too.

What it means: a design that fails quietly is more dangerous than one that fails often. The router failed 4 tasks of 70; the fixed workflow failed 10, and its worst one looks like a success.

Cost of an error: who catches the mistake?

A wrong decision is cheap when something catches it first. Here, the safety net is a person’s approval, which Module 4 builds. The approvals column counts that work. The fixed workflow asks on 19 tasks, the router and the chat-small agent on 25, the chat-strong agent on 28. The labels have 28 tasks with actions.

The worst errors are proposals with actions that nobody asked for, because a person in a hurry may approve them. The chat-small agent made two in this run. It reshipped a chair on T-90207, where the policy says to ask which order. And on T-90305, a ticket in German, it read the German well and proposed a reship. Grace’s policy says that a ticket in a language other than English or French goes to a person (H5). The router made no proposal with an action that the policy did not expect. Module 4 measures what each design would write if every proposal were approved.

So the cost of an error grows with the model’s freedom. In the router, rules.py decides the amounts. In the agent, the model writes them, and code can only check their form.

Observability: can you see why?

When the router fails, you read one thing: the route. On T-80002, "path": "late" showed at once that a signal was lost. When the fixed workflow fails, the keyword note shows which word did it.

When an agent fails, you see its calls and its resolution, but not why it chose them. On T-90706, the agent got the outcome right and wrote the reply in French for an English ticket. Its trace shows no reason for that. To find failures like this, you must read the outputs, or write a check for each kind of failure. Module 6 builds those checks.

Cost and time

Here is the cost of each design per 1,000 tickets, from the recorded tokens and the prices above.

• chat-small, no model decisions
• chat-small
• chat-strong

Cost per 1,000 tickets: the cost of the 70 recorded tasks, scaled to 1,000. Prices of the recordings: chat-small US$0.10 / 0.50, chat-strong US$2.00 / 10.00 per million input / output tokens.

Text description of the chart

A column chart of the cost per 1,000 tickets. Fixed workflow: US$0.04. Router with chat-small: US$0.24. Agent with chat-small: US$0.58. Agent with chat-strong and JSON steps: US$11.10. The chat-strong agent costs about 46 times the router.

The chat-strong agent is the most accurate design: 70 of 70, and 70, 69 and 69 in three repeats. But it costs about 46 times the router. It also waits more than twice as long for the model: a median of 5.91 s per task against 2.29 s. One of its 70 also depends on a label that changed after the runs. On T-65001, the label review followed the models’ reading of the policy, chat-strong’s among them.

For Larkfield’s 45 tickets a day, all of these are small. The router costs about US$0.011 a day, the chat-small agent US$0.026 and the chat-strong agent US$0.50. The ratio is what matters when the volume grows.

Two more lines of the table teach something. The router with chat-strong scored 65, one less than with chat-small. A stronger model does not fix a limit of the design: a router still has room for one path. And gemma3:4b, a free model on your own computer, runs the same loop but gets 28 of 70. “Free” costs you in decisions.

Tomás’s question, answered with numbers

Back to “make it an agent”. The numbers say:

• The router with chat-small is the default for Larkfield. It got 66 of 70 and 8 of 8 on new wording, with no unexpected action, for US$0.24 per 1,000 tickets. The model reads the words; code makes the decisions that cost money.
• The fixed workflow is cheap and good on the words it knows, 60 of 70. But it fails quietly on new words, including a safety case. Use it only where the input is not free text.
• The chat-small agent is not better than the router here. It got 61 of 70 at more than twice the cost, with proposals that nobody asked for.
• The chat-strong agent earns its place only when tasks vary beyond what a router can name. That means steps that differ from ticket to ticket, such as T-90201 with two orders and two rules.

Choose the simplest design that can work, then measure it. The questions follow variability and the cost of an error; the last box adds observability and cost.

Text description of the diagram

Start with a new kind of support request.

1. Can a wrong action be stopped before it happens, for example by an approval? If not, a person decides and the system only drafts.
2. If yes: is the wording always the same, because it comes from a form or another system? If yes, use a fixed workflow: code decides every step.
3. If it is free text: can you name every path in a short list? If yes, use a router: the model picks the path, and code reads and decides.
4. If the steps differ from task to task: use an agent. The model chooses the steps, and code checks every one.

Whatever you choose, measure it on your own tasks. Check new wording, the cost of an error, the money and the time. Check also whether you can see why a run failed.

Module practice: choose a pattern for three requests

Worked example

Larkfield’s website gets a new form: “Where is my order?”. It has one field, the order ID, and the customer is signed in. Amira follows the picture:

1. Can a wrong action be stopped? There is no action: the answer is a reply with the status.
2. Is the wording always the same? Yes: the form sends an order ID, not free text.
3. So a fixed workflow is enough: read the order, apply rule W1, fill a reply template. No model is needed.
4. What to measure: that every order status gives the right reply. A test per status covers it, because code decides everything.

Why it is sufficient: every input has the same shape, and rule W1 is a date calculation. A model would add cost and variation and remove nothing.

Your task

Choose a design and the patterns for each of these three fictional requests. For each one, answer the four questions of the worked example. Then write one sentence that says why your choice is sufficient: why a simpler design would fail, and why a more complex one adds nothing.

1. Nightly stock notes. Every night, the warehouse system sends a file with one row per item damaged in the warehouse: SKU, quantity, reason code. For each row, the system must add a stock adjustment note.
2. Camille’s returns inbox. Camille, who handles French tickets, gets 60 emails a day in French and English about returns. Each one must go to one of five return reasons, and each reason has its own fixed steps and a label to propose.
3. A business customer’s question. A garden-centre chain has 12 open orders. It asks: “Which of our orders will miss the opening of our new store on 15 November, and what can you do about each one?” The answer needs reads of several orders, their boxes and the delivery policy. The order of the reads depends on what each read shows.

Show a model answer

1. Fixed workflow, a chain. The input is a file with fixed columns, not free text. The rule for each reason code can be written in code, and a wrong note can be corrected by a person. Sufficient because no step needs to read words; a model would only add cost and runs that differ.
2. Router. The emails are free text in two languages, so code cannot read them reliably. But the paths are a known list of five, and each path is fixed. The model picks the reason; code reads the order and decides the label; a person approves it. Sufficient because one model decision covers the only varied part. An agent would let the model also choose reads and actions that the five paths already fix.
3. Agent, planner–executor, with parallel reads. The steps differ by order: one order needs its boxes, another the delivery policy. The model plans and chooses the reads; code runs read-only tools and checks every proposal, and a person approves any change. Sufficient because no short list of paths can name every combination. Measure it on several such questions before you trust it, and watch the cost per question.

See it end to endWorkflow or agent? Automate an open-data processCase study

The open-data case study compares a fixed workflow, a router and an agent on 300 real city requests, with their cost and time.

Knowledge check

Check your understanding

Name the three questions that decide how much model a task needs, and give the project's real example for each.

Show the answer

Variability: how much do the inputs vary? The fixed workflow got 2 of 8 new-wording tickets right; the router got 8 of 8. Cost of an error: what does a wrong decision cost, and can something stop it? The chat-small agent proposed a reship on the German ticket T-90305, and a chair for one of two orders on T-90207. Only an approval stops those. Observability: can you see why a run failed? The fixed workflow’s keyword note and its tests show which word misled it. The agent’s French reply to T-90706 had no visible reason.

Check your understanding

Omar, the product manager, asks: 'chat-strong was right on all 70. Why not use it everywhere?' What do you answer, with numbers?

Show the answer

It is the most accurate design, but it costs US$11.10 per 1,000 tickets, about 46 times the router’s US$0.24. It also waits more than twice as long for the model (median 5.91 s against 2.29 s). The router with chat-small got 66 of 70, all 8 new-wording tickets, and proposed no unexpected action. Use chat-strong as an agent only where tasks vary beyond what a router can name. Also, 70 tasks with one repeat is a small sample, and one of its 70 depends on a label that changed after the runs.

Check your understanding

Completion check: for each of the three requests in the module practice, name your design and justify why it is sufficient: why a simpler design would fail, and why a more complex one adds nothing.

Show the answer

Nightly stock notes: a fixed workflow. The input has fixed columns and the rules fit in code, so nothing needs a model; a model would add cost and variation. Camille’s returns inbox: a router. Free text in two languages needs a model to read it, so a fixed workflow would miss new words. But the five paths are known and fixed, so an agent would only add decisions that code already makes well. The business customer’s question: an agent with a plan and parallel read-only reads. The steps depend on what each read shows, so no short list of paths fits. Code still checks every step, and a person approves any change.

Recap

• Choose the simplest design that works, and add model decisions only where a measurement shows that they pay. Three questions decide: variability, the cost of an error, observability. Then compare cost and time.
• On the 70 tasks: fixed workflow 60, router (chat-small) 66, agent (chat-small) 61, agent (chat-strong, JSON steps) 70. Per 1,000 tickets: US$0.04, US$0.24, US$0.58 and US$11.10.
• The fixed workflow got 2 of 8 new-wording tickets right, and it fails quietly: the safety ticket T-90706 got a status reply.
• The router with chat-small is Larkfield’s default. An agent earns its place only when tasks vary beyond what a router can name.
• The project now has eval and compare, and python -m pytest shows 23 passed: it matches the snapshot end-of-m01.

Module 2 starts from this project. It gives each task an explicit state, gives each tool a contract, and makes the loop stop.

Next: Explicit task state