Workflow and agent boundaries

What you will be able to do

See the difference between a workflow and an agent in one real ticket: who decides each step, the code or the model. Then set up the course project, so that you can run every design yourself.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• How an application offers a tool to a model and runs the call itself, from Execute in application code in Building Reliable Applications with LLM APIs.
• Why a loop with a model needs limits, from Bound the loop.
• What an evaluation set is, from Select representative cases in Evaluating and Testing AI Systems. Lesson 3 of this module uses one.

What you need on your computer

• Python 3.12 or newer, and a terminal. Isolate dependencies shows how a virtual environment works.
• An editor such as VS Code, and an internet connection to download the project and its packages (about 70 MB with the environment).
• Git is optional: a ZIP file works too.

You need no account, no API key and no payment. Every model decision in this course has a real recorded response, and the project replays it.

Tested with Python 3.12.13 on macOS (Apple silicon), with openai 3.27.0, pydantic 2.14.0 and pytest 9.1.1. The Windows and Linux commands follow the Python course; they were not run for this lesson.

Models and agent frameworks change fast, so the numbers in this course are a snapshot. The method lasts longer than the numbers.

A recipe and a cook

You can make dinner in two ways. With a recipe, every step is written down before you start: chop the onion, fry it for five minutes, add the rice. You follow the steps in order. If the shop had no onions, the recipe does not tell you what to do.

Or you can ask a cook to make something from what is in the fridge. The cook looks, decides what to make, and chooses each next step from what they find. A good cook handles the missing onion. But you cannot know the steps in advance, and two evenings with the same fridge can give two different dinners.

Most kitchens use both. The restaurant’s menu is a set of recipes, and the cook decides only where a recipe cannot.

The problem: who decides the next step?

In Building Reliable Applications with LLM APIs, Larkfield’s assistant read a ticket, looked up the order and drafted a reply. A person did the rest. Grace, the support lead, now wants more. For common cases, such as a late delivery or a damaged item, the system should resolve the ticket. It looks up the facts, applies her policy and proposes an action, such as a return label or a refund.

To resolve a ticket, something must decide five things. What does the ticket ask? Which order is it about? Which facts must be read? What should happen? And what does the reply say?

Tomás, an AI engineer, has a quick answer: “Make it an agent and let the model decide everything.” Grace has a different worry: “Show me exactly what you will do before you do it.” Both have a point. And neither says who should make each of the five decisions. That is the question of this module.

The idea: a workflow, an agent, and the line between them

A workflow is a task whose steps are fixed in code, like a recipe. The code decides what happens next. A model can still do one job inside a step, for example write the reply.

An agent is a model that chooses its own next step, like the cook. It works in a loop: it asks for a tool (a function that the application runs, such as get_order), sees the result, and decides again. It stops when it thinks the task is done. The name says what is new: the model acts for the task, as an agent acts for a client. It does more than write text.

Between the two sits the router. The model makes one decision, which path the ticket takes, and code does everything after that. Routing means sending each item to the right path, as a post office sorts letters.

Orchestration is the name for the whole job of arranging the steps: which ones run, in what order, and who decides. This course is about orchestration. The picture shows the line between code and model in the three designs. Each box is one of the five decisions.

Who decides each of the five decisions in the three designs of the project. Green boxes: code. Blue boxes with a thick border: the model.

Text description of the diagram

Three designs, each with the same five decisions. From top to bottom: what the ticket asks, which order, which facts to read, what to propose, and the words of the reply.

1. Fixed workflow: code decides the first four. Keyword lists find what the ticket asks. A regular expression finds the order ID. The code always reads the order. The file rules.py decides the proposal. The model writes only the reply, in one call.
2. Router: the model decides what the ticket asks and which order, in one answer: the path. Then code decides the reads and the proposal, with the same rules.py. The model writes the reply.
3. Agent: the model decides all five. Its first tool call shows what it thinks the ticket asks and which order. It chooses each next read. Its last call, finish, holds the proposal and the reply.

Read the picture from left to right: the blue boxes grow. Every blue box is a place where the result can change from run to run, and where the wording of a new ticket matters. Every green box always gives the same answer for the same input. Lesson 3 of this module measures what each one costs.

Set up the project

The project of this course is resolution-workflow. It is in the labs repository of Nextia Learning, as snapshots: copies of the project at the start and at the end of each module. You start from start.

The tickets, orders and payments are made up for this course. No real customer wrote them, and every write goes to a local practice database: a SQLite file in the project folder. Nothing reaches a real system. Until Module 4, nothing is written at all: every design only proposes.

Get the files

1. Open a terminal and move to your projects folder. If it does not exist, make it first with mkdir ~/projects.

   cd ~/projects

2. If you have Git, clone the labs repository:

   git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.git

   If you do not have Git, download the ZIP file of the repository. Unzip it in ~/projects. Its folder is nextia-learning-labs-main: use that name in the next step.

3. Copy the start snapshot into a new folder, resolution-workflow:

   macOS

   cp -R nextia-learning-labs/C10/snapshots/start resolution-workflow

   Windows

   Copy-Item -Recurse nextia-learning-labs\C10\snapshots\start resolution-workflow

   Linux

   cp -R nextia-learning-labs/C10/snapshots/start resolution-workflow

4. Move into the project and open it in your editor:

   cd resolution-workflow
   code .

   If code is not found, open the folder from the editor’s File menu.

You work in your copy. The labs folder stays as it is, so you can copy a snapshot again at any time.

Make the environment

1. Make and activate a virtual environment:

   macOS

   python3 -m venv .venv
   source .venv/bin/activate

   Windows

   python -m venv .venv
   .venv\Scripts\Activate.ps1

   Linux

   python3 -m venv .venv
   source .venv/bin/activate

   The prompt now starts with (.venv). From here, type python on every system.

2. Install the packages:

   python -m pip install -r requirements.txt

   The last line of the output:

   Successfully installed annotated-types-0.8.0 anyio-4.15.1 h11-0.16.0 httpcore2-2.13.1 httpx2-2.13.1 idna-3.20 iniconfig-2.3.1 jiter-0.17.0 openai-3.27.0 packaging-26.3 pluggy-1.6.0 pydantic-2.14.0 pydantic-core-2.50.0 pygments-2.21.0 pytest-9.1.1 sniffio-1.3.1 truststore-0.10.4 typing-extensions-4.16.0 typing-inspection-0.4.4

   A notice about a new pip version is safe to ignore.

requirements.txt asks for three packages: openai, pydantic and pytest. The others are what those three need. The project uses no agent framework: plain Python shows every step, and Module 4 compares one framework with it.

The known-good first run

1. Run the tests:

   python -m pytest

   The end of the output:

   tests/test_config.py ..                                                  [ 14%]
   tests/test_first_run.py ..                                               [ 28%]
   tests/test_replay.py ....                                                [ 57%]
   tests/test_tools.py ......                                               [100%]
   
   ============================== 14 passed in 0.64s ==============================

2. Run the course’s first ticket, T-80008:

   python -m resolver run T-80008

   The output:

   Run T-80008-agent-chat-small | single agent with tool calling: the model chooses every step | chat-small
   Outcome: reply_only (rule W1) | 2 model call(s), 1 tool call(s) | 4580 tokens in, 117 out | US$0.00052
   Reply draft: Your garden shed order LK-581106 shipped on Monday, October 5. Its expected delivery date is Tuesday, October 13. It is currently in transit, and there is no delivery date showing yet.

Your output must be the same, character for character, except the test time. The mock provider replays the recorded answers of a real model, gpt-6-luna as chat-small on Azure, so nothing is random. US$0.00052 is what the recorded calls cost; your run cost nothing. The first run also made work/larkfield.sqlite, your practice database: a copy of data/larkfield.sqlite.

The setup check. 14 passed tests and the known-good line for T-80008 mean that you are ready.

Text description of the diagram

First run python -m pytest. If you do not see “14 passed”, check that the prompt starts with (.venv), install the requirements again, and run the tests again. When you see “14 passed”, run python -m resolver run T-80008. If the Outcome line ends with “4580 tokens in, 117 out”, the setup works. If it does not, reset: copy the start snapshot again into a new folder, make a new .venv, and start the checks again.

What is in the project

Path	What it holds
data/	The 70 tasks (tasks.jsonl), the practice database, Grace’s policy and 24 policy passages. dataset.md describes them.
recordings/	The recorded model decisions, one file per design and model.
resolver/workflow.py	The fixed workflow and the router.
resolver/agent.py, resolver/loop.py	The agent and the loop around it.
resolver/rules.py	Grace’s rules W1 to W8, written as code.
resolver/tools.py	The read tools: get_order, get_customer_orders, get_payments, search_policy.
tests/	The 14 tests that you ran.

Reset the project

To start again, deactivate the environment, rename your folder (for example mv resolution-workflow resolution-workflow-old, or Rename-Item on Windows), and copy the snapshot again. To reset only the practice database, delete the folder work. The next command makes a fresh copy.

One ticket, three designs

Now the main event. T-90103 is a simple ticket that every design gets right, so you can compare how, not whether.

1. Show the ticket and what Grace’s policy expects:

   python -m resolver show T-90103 --expected

   T-90103 | customer C-50533 | slice simple
   Attachments: IMG_3381.jpg
   My cedar planter (order LK-640436) came with one side panel cracked right through. It must have happened in transit, the box was crushed. I've attached a photo. I'd like a replacement please.
   Expected: resolve (rule W5) [{'order_id': 'LK-640436', 'quantity': 1, 'sku': 'PLANTER', 'tool': 'reship_item'}, {'order_id': 'LK-640436', 'reason': 'damaged', 'sku': 'PLANTER', 'tool': 'create_return_label'}]
   Why: Delivered 5 October, photo, 89 dollars: reship and a free return label for the damaged planter.

2. Run the fixed workflow, with its trace:

   python -m resolver run T-90103 --variant fixed --trace

3. Run the router and the agent the same way, with --variant router and --variant agent.

The line Expected is the task’s label: the outcome and the exact actions that the policy gives. The course author wrote the expected outcomes by applying Grace’s written policy, and a script checked them. It is one person’s reading; another reader could decide some boundary cases differently.

Rule W5 of Grace’s policy covers a damaged item: within 14 days, with a photo, send a new one if it is in stock. Items of 50 dollars or more also get a free return label. All three designs propose exactly that. The traces show who decided it. A trace is the timeline of one run. It has one line per event, such as a model call, a tool call or the proposal. Here is the fixed workflow’s trace:

  0  note      keywords {"rule": null, "request_kind": "damaged"}
  0  tool      get_order({"order_id": "LK-640436"}) -> ok
  1  model     chat-small: fixed  (352 in, 28 out, 1.057 s)
  1  proposal  resolve (W5): reship_item LK-640436, create_return_label LK-640436
  1  stop      finished

Code found the word “cracked” in a keyword list and called the request damaged. Code read the order, and rules.py made the proposal. The model was called once, at step 1, only to write the reply. The router’s trace starts with a model call instead:

  1  model     chat-small: router  (1799 in, 41 out, 1.098 s)
  1  note      route {"path": "damaged", "rule": null, "order_id": "LK-640436", "product": "cedar planter", "item_used":…
  1  tool      get_order({"order_id": "LK-640436"}) -> ok
  2  model     chat-small: router_draft  (352 in, 28 out, 1.057 s)
  2  proposal  resolve (W5): reship_item LK-640436, create_return_label LK-640436

The model read the ticket and chose the path damaged, with the order and the product. After that, the same code as the fixed workflow took over. In the agent’s trace, both model calls are decisions:

  1  model     chat-small: get_order({"order_id":"LK-640436"})  (2185 in, 21 out, 1.099 s)
  1  tool      get_order({"order_id": "LK-640436"}) -> ok
  2  model     chat-small: finish({"outcome":"resolve","rule":"W5","actio…)  (2433 in, 199 out, 2.295 s)
  2  proposal  resolve (W5): reship_item LK-640436, create_return_label LK-640436

The model chose to read the order, then wrote the whole resolution itself in a call to finish. Code still ran the read and checked the resolution, but it decided neither. The cost lines tell the price of that freedom. The fixed workflow used 352 tokens in, the router 2,151 and the agent 4,618.

Step through the three runs in the explorer. Watch the chips at the top: they show who owns each of the five decisions.

Explore

One ticket through a fixed workflow, a router and an agent

T-90103, customer C-50533: “My cedar planter (order LK-640436) came with one side panel cracked right through. It must have happened in transit, the box was crushed. I've attached a photo. I'd like a replacement please.” Attachment: IMG_3381.jpg.

Grace's policy expects resolve (rule W5). Real recordings: gpt-6-luna as chat-small on Azure, replayed by the project's own code.

Who decides in the fixed workflow

• Code What does the ticket ask?
• Code Which order?
• Code Which facts to read?
• Code What to propose?
• Model The words of the reply

Model calls

1

Tokens so far

352 in, 28 out

Decisions so far

4 by code, 1 by the model

Result

resolve (W5)

Fixed workflow, step 6 of 6. 1 model call(s), 352 tokens in, US$0.00005. This is what Grace's policy expects: resolve (rule W5).

1. 1 Code Keyword lists: the kind is damaged decides: what the ticket asks

   The ticket contains "cracked", so the kind is damaged. No H1-H5 keyword matched.

2. 2 Code A regular expression finds LK-640436 decides: which order

   The pattern LK- and six digits finds the order ID in the ticket text.

3. 3 Code Reads get_order(LK-640436) decides: which facts to read

   The code always reads the order for this kind. Result: LK-640436: delivered 2026-10-05, 4 days ago; cedar planter 89.00 dollars (in stock).

4. 4 Code Decides in code: resolve (rule W5) decides: what to propose

   rules.py applies rule W5: delivered 4 days ago; in stock: True. Proposal: reship_item LK-640436 PLANTER x1; create_return_label LK-640436 PLANTER (damaged).

5. 5 Model The model drafts the reply from the decision decides: the words of the reply

   We’ll arrange a replacement cedar planter and provide a return label for the damaged one once a team member has checked your ticket.

   352 tokens in, 28 out, 1.06 s

6. 6 Code Proposal: resolve (rule W5), waiting for a person's approval

   Nothing is written. Every design only proposes; a person's approval comes in Module 4.

T-90103, real recordings of gpt-6-luna as chat-small on Azure, replayed by the project's own code. Green: code decides. Blue: the model decides.

Text description of the explorer

The explorer shows ticket T-90103, the cracked cedar planter, and one design at a time. Five chips show the decision points: what the ticket asks, which order, which facts to read, what to propose, and the words of the reply. Each chip says Code or Model.

Fixed workflow, 6 steps. 1, code: the keyword lists find “cracked”, so the kind is damaged. 2, code: a regular expression finds LK-640436. 3, code: it reads get_order; the planter was delivered on 5 October, 4 days ago, costs 89.00 dollars and is in stock. 4, code: rules.py applies W5 and proposes reship_item and create_return_label. 5, model: it drafts the reply (352 tokens in, 28 out). 6: the proposal waits for a person’s approval; nothing is written. Four decisions by code, one by the model.

Router, 5 steps. 1, model: it picks the path damaged, with the order LK-640436 (1,799 tokens in, 41 out). 2, code: it reads the order. 3, code: rules.py applies W5. 4, model: it drafts the reply. 5: the proposal. Two decisions by code, three by the model.

Agent, 5 steps. 1, model: it asks for get_order with LK-640436 (2,185 tokens in, 21 out). 2, code: it checks that the order is this customer’s and runs the read. 3, model: it writes the resolution with finish (2,433 tokens in, 199 out). 4, code: it checks the resolution and accepts it. 5: the proposal. No decision by code, five by the model; code only runs and checks.

In the agent, code makes none of the five decisions. It still has a job: it runs the read, checks that the order belongs to this customer, and checks the shape of the resolution. That boundary, between deciding and checking, comes back in every module.

Guided practice: the neighbour’s order

T-80001 asks about another customer’s order. Tom writes for his neighbour, Mrs Okafor. Rule H4 says: a person handles it. Predict, for each design, whether it reads the order before it decides.

1. Show the ticket: python -m resolver show T-80001 --expected. The expected outcome is hand_to_person (rule H4).

2. Run the three designs:

   python -m resolver run T-80001 --variant fixed
   python -m resolver run T-80001 --variant router
   python -m resolver run T-80001 --variant agent

3. Check the second line of each output. All three end with hand_to_person (rule H4).

Show the three outcome lines and what they mean

Outcome: hand_to_person (rule H4) | 1 model call(s), 0 tool call(s) | 299 tokens in, 18 out | US$0.00004
Outcome: hand_to_person (rule H4) | 2 model call(s), 0 tool call(s) | 2111 tokens in, 57 out | US$0.00024
Outcome: hand_to_person (rule H4) | 1 model call(s), 0 tool call(s) | 2192 tokens in, 99 out | US$0.00027

No design read the order: 0 tool call(s) everywhere. The fixed workflow found “my neighbour” in its H4 keyword list. The router chose the path hand_to_person with rule H4. The agent made only one call: it went straight to finish. This shows a second freedom of an agent. It chooses not only which steps to take, but also how many. Here it took fewer steps than the router.

Your turn: two orders with garden chairs

T-90207 says: “One of the garden chairs I bought from you arrived with a broken leg. Photo attached.” The customer has two orders with two garden chairs each, and gives no order ID. Grace’s policy expects ask_customer: ask which order. Before you run anything, predict which designs ask, and which one proposes a change. Then run the three designs with --trace.

Show the answer

The fixed workflow and the router ask for the order ID: ask_customer. Their code listed the customer’s orders, found more than one match for “chair”, and stopped. The agent read the list, then both orders, and proposed reship_item LK-641654: a new chair for one of the two orders, picked by the model.

  2  model     chat-small: get_order({"order_id": "LK-641654"}), get_order({"order_id": "LK-641543"})  (2281 in, 55 out, 1.218 s)
  3  proposal  resolve (W5): reship_item LK-641654

The agent saw the same facts as the code, and then guessed. Nothing in its output says that it guessed. If a person approved this proposal without reading it, a chair would go out for an order that may be the wrong one. Module 4 builds the approval that stops this.

Common mistake

Stopped: no_recording

Symptom: Tomás wants the stronger model as the agent and runs:

python -m resolver run T-90103 --variant agent --model chat-strong

The output has no proposal:

Run T-90103-agent-chat-strong | single agent with tool calling: the model chooses every step | chat-strong
Stopped: no_recording. No proposal: the ticket goes to a person.

Cause: the mock provider replays only the decisions that were recorded. chat-strong cannot use tool calling in Chat Completions; Describe a tool showed its real HTTP 400. So the course recorded chat-strong with --variant agent_structured instead: the same loop, with one JSON step per call. No recording exists for chat-strong with --variant agent.

Fix: run python -m resolver run T-90103 --variant agent_structured --model chat-strong. It proposes the same reship and label.

What it means: a run that cannot get a decision stops and hands the ticket to a person. It does not crash, and it does not guess. Module 2 gives every stop like this a name.

Which one to use

The three designs got the planter right. They differ in what they cost and in where they can go wrong. The fixed workflow is cheap and always the same, but it knows only the words in its lists. The router lets the model read the ticket once and keeps the rest in code. The agent can follow a ticket that no one planned for. But it decides more, so it can also go wrong in more places, as T-90207 showed.

The next lesson names five common patterns that sit along this line. Lesson 3 then measures the three designs on all 70 tasks and gives Tomás an answer with numbers.

Knowledge check

Check your understanding

In your own words: what is the difference between a workflow and an agent? Use the five decisions of a support ticket in your answer.

Show the answer

In a workflow, code decides the steps. For a ticket, code finds what it asks and which order it is about, and it decides which facts to read and what to propose. A model may do one job inside a step, such as writing the reply. In an agent, the model chooses its own next step in a loop. It decides which tools to call, in what order and how many, and what to propose. Code still runs each tool and checks each proposal, but it does not choose them.

Check your understanding

Predict: a customer writes a new ticket in words that the keyword lists do not contain. Which design is most likely to misread it, and why?

Show the answer

The fixed workflow. Its code decides what the ticket asks by looking for words in fixed lists. Words that are not in the lists match nothing, and the code falls back to a default path. The router and the agent ask the model to read the ticket, and a model can understand words that no one listed. Lesson 3 measures this on eight tickets written in new words.

Check your understanding

Practical check: run T-90207 with the agent and with the router. What does each propose, and which proposal would change the practice database if a person approved it without reading?

Show the answer

The router proposes ask_customer: it asks which order, because the customer has two orders with garden chairs. The agent proposes resolve with reship_item LK-641654, a new chair for one order that the model picked. Only the agent’s proposal has an action, so only it would change the database if it were approved. The policy expects ask_customer.

Case studies in this course

At the end of the course, you use what you learn on a realistic project, from the data to the recommendation. Complete at least one for the certificate; the others are optional.

• Evaluate a customer-service agent on a public benchmarkRun an agent on public customer-service tasks with mock tools and a written policy, and score task success, policy violations and side effects over repeated runs.
• Test an agent against instructions in tool resultsMeasure how often instructions hidden in tool results take over an agent, with and without defences, next to how often the agent still completes its task.
• Workflow or agent? Automate an open-data processAutomate a public service process from open data twice, as a fixed workflow and as an agent, and compare errors, cost and time on the same tasks.

Recap

• A workflow fixes the steps in code. An agent is a model that chooses its next step in a loop, with tools that the application runs. A router lets the model pick one path, and code does the rest.
• Orchestration is the arranging of the steps: which run, in what order, and who decides. For a support ticket, there are five decisions.
• On T-90103, all three designs proposed the same resolution. The fixed workflow made four decisions in code and used 352 input tokens; the agent made all five with the model and used 4,618.
• More decisions by the model means more places where a new wording or a guess changes the result. The agent’s chair on T-90207 showed it.
• The known-good first run: python -m pytest gives 14 passed, and python -m resolver run T-80008 ends with 4580 tokens in, 117 out | US$0.00052. Every design only proposes; nothing is written until Module 4.

Next, you meet five patterns that combine these designs, each with a Larkfield ticket.

Next: Common patterns