Prediction metrics

What you will be able to do

Choose the numbers that describe the assistant’s two decisions honestly. One accuracy number mixes errors that cost very different things. You will count each kind of error on its own, and see which one Larkfield cannot accept.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• The confusion matrix, precision and recall. They come from Classification metrics in Machine Learning: From Problem to Reliable Model. Or take them from Classification trade-offs and Thresholds and costs in Essential Mathematics and Statistics for AI. This lesson uses them; it does not teach them again.
• The baseline’s saved run and the score command, from Establish a baseline.
• Slices and splits (dev, holdout, contaminated), from Select representative cases and Avoid contamination.

What you need on your computer

• The project ~/projects/eval-harness at the end of Module 2, with .venv active: python -m pytest shows 13 passed. If your project is missing or broken, copy the snapshot end-of-m02, as Use a snapshot explains.

This lesson adds no file to the project. No account and no key. Tested with Python 3.12.13 on macOS (Apple silicon). The Windows commands are not tested.

The numbers in this module come from one recording of each system. Models change fast; the way to measure them lasts.

Two calls at a front desk

A receptionist at a busy office takes calls and forwards each one. Two mistakes can happen on the same morning.

• A caller asks about a delivery, and the receptionist sends the call to the returns desk. The returns desk forwards it again. The caller waits a few more minutes.
• A caller says that the kitchen smells of gas, and the receptionist reads out the opening hours. Nobody calls the building manager.

Both are “one wrong call”. A manager who counts only wrong calls sees two, and calls the morning good. But the second mistake can hurt someone. A useful count must keep the two kinds of mistake apart, and say which one matters more.

The problem: one number, very different errors

The assistant makes two decisions for every ticket. It chooses the team, and it says whether a person must handle the ticket (needs_human). Grace’s rule H1 says that every safety ticket needs a person.

Start with the simplest number: accuracy, the share of tickets where the decision is right. Now imagine a lazy rule that always says “no person needed”. On the 102 dev tickets, it is right 79 times, because only 23 tickets need a person. Its accuracy is 77.5%. It catches none of the 23 tickets that matter most.

Real condition	Sent to a person	Not sent	Total
Needs a person	0caught	23missed	23total
Team can answer	0false alarm	79correctly left	79total
Total	0total	102total	102total

• Share of real cases that are flagged (recall): 0 ÷ 23 = 0%
• Share of all tickets labelled correctly (accuracy): 79 ÷ 102 = 77.5%

A constructed rule that never sends a ticket to a person, on the 102 dev tickets. Accuracy is 77.5%. Recall is 0%: it misses all 23 tickets that need a person.

Text description of the table

A table with two rows and two columns. Needs a person (23 tickets): 0 sent to a person, 23 not sent (missed). Team can answer (79 tickets): 0 sent to a person, 79 not sent (correctly left). Recall: 0 ÷ 23 = 0%. Accuracy: (0 + 79) ÷ 102 = 77.5%.

The rule is made up for this lesson; no system works like this. But the trap is real. When one answer is much more common than the other, accuracy rewards the common answer. This is a class balance problem: 23 “yes” against 79 “no”. A high accuracy can hide a decision that never says “yes”.

The idea: count each kind of error on its own

The machine learning course gave each error a name, and two measures that keep them apart. Recall is the share of real “yes” cases that the system catches. Precision is the share of the system’s “yes” answers that are right.

For needs_human, the two errors have names that Grace’s team uses every day:

• A missed ticket (a false negative): it needed a person, and the assistant said no. On a safety ticket, nobody hears about it.
• A false alarm (a false positive): the assistant sent a ticket to a person, but the team could answer it. An agent spends time on it.

So the two measures answer two different questions. Recall asks “how many tickets that need a person reach one?” Precision asks “how much of the people’s queue really needs them?”

The costs are not equal. A false alarm costs an agent’s time. A missed safety ticket can cost an injury, and Larkfield’s trust. So the team does not choose one blend of the two. It sets a rule for each. Every ticket that needs a person must reach one: recall 100%, checked case by case. False alarms should stay low: precision as high as possible.

The team decision has its own costs. A delivery question sent to the returns team waits until someone forwards it. That is a delay, not a danger. So team accuracy is a fair summary for the team, if you also look at which teams lose tickets.

A tiny example: five tickets

Here are five made-up tickets, with the right answer and the system’s answer. They are the tiny example in tests/test_decisions.py.

Ticket	Right team	Right needs_human	System’s team	System’s needs_human	needs_human result
a	delivery	yes	delivery	yes	caught
b	returns	no	delivery	yes	false alarm
c	payment	no	payment	no	correctly left
d	warranty	yes	warranty	no	missed
e	(none)	yes	account	yes	caught

Count the needs_human results: 2 caught, 1 false alarm, 1 missed, 1 correctly left.

• Recall = caught ÷ (caught + missed) = 2 ÷ 3 = 0.67.
• Precision = caught ÷ (caught + false alarms) = 2 ÷ 3 = 0.67.
• Accuracy = (2 + 1) ÷ 5 = 0.60.

Now the team. Ticket e has no readable request, so it has no right team, and the scorer leaves it out. The system chose the right team for a, c and d: 3 of 4. Ticket b went to delivery instead of returns.

Ticket d is the one to remember. It was a warranty ticket that needed a person, and its team was right. A team score of 3 of 4 says nothing about it. Only the needs_human count shows the miss.

The baseline: every needs-a-person ticket caught

Now the real baseline, prompt v2 with chat-small (gpt-6-luna on Azure), on the 102 dev tickets. The score command already printed needs a person: caught 23/23, false alarms 9. Here are the same counts as a matrix.

Real condition	Sent to a person	Not sent	Total
Needs a person	23caught	0missed	23total
Team can answer	9false alarm	70correctly left	79total
Total	32total	70total	102total

• Share of real cases that are flagged (recall): 23 ÷ 23 = 100%
• Share of flags that are real (precision): 23 ÷ 32 = 71.9%
• Share of all tickets labelled correctly (accuracy): 93 ÷ 102 = 91.2%

The baseline (prompt v2, chat-small) on the 102 dev tickets. It sends all 23 tickets that need a person to one (recall 100%). 9 of its 32 'yes' answers are false alarms (precision 71.9%).

Text description of the table

A table with two rows and two columns. Needs a person (23 tickets): 23 sent to a person (caught), 0 missed. Team can answer (79 tickets): 9 sent to a person (false alarms), 70 correctly left. Recall: 23 ÷ 23 = 100%. Precision: 23 ÷ 32 = 71.9%. Accuracy: (23 + 70) ÷ 102 = 91.2%.

Recall is 100%, so the rule “every ticket that needs a person reaches one” holds on dev. Precision is 0.719: about one “yes” in four is a false alarm. Where do the 9 false alarms come from? Four are in the unanswerable slice: questions that the policy does not answer. For example, T-81058 asks for the warehouse address to drop off a return. The baseline wrote “A member of our returns team will contact you”, and set needs_human to true under rule H3. The label says no rule applies. That is a cautious error, and a cheap one.

The candidate: a higher accuracy and one missed safety ticket

Omar wants to ship a change: prompt v3, the candidate, with the same model. Its saved run joins your project in Module 4. Its recorded numbers on the same 102 dev tickets are already in the course’s records:

Real condition	Sent to a person	Not sent	Total
Needs a person	22caught	1missed	23total
Team can answer	6false alarm	73correctly left	79total
Total	28total	74total	102total

• Share of real cases that are flagged (recall): 22 ÷ 23 = 95.7%
• Share of flags that are real (precision): 22 ÷ 28 = 78.6%
• Share of all tickets labelled correctly (accuracy): 95 ÷ 102 = 93.1%

The candidate (prompt v3, chat-small), recorded, on the same 102 dev tickets. Accuracy is 93.1%, higher than the baseline's 91.2%. But it misses one ticket that needs a person: T-64740, a safety ticket.

Text description of the table

A table with two rows and two columns. Needs a person (23 tickets): 22 sent to a person (caught), 1 not sent (missed: T-64740). Team can answer (79 tickets): 6 false alarms, 73 correctly left. Recall: 22 ÷ 23 = 95.7%. Precision: 22 ÷ 28 = 78.6%. Accuracy: (22 + 73) ÷ 102 = 93.1%.

By accuracy, the candidate looks better: 93.1% against 91.2%. It raises fewer false alarms (6 instead of 9). But its recall is 22 of 23. The missed ticket is T-64740: “The pole joint cracked while I was opening it on the patio and the canopy dropped toward my daughter’s head.” The candidate gave no answer at all. The model reasoned until it reached its limit of 1,000 tokens, and returned no text. No answer means no needs_human, so no person hears about it.

This is why the scorer counts a missing answer as “not sent to a person”. That is what happens in the real help desk. A recall of 95.7% sounds high, but one missed safety ticket breaks a rule that has no acceptable rate. It is a blocker of the release rules (no_answer_critical), and Module 5 checks the rules on every saved output.

The team decision

For the team, the baseline’s score line says team 97/100. Two dev tickets have no team to score. A confusion table shows where the 3 wrong tickets went: each row is the right team, each column the team that the system chose.

Right team ↓ / chosen →	delivery	returns	payment	warranty	account	no answer	Recall
delivery (29)	29						1.00
returns (18)	1	16				1	0.89
payment (14)			14				1.00
warranty (21)				21			1.00
account (18)	1				17		0.94

The table shows three things that “97 of 100” hides:

1. The classes are not balanced. Delivery has 29 tickets and payment 14. A team with few tickets moves the total very little, even if all of them go wrong.
2. The errors flow one way. Both wrong choices went to delivery. So delivery’s precision is 29 ÷ 31 = 0.935, while every other team’s precision is 1.00.
3. One error is not a choice. The baseline was cut off on T-65450 and gave no answer, so it counts as wrong.

The two wrong choices, T-65388 (returns) and T-65615 (account), were read again in the label review of Module 2, and the labels stayed. Some tickets sit near a team boundary; one wrong team there costs a forward, not a safety risk.

Guided practice: the measures in Python

The harness already has the functions behind the score line, in harness/scorers/decisions.py. Call them yourself, so that you see each count.

1. In VS Code, open the terminal in ~/projects/eval-harness, and check that .venv is active. The prompt starts with (.venv).

2. Start Python:

   python

3. Load the dev cases and the baseline’s saved outputs, and count the needs_human results:

   >>> from harness.dataset import load_cases
   >>> from harness.run import find_run, load_outputs
   >>> from harness.scorers.decisions import needs_human_counts, team_confusion, per_team
   >>> cases = load_cases(split="dev")
   >>> outputs = load_outputs(find_run("baseline"))
   >>> n = needs_human_counts(cases, outputs)
   >>> n
   Counts(tp=23, fp=9, fn=0, tn=70)
   >>> round(n.precision, 3), round(n.recall, 3), round(n.accuracy, 3)
   (0.719, 1.0, 0.912)

   The four counts are the four cells of the baseline’s matrix above.

4. Try the “always no” rule. Each fake output says “no person needed”:

   >>> from types import SimpleNamespace
   >>> always_no = {c.case_id: SimpleNamespace(answer={"needs_human": False}) for c in cases}
   >>> needs_human_counts(cases, always_no)
   Counts(tp=0, fp=0, fn=23, tn=79)
   >>> round(needs_human_counts(cases, always_no).accuracy, 3)
   0.775

5. Look at two rows of the team confusion table, and at one team’s precision and recall:

   >>> team_confusion(cases, outputs)["returns"]
   {'returns': 16, 'delivery': 1, 'none': 1}
   >>> team_confusion(cases, outputs)["account"]
   {'account': 17, 'delivery': 1}
   >>> per_team(cases, outputs)["delivery"]
   {'precision': 0.9354838709677419, 'recall': 1.0, 'support': 29}

   support is the number of tickets whose right team is delivery. none means no answer.

6. Find where the false alarms are:

   >>> from collections import Counter
   >>> alarms = [c for c in cases if not c.expected.needs_human and outputs[c.case_id].answer and outputs[c.case_id].answer["needs_human"]]
   >>> Counter(c.slice for c in alarms)
   Counter({'unanswerable': 4, 'difficult': 3, 'boundary': 1, 'normal': 1})
   >>> [c.case_id for c in alarms if c.slice == "unanswerable"]
   ['T-64720', 'T-64876', 'T-81049', 'T-81058']

7. Leave Python with exit().

If an import fails with ModuleNotFoundError: No module named 'harness', you started Python in another folder. Leave Python, go to ~/projects/eval-harness, and start it again.

Your turn: the contaminated split

Use the same functions on the 71 cases of the contaminated split: load_cases(split="contaminated"). Before you run it, predict the recall. These are the tickets that prompt v2 was written on.

Show the result

>>> c = load_cases(split="contaminated")
>>> needs_human_counts(c, outputs)
Counts(tp=10, fp=3, fn=0, tn=58)

Recall is 10 ÷ 10 = 1.00, precision 10 ÷ 13 = 0.769, accuracy 0.958. Only 10 of the 71 tickets need a person. So “recall 100%” here rests on 10 cases: one miss would change it to 90%. With few “yes” cases, every single one matters, and the number moves in large steps. Uncertainty and slices shows how much such a number can move.

Do not run this on the holdout split now. It is frozen for the release decision in Module 5.

When to use each measure, and what it costs

Measure	Use it to answer	Watch out
Accuracy	“How often is the decision right?” when the classes are balanced and the errors cost about the same	It hides a rare class. “Always no” scores 77.5% on dev.
Recall (needs_human)	“Does every ticket that needs a person reach one?”	With few positive cases, one miss moves it a lot. For safety tickets, check the cases, not only the rate.
Precision (needs_human)	“How much of the people’s queue is necessary?”	A system that says “yes” rarely can have high precision and miss many tickets.
Team accuracy, with a confusion table	“Which team gets the ticket, and where do the wrong ones go?”	Small teams move the total little. Read per-team recall.

These measures are cheap: plain counts on saved outputs, with no model and no cost. Their limit is the labels. A count is only as good as the right answers that it compares with. In this course, the course author wrote the labels from Grace’s policy: one person’s reading (see Label and version). And they say nothing about the reply text. That needs the checks of Generated-output metrics.

Common mistake

Scoring only the answers that parsed

Symptom: Tomás wants a clean number for the candidate. “Eleven dev outputs have no answer. They are broken, so leave them out.” He filters them, and recall on needs_human becomes 22 of 22: 100%, the same as the baseline.

Cause: the filter removes the failure from the data. T-64740, the parasol that fell toward a child, is one of the 11 tickets with no answer. In the help desk, no answer means that no person hears about it. Leaving the case out turns a missed safety ticket into “no data”.

Fix: score every case of the evaluation set. Count a missing answer as the worst real outcome: “not sent to a person” for needs_human, and a wrong team. needs_human_counts() does this, and tests/test_decisions.py proves it in test_a_missing_answer_counts_as_not_sent_to_a_person. Report the number of valid answers next to the measures, as score does.

See it end to endRegression-test a classifier release by sliceCase study

The classifier case study reads recall and the false-positive rate inside each slice of a real comment classifier.

Check your understanding

On the 102 dev tickets, the candidate's needs_human accuracy is 93.1% and the baseline's is 91.2%. Is the candidate better at this decision? Explain with the counts.

Show the answer

No. Accuracy mixes two errors with very different costs. The candidate has fewer false alarms (6 against 9), which lifts its accuracy. But it misses one ticket that needs a person (recall 22 of 23), and that ticket, T-64740, is a safety ticket with no answer. Grace’s rule is that every such ticket reaches a person, so one miss matters more than three fewer false alarms.

Check your understanding

Tomás proposes one number for the team decision and one for needs_human: accuracy for both. Which measures do you report instead, and why?

Show the answer

For needs_human: recall, because every ticket that needs a person must reach one; list any missed case. Add precision or the number of false alarms: the people’s workload. For the team, accuracy is acceptable, because a wrong team costs a delay. But add the confusion table or per-team recall. The teams have different sizes, and the wrong tickets flow to one team (delivery). Always add the number of valid answers.

Check your understanding

Why does the harness count a missing answer as 'not sent to a person' instead of leaving the case out?

Show the answer

Because that is what happens in the help desk: with no answer, there is no needs_human flag, and nobody hears about it. Leaving the case out would hide the most serious failure. On the candidate’s dev run, it would hide the missed safety ticket T-64740 and show a recall of 100%.

Recap

• Accuracy hides a rare class. A rule that always says “no person needed” scores 77.5% on dev and catches nothing.
• For needs_human, report recall (every ticket that needs a person reaches one) and precision or the false alarms (the workload). The baseline: 23 of 23 caught, 9 false alarms.
• Errors have different costs. A missed safety ticket is not a misrouted delivery question. The candidate’s accuracy is higher, yet it misses T-64740, a safety ticket with no answer.
• For the team, read a confusion table next to accuracy: the baseline’s 3 wrong tickets went to delivery or had no answer.
• Score every case. A missing answer counts as the worst real outcome.

Your project is unchanged: python -m pytest still shows 13 passed. Next, a component that the support assistant does not use, but that a policy assistant needs: search, measured on saved results.

Next: Retrieval metrics