Decide, review or send to a person
- Lesson 33
- 35 min
- Module 8: Confidence and people in the loop
Where we are
We know which confidence numbers we can trust. Now we use them to decide. In this lesson, we split the messages into three zones (decide, review, send to a person), count the work each zone creates for Grace's team, and keep the review load under what the team can carry.
Lesson 33 of 37 · Text Classification and Routing: Send Each Message to the Right PlaceDecide, review or send to a person
1. The routing problem and its cost
2. Labels you can trust
3. Baselines: rules and classical models
4. Embeddings and small fine-tuned models
5. LLMs as classifiers
6. Decision models, cascades and distillation
7. Harder label shapes
8. Confidence and people in the loop
9. Run it in production
What you will be able to do
Split the messages into three zones by confidence: the router decides, a person reviews, or a senior person reads first. Choose the thresholds on the validation messages, under the number of reviews that Grace’s team can carry. Then count the work that each zone makes.
NotebookDo this lesson in a notebook
The notebook writes the three zones and the search for the thresholds in a few lines of pandas. It draws the confidence by zone and has the task and the failure case of this lesson, with checks. It installs pandas, scikit-learn and matplotlib and downloads the data. About 25 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- The cost table, the cost per 1,000 messages and the explorer with one threshold, from The cost of each mistake.
- Calibration and
route calibration, from Is the confidence true?. - Why you choose on the validation messages and test once, from Train, validation and test sets in the machine learning course.
What you need on your computer
Your larkroute project with the files of the last lesson (27 tests pass), or the notebook above.
Tested on macOS (Apple silicon) with Python 3.12.13. The Windows commands were not tested.
Three trays on the sorting table
Section titled “Three trays on the sorting table”Back at the post office. The sorter now has three places for a letter, not one.
- If the address is clear, the letter goes straight into its tray.
- If the address is smudged, it goes to the check desk. A clerk reads it closely and puts it in the right tray. This takes a minute.
- Some letters go to the supervisor’s desk, however clear the address is: a registered letter, or a parcel that leaks. A senior person must look first.
The clerk at the check desk can read only so many letters a day. If the sorter sends half the letters there, the desk overflows, and letters wait longer than if they had been sorted at once. So the manager sets a limit: “at most 1 letter in 10 to the desks”.
The problem: one threshold does two jobs badly
Section titled “The problem: one threshold does two jobs badly”In Module 1, the explorer had one threshold. Above it, the model decided. Below it, a person reviewed. Two things were missing.
First, some messages need a senior person whatever the confidence. A needs_person message is a safety problem, a legal threat or an escalated complaint. When the model is sure that a message is one, it must not “decide” in the normal sense. It must send it to a person.
Second, the explorer chose its threshold on the test messages. Its cheapest point, 0.57 with US$144.2 per 1,000, was found by looking at the same messages that it was scored on. That number is too good to be true for the next 2,000 messages. You will see by how much at the end of this lesson.
The idea: three zones, chosen under a limit
Section titled “The idea: three zones, chosen under a limit”Two thresholds, high and low, split the messages into three zones:
| Zone | Rule | What happens |
|---|---|---|
| Decide | confidence high or more |
The message goes straight to the model’s route. |
| Review | between low and high |
An agent reads it and sends it on: US$1.50. |
| Person | under low, or the route is needs_person |
A senior person reads it first. |
The review limit is the largest share of messages that the review zone and the person zone may hold together. It is Grace’s decision, not the model’s. Her team can read about 10% of the messages.
Then the search is simple. Try every pair of thresholds on the validation messages. Drop the pairs that break the limit. Keep the cheapest of the rest. Score that pair once on the test messages.
A tiny worked example
Section titled “A tiny worked example”Here is a practice problem we made on purpose: 8 messages, high = 0.80 and low = 0.40.
| Message | Model’s route | Confidence | Zone | Label | Cost |
|---|---|---|---|---|---|
| 1 | delivery | 0.97 | decide | delivery | 0 |
| 2 | billing | 0.91 | decide | billing | 0 |
| 3 | orders | 0.85 | decide | product | 4 |
| 4 | needs_person | 0.88 | person | needs_person | 0 |
| 5 | returns_refunds | 0.66 | review | returns_refunds | 1.50 |
| 6 | product | 0.52 | review | returns_refunds | 1.50 |
| 7 | account | 0.35 | person | account | 2 |
| 8 | delivery | 0.95 | decide | delivery | 0 |
Count the zones: 4 decided (50%), 2 reviewed (25%), 2 to a person (25%). Review and person together hold 4 of 8, so this pair breaks a 10% limit by far. Message 3 is a wrong decision (US$4). Message 6 would have been wrong too, but the review caught it for US$1.50. Message 7 reached a senior person, who sent it on: US$2, the cost of a team’s message sent to a person. The total is US$9 for 8 messages, or US$1,125 per 1,000. Real data has far fewer unsure messages than this example.
Choose the zones for Larkfield
Section titled “Choose the zones for Larkfield”The project’s new command route zones does the search with TF-IDF + logistic regression. It uses decide(), report() and choose() from larkroute/zones.py, which you copied in the last lesson.
-
In your project folder, with the virtual environment on, copy the command from the snapshot
end-of-m08. On macOS and Linux:Terminal window S=../nextia-learning-labs/C36/snapshots/end-of-m08cp $S/larkroute/commands/zones.py larkroute/commands/On Windows (PowerShell):
Terminal window $S = "..\nextia-learning-labs\C36\snapshots\end-of-m08"Copy-Item $S\larkroute\commands\zones.py larkroute\commands\ -
Run the tests. You still see
27 passed: the tests ofzones.pycame with the last lesson.Terminal window python -m pytest -q -
Choose the zones with a review limit of 10%. It takes about 5 seconds.
Terminal window route zones --review-max 0.10
Chosen on validation (review and person zones at most 10%): decide at 0.58 or more, send to a person below 0.00.validation: decide 90.2%, review 5.4%, person 4.4%; errors in the decide zone 2.1%; cost 305.8 dollars per 1,000 test: decide 89.7%, review 5.6%, person 4.8%; errors in the decide zone 1.6%; cost 148.2 dollars per 1,000At about 65 messages a day, the review and person zones are about 7 messages a day for Grace's team.Read the output
Section titled “Read the output”- Decide at 0.58. The model decides when its confidence is 0.58 or more.
- Person below 0.00. For this model, the cheapest search result has no low-confidence person zone. Its person zone holds only the messages that it routes to
needs_personwith a confidence of 0.58 or more. On test, that is 95 messages, and 94 of them really areneeds_person. - Validation: 90.2% decided, 5.4% reviewed, 4.4% to a person. Together, 9.8%: under the limit.
- Test: 89.7%, 5.6% and 4.8%. Together, 10.4%: a little over the limit. A limit chosen on 2,000 messages holds only roughly on the next 2,000.
- Errors in the decide zone: 2.1% and 1.6% of the decided messages are wrong. These are the mistakes that nobody checks.
The chart shows where the zones fall on the test messages.
Text description of the diagram
A histogram with the confidence from 0 to 1 across, in 20 bins of 0.05, and the messages per bin up, cut at 250. A dashed line marks the decide threshold at 0.58. Every bar left of the line is orange: the review zone, 111 messages in all, from 1 message near 0.15 to 25 messages between 0.50 and 0.55. The bar from 0.55 to 0.60 has 25 orange messages and 13 blue ones. Right of the line, the bars are blue (decided) with a thin grey top (the person zone, 95 messages routed to needs_person). The blue bars grow from 28 messages at 0.60 to 0.65 to 205 at 0.90 to 0.95. The last bar, 0.95 to 1.00, holds 1,292 messages and is cut at the top of the chart. The legend says: decide 1,794 (89.7%), review 111 (5.6%), person 95 (4.8%).
Most messages sit in the last bar. The review zone is a thin tail on the left. This is why a small review budget removes so many mistakes: the unsure messages are few, and many of them are wrong. If the model had decided the 111 test messages of the review zone, only 57.7% of them would have been right.
Validation says US$305.8, test says US$148.2
Section titled “Validation says US$305.8, test says US$148.2”The same thresholds cost twice as much on validation as on test. Both parts have 2,000 messages, with the same mix of routes and languages. Section 3 of the notebook counts the messages of each part by their cost in dollars:
validation: {1.5: 109, 4.0: 32, 20.0: 2, 40.0: 2, 200.0: 1}test : {1.5: 111, 2.0: 1, 4.0: 27, 20.0: 1}Read it as “cost: messages”. The reviews cost about the same in both parts: 109 and 111 at US$1.50. The difference is in the rare, expensive mistakes of the validation part:
| Mistake in the decide zone | Validation | Test |
|---|---|---|
| Missed account takeover (US$200) | 1: US$200 | 0 |
needs_person message sent to a team (US$40) |
2: US$80 | 0 |
| Urgent message in the wrong queue (US$20) | 2: US$40 | 1: US$20 |
| Normal message in the wrong queue (US$4) | 32: US$128 | 27: US$108 |
One account takeover costs US$200. On 2,000 messages, that alone is US$100 per 1,000. The test part was luckier.
So thresholds chosen on 2,000 messages carry this noise. The cost of a zone is decided by a few rare, expensive mistakes. Report both numbers. Tell Grace “somewhere between about US$150 and US$300 per 1,000 messages on this inbox”, not “US$148.2”. Larkfield today, with a person for everything, costs US$1,500.
The work for Grace’s team
Section titled “The work for Grace’s team”A zone is work for people. The history has about 65 messages a day.
Text description of the diagram
A flow diagram. At the top: about 65 messages a day. An arrow leads to the model, which gives each message a route and a confidence. Three arrows lead to three boxes. The blue box, decide, 0.58 or more: about 59 messages a day go straight to a team. About 1 a day of them is wrong, 1.6%. The orange box, review, under 0.58: about 4 a day; an agent reads each one and sends it on, for US$1.50 each. The grey box, person, the route is needs_person: about 3 a day; a senior person reads each one first. Arrows from the review box and the person box lead to a note: Grace’s team reads review and person, about 7 messages a day. The thresholds were chosen for at most 10%: 9.8% on validation and 10.4% on test.
About 7 messages a day for Grace’s team, instead of all 65. And about 1 wrong decision a day that nobody checks. Grace can now ask a clear question: is one wrong queue a day worth 7 reviews a day? The cost table says yes. Her team’s time says it fits.
Explore the threshold under the limit
Section titled “Explore the threshold under the limit”The explorer below is the one from Module 1, with Grace’s limit added. Its curve shades the thresholds that send more than 10% of the test messages to review. The mark shows the threshold chosen on validation.
Explore
The decide threshold under a 10% review limit
Predict first: The threshold starts at 0.58, chosen on validation. Before you move it: is the cheapest threshold on the test messages near 0.58 or far from it? And where does the review zone pass 10%?
- Total cost
- Mistakes
- Reviews by a person
- Decided by the model
- 1,889 (94.5%)
- Sent to review
- 111 (5.6%)
- Accuracy of the decided
- 98.5%
- Review cost
- US$166.50
- Mistake cost
- US$130
- Total per 1,000 messages
- US$148.2
- Review limit 10%
- kept
At 0.58, the model decides 1,889 of 2,000 messages and sends 111 (5.6%) to review. The reviews cost US$166.50 and the mistakes that remain cost US$130: US$148.2 per 1,000 messages. Within the review limit of 10%, the lowest cost is US$144.2 per 1,000, at 0.57.
Text description of the explorer
The explorer has one slider, the decide threshold, from 0.30 to 1.00. A line chart shows the total cost per 1,000 test messages, the cost of the mistakes and the cost of the reviews, against the threshold. The thresholds from about 0.71 to 1.00 are shaded: there, more than 10% of the messages go to review. A mark at 0.58 says “chosen on validation”. Below the chart, tiles give the exact values at the chosen threshold. At 0.58: 1,889 messages decided and 111 sent to review. The review cost is US$166.5, the mistake cost US$130, and the total US$148.2 per 1,000. The review limit of 10% is kept. A status line names the cheapest threshold within the limit on these test messages: US$144.2 at 0.57.
What the explorer shows
Section titled “What the explorer shows”- At 0.58, the total is US$148.2, the same as
route zonesprinted for test. 111 messages (5.6%) go to review. - The cheapest point on the test messages is 0.57, at US$144.2: one step away. Here, the validation choice was close to the best possible test choice. That is luck as much as skill: the validation curve has its own bumps.
- The shading starts near 0.71. From there, more than 200 messages (10%) go to review. The total rises again: US$181.0 at 0.70 and US$227.5 at 0.80.
One difference with route zones. The explorer counts only the review zone against the limit. The messages that the model routes to needs_person stay in its “decided” count. route zones counts review and person together, which is what Grace’s team really reads.
A model with a real person zone
Section titled “A model with a real person zone”The search found no low-confidence person zone for TF-IDF. The course’s runs made the same search for the fine-tuned encoder of Module 4 (measured). The project’s command does not run the encoder. The search found a real person zone:
| Decide | Review | Person | Test cost per 1,000 | |
|---|---|---|---|---|
| TF-IDF + LR | 0.58 or more | under 0.58 | route needs_person |
US$148.2 |
| Fine-tuned encoder | 0.78 or more | 0.40 to 0.78 | under 0.40, or route needs_person |
US$74.5 |
On test, the encoder decides 92.6% of the messages, reviews 2.3% and sends 5.0% to a person. The search chose a person zone because the cost table makes it cheap here. A senior person’s look costs US$2 for a team’s message. For a needs_person message, it costs nothing, and a review costs US$1.50. A better model gives better zones: half the cost of TF-IDF, with fewer reviews.
Change one thing: Grace hires a second agent
Section titled “Change one thing: Grace hires a second agent”Predict first: if the team can read 20% of the messages, will the test cost go down?
route zones --review-max 0.20Chosen on validation (review and person zones at most 20%): decide at 0.74 or more, send to a person below 0.00.validation: decide 83.8%, review 12.0%, person 4.2%; errors in the decide zone 0.7%; cost 220.0 dollars per 1,000 test: decide 83.9%, review 11.7%, person 4.4%; errors in the decide zone 0.7%; cost 197.5 dollars per 1,000At about 65 messages a day, the review and person zones are about 11 messages a day for Grace's team.On validation, the larger limit is cheaper: US$220.0 against US$305.8. On test, it is more expensive: US$197.5 against US$148.2. The two parts disagree again, for the same reason as before. The extra reviews remove the validation part’s expensive mistakes, which the test part did not have.
With --review-max 0.05, the command decides at 0.33: US$444.0 on validation and US$290.5 on test, about 4 messages a day. A smaller team pays in mistakes.
Your turn
Section titled “Your turn”Section 5 of the notebook asks you to make the same choice in pandas. Write the call to choose() with review_max=0.20 on the validation messages, and put the decide threshold in high_20.
Show the answer
high_20, low_20, on_val_20 = choose(validation, p_val, review_max=0.20)print(high_20, low_20, on_val_20)0.74 0.0 {'decide': np.float64(0.838), 'review': np.float64(0.12), 'person': np.float64(0.042), 'errors_in_decide': np.float64(0.0066), 'cost_per_1000': 220.0}The check cell prints Check passed: high_20 is 0.74. The notebook’s zones(), report() and choose() follow the same rules as the project’s larkroute/zones.py, so they agree with route zones.
Common mistake
Thresholds chosen on the test messages
Symptom: Tomás says: “Why choose on validation? The test messages are what we report. Let’s choose there.” Section 6 of the notebook does it. With the same 10% limit, it decides at 0.57, and the test cost is US$144.2, better than US$148.2.
Cause: the search tried about 600 pairs of thresholds on the same 2,000 test messages, and kept the best. Some of its gain is the noise of those messages. On the validation messages, which this choice did not see, the same thresholds cost US$311.8.
Fix: choose on validation and score the test messages once, at the end, as route zones does. Then report the test cost with its noise. Here, the two choices are close (0.57 and 0.58). With a smaller test set or more thresholds to tune, the gap can be large.
Consequence in AI work: a cost that was tuned on the test set promises a saving that the next month does not keep. The team trusts the zones less, and starts to read every message again.
Knowledge check
Section titled “Knowledge check”Check your understanding
What does each of the three zones do, and why does a message that the model routes to needs_person go to the person zone even when the model is sure?
Show the answer
Decide: the confidence is at or above the decide threshold, and the message goes straight to the model’s route. Review: the confidence is between the two thresholds, and an agent reads the message and sends it on (US$1.50). Person: the confidence is under the low threshold, and a senior person reads it first. A needs_person message is a safety problem, a legal threat or an escalated complaint. That route means “a senior person reads it first”, so a sure needs_person decision is a decision to send it to a person. For TF-IDF on test, that is 95 messages (4.8%).
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
route zones --review-max 0.10 gives US$305.8 per 1,000 on validation and US$148.2 on test, for the same thresholds. What do you tell Grace, and why?
Show the answer
Tell her that the zones cost about US$150 to US$300 per 1,000 messages on this inbox. A person on every message costs US$1,500. The two numbers differ because a few rare, expensive mistakes decide the cost. The validation decide zone held one missed account takeover: US$200, so US$100 per 1,000 on its own. It also held two needs_person messages sent to teams (US$80) and two urgent messages in the wrong queue (US$40). The test part had no takeover and no needs_person mistake. Thresholds chosen on 2,000 messages carry this noise, so report both and do not promise the lower one.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: Grace's team can now read 20% of the messages. route zones chooses 0.74 instead of 0.58. Why can the test cost go up, from US$148.2 to US$197.5, when more messages are reviewed?
Show the answer
The command chooses on validation. There, the extra reviews remove expensive mistakes, and the cost falls from US$305.8 to US$220.0. On test, those expensive mistakes were not in the decide zone at 0.58, so the extra reviews mostly add US$1.50 each. More reviews help only where they catch expensive mistakes, and two samples of 2,000 messages can disagree about where those are.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Three zones: decide (confidence at or above
high), review (betweenlowandhigh, US$1.50), person (underlow, or the route isneeds_person). - The review limit is the team’s capacity: review and person together. Choose the thresholds on validation, under the limit, at the lowest cost. Score them once on test.
- For TF-IDF + logistic regression with a 10% limit: decide at 0.58. On test, decide 89.7%, review 5.6%, person 4.8%, 1.6% wrong in the decide zone, US$148.2 per 1,000 (validation US$305.8). About 7 messages a day for Grace’s team.
- Thresholds chosen on 2,000 messages carry noise: one missed account takeover is US$100 per 1,000. Report both numbers.
- The fine-tuned encoder has a real person zone: it decides at 0.78 or more and sends messages under 0.40 to a person. Test cost: US$74.5 per 1,000.
- Choosing on the test messages looks better (US$144.2) and is not honest. The validation messages say US$311.8 for that choice.
People now read the unsure messages, and some of them move a message to another queue. Next, you turn those moves into new labels, and you check them first, because a quick move is not always a right one.
Next: Learn from corrections