Skip to content
Français

Decide, review or send to a person

  • Lesson 33
  • 35 min
  • Module 8: Confidence and people in the loop

Where we are

We know which confidence numbers we can trust. Now we use them to decide. In this lesson, we split the messages into three zones (decide, review, send to a person), count the work each zone creates for Grace's team, and keep the review load under what the team can carry.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy
Lesson 33 of 37 · Text Classification and Routing: Send Each Message to the Right PlaceDecide, review or send to a person

What you will be able to do

Split the messages into three zones by confidence: the router decides, a person reviews, or a senior person reads first. Choose the thresholds on the validation messages, under the number of reviews that Grace’s team can carry. Then count the work that each zone makes.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook writes the three zones and the search for the thresholds in a few lines of pandas. It draws the confidence by zone and has the task and the failure case of this lesson, with checks. It installs pandas, scikit-learn and matplotlib and downloads the data. About 25 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 35 minutes

What you need to know

What you need on your computer

Your larkroute project with the files of the last lesson (27 tests pass), or the notebook above.

Tested on macOS (Apple silicon) with Python 3.12.13. The Windows commands were not tested.

Back at the post office. The sorter now has three places for a letter, not one.

  • If the address is clear, the letter goes straight into its tray.
  • If the address is smudged, it goes to the check desk. A clerk reads it closely and puts it in the right tray. This takes a minute.
  • Some letters go to the supervisor’s desk, however clear the address is: a registered letter, or a parcel that leaks. A senior person must look first.

The clerk at the check desk can read only so many letters a day. If the sorter sends half the letters there, the desk overflows, and letters wait longer than if they had been sorted at once. So the manager sets a limit: “at most 1 letter in 10 to the desks”.

The problem: one threshold does two jobs badly

Section titled “The problem: one threshold does two jobs badly”

In Module 1, the explorer had one threshold. Above it, the model decided. Below it, a person reviewed. Two things were missing.

First, some messages need a senior person whatever the confidence. A needs_person message is a safety problem, a legal threat or an escalated complaint. When the model is sure that a message is one, it must not “decide” in the normal sense. It must send it to a person.

Second, the explorer chose its threshold on the test messages. Its cheapest point, 0.57 with US$144.2 per 1,000, was found by looking at the same messages that it was scored on. That number is too good to be true for the next 2,000 messages. You will see by how much at the end of this lesson.

The idea: three zones, chosen under a limit

Section titled “The idea: three zones, chosen under a limit”

Two thresholds, high and low, split the messages into three zones:

Zone Rule What happens
Decide confidence high or more The message goes straight to the model’s route.
Review between low and high An agent reads it and sends it on: US$1.50.
Person under low, or the route is needs_person A senior person reads it first.

The review limit is the largest share of messages that the review zone and the person zone may hold together. It is Grace’s decision, not the model’s. Her team can read about 10% of the messages.

Then the search is simple. Try every pair of thresholds on the validation messages. Drop the pairs that break the limit. Keep the cheapest of the rest. Score that pair once on the test messages.

Here is a practice problem we made on purpose: 8 messages, high = 0.80 and low = 0.40.

Message Model’s route Confidence Zone Label Cost
1 delivery 0.97 decide delivery 0
2 billing 0.91 decide billing 0
3 orders 0.85 decide product 4
4 needs_person 0.88 person needs_person 0
5 returns_refunds 0.66 review returns_refunds 1.50
6 product 0.52 review returns_refunds 1.50
7 account 0.35 person account 2
8 delivery 0.95 decide delivery 0

Count the zones: 4 decided (50%), 2 reviewed (25%), 2 to a person (25%). Review and person together hold 4 of 8, so this pair breaks a 10% limit by far. Message 3 is a wrong decision (US$4). Message 6 would have been wrong too, but the review caught it for US$1.50. Message 7 reached a senior person, who sent it on: US$2, the cost of a team’s message sent to a person. The total is US$9 for 8 messages, or US$1,125 per 1,000. Real data has far fewer unsure messages than this example.

The project’s new command route zones does the search with TF-IDF + logistic regression. It uses decide(), report() and choose() from larkroute/zones.py, which you copied in the last lesson.

  1. In your project folder, with the virtual environment on, copy the command from the snapshot end-of-m08. On macOS and Linux:

    Terminal window
    S=../nextia-learning-labs/C36/snapshots/end-of-m08
    cp $S/larkroute/commands/zones.py larkroute/commands/

    On Windows (PowerShell):

    Terminal window
    $S = "..\nextia-learning-labs\C36\snapshots\end-of-m08"
    Copy-Item $S\larkroute\commands\zones.py larkroute\commands\
  2. Run the tests. You still see 27 passed: the tests of zones.py came with the last lesson.

    Terminal window
    python -m pytest -q
  3. Choose the zones with a review limit of 10%. It takes about 5 seconds.

    Terminal window
    route zones --review-max 0.10
Chosen on validation (review and person zones at most 10%): decide at 0.58 or more, send to a person below 0.00.
validation: decide 90.2%, review 5.4%, person 4.4%; errors in the decide zone 2.1%; cost 305.8 dollars per 1,000
test: decide 89.7%, review 5.6%, person 4.8%; errors in the decide zone 1.6%; cost 148.2 dollars per 1,000
At about 65 messages a day, the review and person zones are about 7 messages a day for Grace's team.
  • Decide at 0.58. The model decides when its confidence is 0.58 or more.
  • Person below 0.00. For this model, the cheapest search result has no low-confidence person zone. Its person zone holds only the messages that it routes to needs_person with a confidence of 0.58 or more. On test, that is 95 messages, and 94 of them really are needs_person.
  • Validation: 90.2% decided, 5.4% reviewed, 4.4% to a person. Together, 9.8%: under the limit.
  • Test: 89.7%, 5.6% and 4.8%. Together, 10.4%: a little over the limit. A limit chosen on 2,000 messages holds only roughly on the next 2,000.
  • Errors in the decide zone: 2.1% and 1.6% of the decided messages are wrong. These are the mistakes that nobody checks.

The chart shows where the zones fall on the test messages.

A histogram of the TF-IDF confidence of the 2,000 test messages in 20 bins of 0.05, coloured by zone, with a dashed line at 0.58. Review, in orange, fills every bar under 0.58 and part of the bar from 0.55 to 0.60: 111 messages (5.6%). Decide, in blue: 1,794 messages (89.7%). Person, in grey, sits on top of the blue bars above 0.6: 95 messages (4.8%) whose route is needs_person. The last bar, 0.95 to 1.0, holds 1,292 messages and is cut at 250.
The confidence of TF-IDF + logistic regression on the 2,000 test messages, by zone (measured). Most messages are very sure; the review zone is the thin tail on the left of 0.58.
Text description of the diagram

A histogram with the confidence from 0 to 1 across, in 20 bins of 0.05, and the messages per bin up, cut at 250. A dashed line marks the decide threshold at 0.58. Every bar left of the line is orange: the review zone, 111 messages in all, from 1 message near 0.15 to 25 messages between 0.50 and 0.55. The bar from 0.55 to 0.60 has 25 orange messages and 13 blue ones. Right of the line, the bars are blue (decided) with a thin grey top (the person zone, 95 messages routed to needs_person). The blue bars grow from 28 messages at 0.60 to 0.65 to 205 at 0.90 to 0.95. The last bar, 0.95 to 1.00, holds 1,292 messages and is cut at the top of the chart. The legend says: decide 1,794 (89.7%), review 111 (5.6%), person 95 (4.8%).

Most messages sit in the last bar. The review zone is a thin tail on the left. This is why a small review budget removes so many mistakes: the unsure messages are few, and many of them are wrong. If the model had decided the 111 test messages of the review zone, only 57.7% of them would have been right.

Validation says US$305.8, test says US$148.2

Section titled “Validation says US$305.8, test says US$148.2”

The same thresholds cost twice as much on validation as on test. Both parts have 2,000 messages, with the same mix of routes and languages. Section 3 of the notebook counts the messages of each part by their cost in dollars:

validation: {1.5: 109, 4.0: 32, 20.0: 2, 40.0: 2, 200.0: 1}
test : {1.5: 111, 2.0: 1, 4.0: 27, 20.0: 1}

Read it as “cost: messages”. The reviews cost about the same in both parts: 109 and 111 at US$1.50. The difference is in the rare, expensive mistakes of the validation part:

Mistake in the decide zone Validation Test
Missed account takeover (US$200) 1: US$200 0
needs_person message sent to a team (US$40) 2: US$80 0
Urgent message in the wrong queue (US$20) 2: US$40 1: US$20
Normal message in the wrong queue (US$4) 32: US$128 27: US$108

One account takeover costs US$200. On 2,000 messages, that alone is US$100 per 1,000. The test part was luckier.

So thresholds chosen on 2,000 messages carry this noise. The cost of a zone is decided by a few rare, expensive mistakes. Report both numbers. Tell Grace “somewhere between about US$150 and US$300 per 1,000 messages on this inbox”, not “US$148.2”. Larkfield today, with a person for everything, costs US$1,500.

A zone is work for people. The history has about 65 messages a day.

A flow from 'About 65 messages a day' to 'TF-IDF + LR gives each message a route and a confidence', which splits into three boxes. Decide (0.58 or more): about 59 a day, straight to a team; about 1 a day is wrong (1.6% of them). Review (under 0.58): about 4 a day; an agent reads each one and sends it on, US$1.50 each. Person (the route is needs_person): about 3 a day; a senior person reads it first. Below: Grace's team reads review plus person, about 7 a day; chosen for at most 10%: 9.8% on validation, 10.4% on test.
One day of messages in the three zones, from the test shares (measured) and the history's average of about 65 messages a day.
Text description of the diagram

A flow diagram. At the top: about 65 messages a day. An arrow leads to the model, which gives each message a route and a confidence. Three arrows lead to three boxes. The blue box, decide, 0.58 or more: about 59 messages a day go straight to a team. About 1 a day of them is wrong, 1.6%. The orange box, review, under 0.58: about 4 a day; an agent reads each one and sends it on, for US$1.50 each. The grey box, person, the route is needs_person: about 3 a day; a senior person reads each one first. Arrows from the review box and the person box lead to a note: Grace’s team reads review and person, about 7 messages a day. The thresholds were chosen for at most 10%: 9.8% on validation and 10.4% on test.

About 7 messages a day for Grace’s team, instead of all 65. And about 1 wrong decision a day that nobody checks. Grace can now ask a clear question: is one wrong queue a day worth 7 reviews a day? The cost table says yes. Her team’s time says it fits.

The explorer below is the one from Module 1, with Grace’s limit added. Its curve shades the thresholds that send more than 10% of the test messages to review. The mark shows the threshold chosen on validation.

Explore

The decide threshold under a 10% review limit

  • Total cost
  • Mistakes
  • Reviews by a person
over the review limit05001,0001,500Dollars per 1,000 messages0.30.40.50.60.70.80.91Decide threshold (top probability)chosen on validationthreshold 0.58: US$148.2
over the review limit05001,0001,500Dollars per 1,000 messages0.40.60.81Decide threshold (top probability)chosen on validationthreshold 0.58: US$148.2
Decided by the model
1,889 (94.5%)
Sent to review
111 (5.6%)
Accuracy of the decided
98.5%
Review cost
US$166.50
Mistake cost
US$130
Total per 1,000 messages
US$148.2
Review limit 10%
kept

At 0.58, the model decides 1,889 of 2,000 messages and sends 111 (5.6%) to review. The reviews cost US$166.50 and the mistakes that remain cost US$130: US$148.2 per 1,000 messages. Within the review limit of 10%, the lowest cost is US$144.2 per 1,000, at 0.57.

TF-IDF + logistic regression on the 2,000 test messages (measured). The shaded thresholds send more than 10% of the messages to review. The costs are the course's assumptions.
Text description of the explorer

The explorer has one slider, the decide threshold, from 0.30 to 1.00. A line chart shows the total cost per 1,000 test messages, the cost of the mistakes and the cost of the reviews, against the threshold. The thresholds from about 0.71 to 1.00 are shaded: there, more than 10% of the messages go to review. A mark at 0.58 says “chosen on validation”. Below the chart, tiles give the exact values at the chosen threshold. At 0.58: 1,889 messages decided and 111 sent to review. The review cost is US$166.5, the mistake cost US$130, and the total US$148.2 per 1,000. The review limit of 10% is kept. A status line names the cheapest threshold within the limit on these test messages: US$144.2 at 0.57.

  • At 0.58, the total is US$148.2, the same as route zones printed for test. 111 messages (5.6%) go to review.
  • The cheapest point on the test messages is 0.57, at US$144.2: one step away. Here, the validation choice was close to the best possible test choice. That is luck as much as skill: the validation curve has its own bumps.
  • The shading starts near 0.71. From there, more than 200 messages (10%) go to review. The total rises again: US$181.0 at 0.70 and US$227.5 at 0.80.

One difference with route zones. The explorer counts only the review zone against the limit. The messages that the model routes to needs_person stay in its “decided” count. route zones counts review and person together, which is what Grace’s team really reads.

The search found no low-confidence person zone for TF-IDF. The course’s runs made the same search for the fine-tuned encoder of Module 4 (measured). The project’s command does not run the encoder. The search found a real person zone:

Decide Review Person Test cost per 1,000
TF-IDF + LR 0.58 or more under 0.58 route needs_person US$148.2
Fine-tuned encoder 0.78 or more 0.40 to 0.78 under 0.40, or route needs_person US$74.5

On test, the encoder decides 92.6% of the messages, reviews 2.3% and sends 5.0% to a person. The search chose a person zone because the cost table makes it cheap here. A senior person’s look costs US$2 for a team’s message. For a needs_person message, it costs nothing, and a review costs US$1.50. A better model gives better zones: half the cost of TF-IDF, with fewer reviews.

Change one thing: Grace hires a second agent

Section titled “Change one thing: Grace hires a second agent”

Predict first: if the team can read 20% of the messages, will the test cost go down?

Terminal window
route zones --review-max 0.20
Chosen on validation (review and person zones at most 20%): decide at 0.74 or more, send to a person below 0.00.
validation: decide 83.8%, review 12.0%, person 4.2%; errors in the decide zone 0.7%; cost 220.0 dollars per 1,000
test: decide 83.9%, review 11.7%, person 4.4%; errors in the decide zone 0.7%; cost 197.5 dollars per 1,000
At about 65 messages a day, the review and person zones are about 11 messages a day for Grace's team.

On validation, the larger limit is cheaper: US$220.0 against US$305.8. On test, it is more expensive: US$197.5 against US$148.2. The two parts disagree again, for the same reason as before. The extra reviews remove the validation part’s expensive mistakes, which the test part did not have.

With --review-max 0.05, the command decides at 0.33: US$444.0 on validation and US$290.5 on test, about 4 messages a day. A smaller team pays in mistakes.

Section 5 of the notebook asks you to make the same choice in pandas. Write the call to choose() with review_max=0.20 on the validation messages, and put the decide threshold in high_20.

Show the answer
high_20, low_20, on_val_20 = choose(validation, p_val, review_max=0.20)
print(high_20, low_20, on_val_20)
0.74 0.0 {'decide': np.float64(0.838), 'review': np.float64(0.12), 'person': np.float64(0.042), 'errors_in_decide': np.float64(0.0066), 'cost_per_1000': 220.0}

The check cell prints Check passed: high_20 is 0.74. The notebook’s zones(), report() and choose() follow the same rules as the project’s larkroute/zones.py, so they agree with route zones.

Common mistake

Thresholds chosen on the test messages

Symptom: Tomás says: “Why choose on validation? The test messages are what we report. Let’s choose there.” Section 6 of the notebook does it. With the same 10% limit, it decides at 0.57, and the test cost is US$144.2, better than US$148.2.

Cause: the search tried about 600 pairs of thresholds on the same 2,000 test messages, and kept the best. Some of its gain is the noise of those messages. On the validation messages, which this choice did not see, the same thresholds cost US$311.8.

Fix: choose on validation and score the test messages once, at the end, as route zones does. Then report the test cost with its noise. Here, the two choices are close (0.57 and 0.58). With a smaller test set or more thresholds to tune, the gap can be large.

Consequence in AI work: a cost that was tuned on the test set promises a saving that the next month does not keep. The team trusts the zones less, and starts to read every message again.

Check your understanding

What does each of the three zones do, and why does a message that the model routes to needs_person go to the person zone even when the model is sure?

Show the answer

Decide: the confidence is at or above the decide threshold, and the message goes straight to the model’s route. Review: the confidence is between the two thresholds, and an agent reads the message and sends it on (US$1.50). Person: the confidence is under the low threshold, and a senior person reads it first. A needs_person message is a safety problem, a legal threat or an escalated complaint. That route means “a senior person reads it first”, so a sure needs_person decision is a decision to send it to a person. For TF-IDF on test, that is 95 messages (4.8%).

Check your understanding

route zones --review-max 0.10 gives US$305.8 per 1,000 on validation and US$148.2 on test, for the same thresholds. What do you tell Grace, and why?

Show the answer

Tell her that the zones cost about US$150 to US$300 per 1,000 messages on this inbox. A person on every message costs US$1,500. The two numbers differ because a few rare, expensive mistakes decide the cost. The validation decide zone held one missed account takeover: US$200, so US$100 per 1,000 on its own. It also held two needs_person messages sent to teams (US$80) and two urgent messages in the wrong queue (US$40). The test part had no takeover and no needs_person mistake. Thresholds chosen on 2,000 messages carry this noise, so report both and do not promise the lower one.

Check your understanding

Predict: Grace's team can now read 20% of the messages. route zones chooses 0.74 instead of 0.58. Why can the test cost go up, from US$148.2 to US$197.5, when more messages are reviewed?

Show the answer

The command chooses on validation. There, the extra reviews remove expensive mistakes, and the cost falls from US$305.8 to US$220.0. On test, those expensive mistakes were not in the decide zone at 0.58, so the extra reviews mostly add US$1.50 each. More reviews help only where they catch expensive mistakes, and two samples of 2,000 messages can disagree about where those are.

Recap

  • Three zones: decide (confidence at or above high), review (between low and high, US$1.50), person (under low, or the route is needs_person).
  • The review limit is the team’s capacity: review and person together. Choose the thresholds on validation, under the limit, at the lowest cost. Score them once on test.
  • For TF-IDF + logistic regression with a 10% limit: decide at 0.58. On test, decide 89.7%, review 5.6%, person 4.8%, 1.6% wrong in the decide zone, US$148.2 per 1,000 (validation US$305.8). About 7 messages a day for Grace’s team.
  • Thresholds chosen on 2,000 messages carry noise: one missed account takeover is US$100 per 1,000. Report both numbers.
  • The fine-tuned encoder has a real person zone: it decides at 0.78 or more and sends messages under 0.40 to a person. Test cost: US$74.5 per 1,000.
  • Choosing on the test messages looks better (US$144.2) and is not honest. The validation messages say US$311.8 for that choice.

People now read the unsure messages, and some of them move a message to another queue. Next, you turn those moves into new labels, and you check them first, because a quick move is not always a right one.

Next: Learn from corrections