Human review by design

What you will be able to do

Decide how much the AI may do without a person. Each step away from a person saves review time and lets more mistakes reach customers. Count both costs before you choose.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

• The four number labels (recorded, price, assumption, constructed), from Read the evidence pack.
• How the assistant’s yearly value comes from agents’ time, from Estimate value and return.
• Prompt v2 (today’s version) and prompt v3 (the cheaper-looking change that failed its release gate), from Define done with evidence.

What you need open

• Review time and cost, Part B, in the evidence pack.
• The autonomy ladder template, copied into your document tool.

A new colleague who writes letters

Think of a new colleague in an accounts office. In her first week, she suggests what a letter to a customer should say, and you write it. A month later, she writes the letters and you correct them before they go out. Later still, you only read and sign. One day, she sends routine letters alone.

You did not decide this by how clever she seemed. You decided it by two things. How much does it cost you to check her work? And what does one wrong letter cost? A reminder about a missing receipt can go out alone early. A letter that says a customer owes money gets checked for much longer.

There is a trap at the “only sign” step. If you sign two hundred good letters a day, you stop reading them. Your signature still says “checked”, but nobody checked.

The problem: the cheapest review is the one that misses

Larkfield’s assistant drafts replies to support tickets, and an agent edits each draft and sends it. Here is a real draft from the error examples. Prompt v3 wrote it in a run recorded in the course Evaluating and Testing AI Systems. A customer asked how to update an expired saved card (ticket T-65350):

“I’m sorry for the hassle, especially with your recent order. You can replace the expired card in the saved-card section of your Larkfield account; please don’t send us the full card number. …”

Larkfield has no saved-card section. The reply is warm, fluent and wrong. What happens to it depends on who reads it before the customer does:

• An agent who edits the draft knows the policy: saved cards go to the account team. She deletes the invented step.
• An agent who only approves sees a polite reply among a hundred polite replies. She may click “send”.
• If nobody reads it, the customer looks for a page that does not exist, and writes again.

So the same draft is harmless at one level and reaches a customer at another. The model did not change; the review did.

The idea: the autonomy ladder

Autonomy is how much the AI does before a person sees its work. This course uses four levels, as in the picture. Each level gives the person a different job, so each has a different cost of review and a different share of mistakes that get through.

The autonomy ladder. Larkfield works at level 2 today (thick border). Each step up the ladder removes some of the person's work.

Text description of the diagram

The picture shows four levels from left to right (top to bottom on a phone). At level 1, suggest, the AI offers text and the agent writes the reply. At level 2, draft, the AI writes a draft and the agent edits and sends it. Larkfield works at this level today, so its box has a thick border. At level 3, act with approval, the agent only approves or rejects the AI’s reply. At level 4, act alone, the AI sends the reply and nobody reads it first. A separate box with a dashed border gives a rule that holds at every level: a ticket that needs a person goes to a person.

The ladder gives you two numbers per level. The cost of review is people’s minutes per ticket × tickets × the cost of an hour. The cost of a mistake is the number of bad outputs that reach a customer × what one bad output costs. Moving up the ladder lowers the first number and raises the second.

A tiny example: 1,000 tickets at the draft level

Take 1,000 tickets that the assistant drafts with prompt v2. A judge model scored 199 of its replies against Grace’s policy (recorded). Of these, 173 were usable, 23 needed rewriting, 3 were wrong and 0 were harmful. The minutes are assumptions from Review time and cost; the results are calculated.

Step	Calculation	Result
1. Usable drafts (recorded share)	1,000 × 173 ÷ 199	869.3
2. Drafts that need rewriting or are wrong	1,000 × 26 ÷ 199	130.7
3. Minutes on usable drafts (assumption: 1.5 minutes to review)	869.3 × 1.5	1,303.9
4. Minutes on bad drafts (assumption: 6 minutes to write by hand + 2 minutes lost)	130.7 × 8	1,045.6
5. Cost of review (assumption: US$32 an hour)	2,349.5 minutes = 39.2 hours, × US$32	US$1,253
6. Wrong replies that reach customers (assumption: the agent misses 10%)	1,000 × 3 ÷ 199 × 10%	1.5

So at the draft level, review costs about US$1,253 per 1,000 drafted tickets. About 1.5 wrong replies in 1,000 still reach a customer. Notice what is missing: the model’s price. It is US$0.34 per 1,000 tickets (recorded). Review costs more than 3,600 times as much.

Larkfield’s ladder: the two costs at four levels

The same method gives every level. At “approve”, an agent reads for about 0.6 minutes, and rewrites the bad replies she rejects. Fast approval means less attention, so she is assumed to miss 30% of the bad ones. At “alone”, nobody reads, so every wrong reply goes out.

Calculated cost of review at each level, prompt v2, base assumptions. Each step up the ladder roughly halves it.

Text description of the chart

A column chart with four columns. Suggest: US$2,667 per 1,000 drafted tickets. Draft: US$1,253. Approve: US$613. Alone: US$0. Each step up the ladder roughly halves the cost of review, and at the last step it is zero.

Calculated wrong replies that reach a customer at each level, prompt v2, base assumptions. The cost of review falls; this number rises.

Text description of the chart

A column chart with four columns. Suggest: 0 wrong replies per 1,000 reach a customer, because the agent writes every reply. Draft: 1.5. Approve: 4.5. Alone: 15.1. The two charts move in opposite directions.

Read the two charts together. From draft to approve, Larkfield would save about US$640 and 20 agent hours per 1,000 drafted tickets. It would also send about 3 more wrong replies per 1,000 to customers: three times as many. At “alone”, it would send about 15 per 1,000.

The model cost is the same at every level. Autonomy is a decision about people’s time and customers’ trust, not about the model’s price.

Reviewer fatigue

The 30% at “approve” is the colleague who signs two hundred letters a day. A person who sees good output again and again starts to expect it. Her approvals get faster, and her attention goes. This is reviewer fatigue. It is the reason “act with approval” can be worse than it looks: the person is still there, but the review is not.

Some airport security checks show screeners test images of forbidden items among the real bags. The test keeps the screeners alert, and it measures them. You can do the same. Mix a few known-bad drafts into the queue, such as the T-65350 draft above, and count how many reviewers catch them. The autonomy ladder template asks for three limits: reviews per person per hour, how you test reviewers, and how often the work rotates.

In the low case of the evidence pack, agents miss half the bad replies at “approve”. Then 7.6 wrong replies per 1,000 reach customers instead of 4.5. Nobody has measured the real share yet.

Discuss: approve, or keep editing?

Nadia, Larkfield’s head of customer operations, must decide on Omar’s proposal: agents stop editing drafts and only approve or reject them.

• Omar (product manager): “Most drafts are fine. Approving takes about 0.6 minutes. We save 20 agent hours per 1,000 tickets.”
• Grace (support lead): “My agents catch invented steps because they edit. If they only click, they will send the saved-card reply.”
• Tomás (AI engineer): “Then let it act alone on simple tickets, like account questions, and keep editing for the rest.”
• Kwame (solution architect): “Whatever we choose, a ticket that needs a person must still go to a person.”

Before you read on, answer these:

1. What does “approve” save per 1,000 drafted tickets, and what does it cost in wrong replies?
2. How much would one wrong reply have to cost before the saving is not worth it?
3. Which numbers in your answer are measured, and which are assumed?
4. Is Tomás’s “simple tickets” idea safer?

The course’s answer. Keep editing for now. Approve saves about US$640 per 1,000 drafted tickets and sends 3 more wrong replies per 1,000 to customers. So it pays only if one wrong reply costs Larkfield less than about US$213 (US$640 ÷ 3, calculated). Nobody knows that cost. A customer sent to a page that does not exist writes again, or leaves.

Also, almost every number in the trade-off is an assumption. The 0.6 minutes is Omar’s estimate, and the 30% miss share is a guess about fatigue. And “0 harmful drafts in 199” still allows up to about 15 harmful replies per 1,000 (the rule of three). Tomás’s idea fails on the evidence: the invented steps in the error examples were on plain account questions, T-65350 and T-65479. “Simple” tickets are where fluent mistakes hide.

So Nadia decides: stay at the draft level, and measure. Time agents for two weeks. Mix known-bad drafts into the queue to measure the miss share. Review about 600 drafts: if none is harmful, the rule of three puts the harm rate below 0.5% (3 ÷ 600). Then decide again.

What would change the answer: three measurements. Approval times and miss shares close to the assumptions. An audit with no harmful draft. A cost of a wrong reply well below US$213. Then “approve” could start with one team.

When to move up the ladder, and what it costs

Move up one level only with evidence for both costs at the new level. You need the review minutes and the share of mistakes that get through. Move up for one kind of work at a time, never for everything at once.

Stay low, or move down, when a mistake is expensive or cannot be undone. Do the same when you cannot check the output later, or when the version changes. A ticket that needs a person goes to a person at every level, and a ticket with no answer goes to a person too.

Each level has its own cost. Higher levels need monitoring and a quick way to switch back. Lower levels cost people’s time every day. At “suggest”, the cost is close to writing by hand. It is US$2,667 per 1,000 tickets, against US$3,200 for 6 minutes by hand at US$32 an hour.

Common mistake

A review step that nobody does

Symptom: the plan says “a person approves every reply”. After a month, approvals take two seconds and no agent has rejected a draft for a week.

Cause: reviewer fatigue. The review is on paper only, so the real level is “act alone”.

Fix: measure approval time and the rejection rate per agent. Mix in known-bad drafts and count the catches. If reviewers miss them, move down a level or change the work, for example shorter review sessions and rotation.

Guided practice: fill in the autonomy ladder

1. Open your copy of the autonomy ladder template and Part B of Review time and cost.

2. Write the product: “Larkfield’s support reply assistant, prompt v2”. Write the unit of work: “one drafted ticket”.

3. Write what one harmful reply costs: US$20 / US$50 / US$150 (assumption, from Part A).

4. Fill in the four rows from Part B: minutes, miss share, review cost and wrong replies per 1,000.

   Check: your draft row says 2.35 minutes, 10%, US$1,253 and 1.5.

5. Fill in “Choose and justify” with your level. Use the two costs, and name the assumption you trust least.

6. Fill in the three reviewer fatigue limits.

Check your table against Part B: suggest US$2,667 and 0; approve US$613 and 4.5; alone US$0 and 15.1. If a row differs, check which minutes you used.

Your turn: the same ladder for prompt v3

Change one thing: the assistant uses prompt v3. Its recorded judge scores are 145 usable, 8 to rewrite, 43 wrong and 2 harmful, of 198. Before you look at Part B, predict: is there a level where prompt v3 is acceptable?

Show the answer

Part B gives, per 1,000 drafted tickets (calculated, base assumptions):

Level	Prompt v2: people cost	Prompt v2: wrong replies reaching customers	Prompt v3: people cost	Prompt v3: wrong replies reaching customers
Suggest	US$2,667	0.0	US$2,667	0.0
Draft	US$1,253	1.5	US$1,728	22.7
Approve	US$613	4.5	US$920	68.2
Alone	US$0	15.1	US$0	227.3

No level above “suggest” is acceptable. At the draft level, prompt v3 costs US$475 more in agent time per 1,000 tickets, because agents rewrite more drafts. It still lets 22.7 wrong replies through. Its model cost is only US$0.14 higher. At “suggest”, it saves nothing over prompt v2. A worse version does not only need more review: it makes review dearer and less safe at every level. The autonomy level belongs to a version, so a new version must earn its level again.

See it end to endGo/no-go memo for a support reply assistantCase study

Omar’s proposal includes the move from editing to approving. In the case study, you decide on it with the whole evidence pack, next to prompt v3 and the rollout to all agents.

Knowledge check

Check your understanding

The same model writes the same drafts at the draft level and at the approve level. Why do more wrong replies reach customers at the approve level?

Show the answer

The drafts are the same, but the review is not. An agent who edits reads every sentence and knows the policy, so she misses few wrong replies (assumed 10%). An agent who only approves reads fast, and after many good drafts she expects good drafts: reviewer fatigue. She is assumed to miss 30%. So the wrong replies that reach customers go from 1.5 to 4.5 per 1,000. Both shares are assumptions; test them with known-bad drafts.

Check your understanding

Predict: prompt v2 writes 3 wrong replies in 199. If agents at the approve level miss half of the bad replies, how many wrong replies per 1,000 reach customers? Compare it with the draft level.

Show the answer

3 ÷ 199 is about 1.5% wrong. Half of them get through: 1,000 × 1.5% × 50% = about 7.6 per 1,000. At the draft level (10% missed), it is 1.5. So the approve level would send about five times as many wrong replies. That is the low case in the evidence pack. It shows why the miss share must be measured before anyone moves up the ladder.

Check your understanding

Omar says: 'The model costs US$0.34 per 1,000 tickets, so review is where the money is. Let the assistant act alone and save it all.' What is right in his statement, and what is missing?

Show the answer

He is right that review costs far more than the model. At the draft level, review costs US$1,253 per 1,000 drafted tickets, and the model only US$0.34 per 1,000 tickets. What is missing is the cost of a mistake. Acting alone, prompt v2 would send about 15 wrong replies per 1,000 to customers. Nobody would read a reply before it goes out. “Save it all” counts one cost and ignores the other. Choose a level with both costs, and with evidence for each.

Recap

• The autonomy ladder has four levels: suggest, draft, act with approval, act alone. Larkfield works at “draft”.
• Each level has a cost of review and a cost of a mistake. Moving up lowers the first and raises the second. For prompt v2, per 1,000 drafted tickets (calculated), review costs US$2,667 / US$1,253 / US$613 / US$0. Wrong replies reaching customers: 0 / 1.5 / 4.5 / 15.1.
• The model’s price is the same at every level. People’s time and customers’ trust decide.
• Reviewer fatigue turns approval into a click. Measure it with known-bad drafts, time limits and rotation.
• Move up one level only with evidence for both costs. A new version earns its level again.
• Models change fast: these recorded scores are a snapshot of one model and one prompt. The method lasts.

Next, you design what the agents see and how they report a bad draft, so that their trust matches the assistant’s real quality.

Next: Earn user trust