The return that cost twice
- Lesson 1
- 25 min
- Module 1: Seeing is a business problem
Where we are
In the machine learning course, every example was a row in a table. Larkfield's returns warehouse has photos instead: 600 returned items a day, checked by eye. In this lesson, we start with a box of eggs at the shop. Then we look at one pack of candles that cost Larkfield twice, and at what each kind of mistake costs.
Lesson 1 of 30 · Vision AI: See, Count and InspectThe return that cost twice
1. Seeing is a business problem
2. Baselines and pretrained classifiers
3. Embeddings, CLIP and few labels
4. Detect and count
6. Inspect for defects
7. Match, search and describe
8. Run vision in production
What you will be able to do
Price the two mistakes that a returns check can make. Then turn the wish “use AI on the photos” into a problem with an output, a camera, a cost and a fallback.
Before you startWhat to know and to have ready
What you need to know
- What a model, a label and a prediction are. If you need a refresher, read Supervised learning in Machine Learning: From Problem to Reliable Model.
- It helps, but it is not required, to know precision and recall from Classification metrics. This lesson explains the one idea that it needs.
What you need on your computer
Nothing. This lesson has no code. You open the course’s images in a notebook in Meet the Larkfield images, the last lesson of this module.
A box of eggs at the shop
Section titled “A box of eggs at the shop”You buy a box of eggs. Before you pay, you open the box and look at each egg for a second. Most days, all the eggs are fine. Sometimes one has a crack.
You can make two mistakes. You can miss a crack, take the box home, and find a broken egg in the fridge. Or you can put back a good box because a mark on one shell looked like a crack. The first mistake costs you an egg and some cleaning. The second costs you nothing much: you take another box.
Now imagine that you check eggs all day, for a living. You check 600 boxes a day. A quick look misses small cracks. A careful look finds them, but it takes a full minute, and you have time for only 100 careful looks. Which boxes get the careful look? That question is the whole problem of this lesson, and of most of this course.
The pack that came back
Section titled “The pack that came back”Larkfield is the online shop for home and garden products that you may know from other courses. In this course, Larkfield sells products from photos, and it checks returned items at its returns warehouse. Hamid leads the warehouse team. Every day, about 600 returned items arrive.
For each item, the team makes one decision. A good item goes back on sale as new. A damaged item goes to the outlet, Larkfield’s shop for second-hand items, where it sells for much less. Today the team decides by eye, with a quick look while they repack the item.
Here is a story that the course uses, with real photos. A customer returned a pack of four tea lights and wrote “not needed”. The team looked at the pack, saw nothing wrong, and put it back on sale as new. The next customer opened the pack and found a candle with a chunk of wax missing. She complained, and Larkfield refunded her. The same pack cost Larkfield twice: once when it came back, and once when it was sold again.

Text description of the diagram
The figure has three panels. The left panel is a photo of a good pack: four round tea lights on a dark cloth, seen from above. Each has a wick in the middle. The middle panel is a photo of a damaged pack, taken in the same way. A red box marks the lower right candle. The right panel shows the inside of the red box four times larger. Now you can see a small grey hollow in the wax, on the left of the wick: a chunk of wax is missing. In the middle photo, at its normal size, the hollow is easy to miss.
Why a quick look is not enough
Section titled “Why a quick look is not enough”Look at the middle photo for one second, as a busy person would. The damage covers 208 of the photo’s 93,120 pixels: about 1 pixel in 450. (A pixel is one small dot of colour. A photo is a grid of them. Images are numbers looks at them closely.) Once you know where to look, you see the damage at once. When you do not know, a quick look misses it.
This is true of most damage on returns. In the course’s returns photos, the most common damage is a dented corner of the metal cup or a spot of another colour. Others are dirt on the wax, missing or melted wax, and a bent wick. Most of it is small. Hamid’s team can find it with a careful check, but a careful check takes about a minute. The team has time for 100 careful checks a day, not 600.
Two mistakes, and what each one costs
Section titled “Two mistakes, and what each one costs”A decision about a returned item can be wrong in two ways. Omar, Larkfield’s product manager, asks the useful question: “What does each mistake cost us?”
Text description of the diagram
The diagram starts on the left with a returned tea light. An arrow leads to a person who looks at it. A careful check takes 1 minute and costs 0.50 dollars, and the team has time for 100 checks a day. A second arrow leads to a grid of four outcomes. The columns say whether the item is really good or damaged. The rows say whether the team sells it as new or sends it to the outlet. A good item sold as new is right and costs nothing. A damaged item sold as new is a mistake: a complaint that costs 45 dollars. That is a refund of 28, return shipping of 9 and 8 for handling the complaint. A good item sent to the outlet is the other mistake: it loses its average price of 28 dollars. A damaged item sent to the outlet is right and costs nothing. The two mistakes have a thick border.
The course uses the same costs in every lesson. They are assumptions: realistic numbers that we chose, not measurements. You can change them for your own work.
| What happens | Cost (dollars) | Why |
|---|---|---|
| A damaged item is sold as new | 45 | Refund 28 + return shipping 9 + handling the complaint 8 |
| A good item is sent to the outlet | 28 | Its average price: at the outlet it loses its value |
| A person checks one item carefully | 0.50 | 1 minute at 30 dollars an hour |
| Hamid’s team | 100 checks a day | For about 600 returns a day |
The last row is Hamid’s review budget: the number of careful checks his team can do in a day.
Notice that the two mistakes do not cost the same. Selling a damaged item costs more than losing a good one, because a complaint adds shipping and a person’s time to the refund. So a good decision rule does not treat them the same. It accepts a few more good items at the outlet to avoid a complaint, but not many more.
A tiny calculation: one day at the warehouse
Section titled “A tiny calculation: one day at the warehouse”How much damage arrives in a day? In the course’s test photos of returns, 18 of 218 items are damaged: 8.3%. At 600 returns a day, that is 600 × 18 ÷ 218 = 49.54 damaged items a day. (A part of an item is not a mistake: it is an average over many days.) The other 550.46 items are good.
Now price four ways to run the warehouse. Each line uses only the numbers above. The results are rounded to one decimal.
| How Hamid’s team decides | The calculation | Cost a day (dollars) |
|---|---|---|
| No careful check: everything goes back on sale | 49.54 damaged sold × 45 | 2,229.4 |
| 100 careful checks a day, on items chosen at random | 100 checks find 49.54 × 100 ÷ 600 = 8.26 damaged items. The other 41.28 are sold: 41.28 × 45 + 100 × 0.50 | 1,907.8 |
| Everything goes to the outlet, to be safe | 550.46 good items lost × 28 | 15,412.8 |
| A perfect helper: it sends only the 49.54 damaged items to a person, who confirms them | 49.54 × 0.50 | 24.8 |
Two results stand out. First, random checks save only about 322 dollars a day. They use the whole budget, but five checks in six land on good items. Second, the perfect helper costs only 24.8 dollars a day, almost 90 times less than no check. The money is not in checking more items. It is in checking the right items.
Text description of the chart
A column chart with three columns, on an axis from 0 to 2,500 dollars a day. No careful check costs 2,229.4 dollars a day. One hundred careful checks on items chosen at random cost 1,907.8. A perfect helper, which sends only the damaged items to a person, costs 24.8. The third column is so short that it almost disappears.
This chart is the first rows of the course’s comparison table: one table that compares every method on the same photos, in the same way. You open it in Meet the Larkfield images. Every method that you build later lands somewhere between the perfect helper and no check. Its place shows what it is worth to Hamid.
Why “use AI on the photos” is not yet a plan
Section titled “Why “use AI on the photos” is not yet a plan”Tomás, Larkfield’s AI engineer, has a quick answer: “We take a photo of every return. Let’s use AI on the photos.” He is right that a camera and a model can help. A model can look at all 600 photos, and it does not get tired. But his sentence is a wish, not a plan. Nobody can build it, measure it or say what it will cost.
Priya, Larkfield’s data scientist, asks four questions before she writes any code:
| Part | The question | For Hamid’s returns |
|---|---|---|
| Output | What exactly must the system say, for each photo? | One of three answers: pass (sell as new), check (a person looks), fail (outlet) |
| Camera | How are the photos taken? | One fixed camera above a table, the same light every day, one photo a pack |
| Cost | What does each mistake cost? | 45 dollars for a damaged item sold, 28 for a good item lost, 0.50 for a check |
| Fallback | What happens when the system is unsure or broken? | A person checks: at most 100 items a day. If the system stops, the team checks by hand as today |
Now the wish becomes a problem: “For each returned pack, take one photo with the fixed camera. Say pass, check or fail. Send at most 100 items a day to a person. Keep the cost of the two mistakes and the checks as low as possible. If the system is unsure or stops, a person decides.”
Each part matters. Without the output, two people build two different systems. Without a fixed camera, a model trained on one light fails under another, as you see in Images are numbers. Without the cost, nobody can say if a model is good enough. And without the fallback, one bad day of the system becomes 600 bad decisions.
Not every problem needs a camera and a model
Section titled “Not every problem needs a camera and a model”Sometimes the cheapest answer is not a model. A barcode tells you which product is in a box, with no mistakes. A seal on the pack shows that nobody opened it. Better light at the packing table can help people more than any model. The course compares every method with these simple answers, and Module 7 comes back to them.
Guided practice: price a day
Section titled “Guided practice: price a day”You can do this practice with a pen, or with a calculator.
- Write the costs: 45 dollars for a damaged item sold, 28 for a good item lost, 0.50 for one check.
- Write the day: 600 returns, 49.54 of them damaged.
- Price a helper that sends exactly 100 items a day to a person. Of these 100 items, 40 are damaged.
- Count the damaged items that the helper misses. They are sold as new.
- Add the cost of these sold items and the cost of the 100 checks.
- Compare your total with random checks (1,907.8) and with the perfect helper (24.8).
Show the answer
The helper misses 49.54 − 40 = 9.54 damaged items, and each one costs 45 dollars. So the cost is 9.54 × 45 + 100 × 0.50 = 429.3 + 50 = 479.3 dollars a day. Here, the person checks every item that the helper sends, so no good item goes to the outlet by mistake.
That is 1,428.5 dollars a day less than random checks, with the same 100 checks. The helper did not add any people. It only chose better which items they check.
Your turn: a smaller team
Section titled “Your turn: a smaller team”Change one thing. Two people in Hamid’s team move to another job, and the team can now do only 50 careful checks a day. Before you calculate, predict: does the cost of random checks go up a little or a lot? Then calculate it.
Show the answer
Fifty random checks find 49.54 × 50 ÷ 600 = 4.13 damaged items. The other 45.41 are sold: 45.41 × 45 = 2,043.6 dollars. The checks cost 50 × 0.50 = 25. The total is 2,068.6 dollars a day.
That is only 160.8 dollars more than with 100 random checks, because random checks were weak anyway. A smaller budget hurts much more when checks are chosen well. A good helper must then put the 50 checks on the items most likely to be damaged.
Common mistake
A model that is right 91.7% of the time
Symptom: Tomás shows a first model. “It is right on 200 of the 218 test photos: 91.7%.” It sounds good, and nothing in the number looks wrong.
Cause: this model says “good” for every photo. The test photos have 200 good and 18 damaged items, so “always good” is right 200 times. It finds none of the damaged items. For Hamid, it is exactly the same as no check: 2,229.4 dollars a day. When one kind of answer is rare, accuracy, the share of right answers, says almost nothing. The machine learning course showed the same trap in Classification metrics.
Fix: judge every method by what Hamid pays: the cost a day of the two mistakes and the checks, with the budget of 100 checks. Also count how many damaged items it finds. This share of the damaged items is the recall.
Consequence in AI work: a high accuracy on rare damage can hide a model that finds nothing. A team that trusts it ships a system that costs the same as no system, and it learns this only from the complaints.
The case studies at the end of the course
Section titled “The case studies at the end of the course”Larkfield keeps one set of problems in view while you learn each method. Real work has other images and other traps, so the course ends with three case studies on real, public images. Each one takes a dataset from the raw files to a recommendation.
Case studies in this course
At the end of the course, you use what you learn on a realistic project, from the data to the recommendation. Complete at least one for the certificate; the others are optional.
- Visual inspection with few defectsInspect industrial parts from an open inspection set: compare a supervised classifier, PatchCore and a CLIP prompt, and choose thresholds from an inspection budget.
- Classify products and pets from photosFine-grained classification on open photos: compare a histogram baseline, zero-shot CLIP, a linear probe and fine-tuning against labelled images per class, with the worst class.
- Count and detect in crowded scenesDetect and count objects in crowded open photos: compare two detectors, a density method and a vision-language model on count error by crowding.
Read the list as a map of where the course goes. The case studies use the whole course, so most learners start them after Module 8.
Knowledge check
Section titled “Knowledge check”Check your understanding
Explain why “use AI on the photos” is not yet a plan for Hamid's returns. Rewrite it with an output, a camera, a cost and a fallback.
Show the answer
The sentence does not say what the system must answer or how the photos are taken. It does not say what a mistake costs, or what happens when the system is unsure. So nobody can build it or measure it, and two methods cannot be compared. A rewrite: “For each returned pack, take one photo with the fixed camera and the same light (camera). Say pass, check or fail (output). A damaged item sold costs 45 dollars, a good item lost 28, a check 0.50 (cost). Send at most 100 items a day to a person, and let a person decide when the system is unsure or stopped (fallback).”
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: a helper sends 100 items a day to a person, and 40 of them are damaged. Will it cost more or less than 100 random checks? Show the calculation.
Show the answer
Less. Of 49.54 damaged items a day, the helper sends 40 to a person, so 9.54 are sold as new: 9.54 × 45 = 429.3 dollars. The 100 checks cost 50 dollars. The total is 479.3 dollars a day. Random checks find only about 8.26 damaged items with the same 100 checks, and cost 1,907.8 dollars a day. The budget is the same; the helper only chooses better which items are checked.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás says: “A damaged item sold costs 45 dollars and a good item lost only 28, so let's send every item that looks a little odd to the outlet.” What is wrong with this plan?
Show the answer
The two costs are close: 45 against 28 dollars. So many false alarms quickly cost more than the complaints they prevent. About 550 of the 600 daily returns are good. Sending everything to the outlet costs 550.46 × 28 = 15,412.8 dollars a day, almost seven times more than no check (2,229.4). A good rule sends an item to the outlet only when the photo shows damage clearly. It sends unsure items to a person, within the budget of 100 checks.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Hamid’s team decides about 600 returned items a day: sell as new or send to the outlet. A quick look misses small damage, and a careful check (1 minute, 0.50 dollars) fits only 100 items a day.
- The two mistakes have different costs. A damaged item sold as new costs 45 dollars, and a good item sent to the outlet costs 28. These are the course’s assumptions.
- In the test photos, 8.3% of returns are damaged: about 49.5 a day. No check costs 2,229.4 dollars a day, 100 random checks 1,907.8, and a perfect helper 24.8. The money is in checking the right items.
- Accuracy hides rare damage: “always good” is right 91.7% of the time and finds nothing.
- A vision problem needs an output, a camera, a cost and a fallback. “Use AI on the photos” has none of them.
You can now price a decision about a returned item. But “pass, check or fail” is only one kind of answer that a vision system can give. The warehouse also needs to count boxes and find a product from a customer’s photo. Next, you meet the five kinds of answer, with one Larkfield photo for each.
Next: Five kinds of answer