The payment that did not fit
- Lesson 1
- 25 min
- Module 1: What "unusual" means
Where we are
In the machine learning course, we trained a model from many labelled examples and chose a threshold for the team. Some problems give us almost no labels: a stolen card is rare, and nobody marks it in advance. In this lesson, we look at one payment that does not fit, and at what a miss and a false alert each cost.
Lesson 1 of 38 · Anomaly Detection: Find What Is UnusualThe payment that did not fit
1. What "unusual" means
2. Simple baselines that are hard to beat
3. Statistical process control and distribution drift
4. Distance, density and isolation
5. Anomalies in time
6. Deep and embedding methods
7. Evaluate with few labels
8. Learn from the reviewers
9. Run a detector in production
What you will be able to do
See why a program needs a definition of “normal” before it can find what is unusual. Then turn a vague wish, “find fraud”, into a problem with a unit, a time, a cost and a daily budget.
Before you startWhat to know and to have ready
What you need to know
- What a table of data is: rows and columns. If you need a refresher, read Describe a dataset in Essential Mathematics and Statistics for AI.
- It helps, but it is not required, to know precision and recall from Classification metrics in Machine Learning: From Problem to Reliable Model. This lesson explains the one idea that it needs.
What you need on your computer
Nothing. This lesson has no code. You open the course data in a notebook in Meet the Larkfield data, the last lesson of this module.
The café owner who knows her customers
Section titled “The café owner who knows her customers”Think of a small café. Every Saturday morning at nine, the same woman comes in. She buys a coffee and a pastry, pays about 15 dollars with her phone, and leaves. The owner does not write anything down. After a few months, she knows this customer’s habit by heart.
Now imagine that the café’s card machine shows eight payments on the same woman’s card. They come at two in the morning, from another country, on a phone the café has never seen, all within a few minutes. The owner does not need a rule book. She knows at once that this does not fit. Perhaps the card was stolen.
Notice what the owner did. She did not ask “is this payment large?” She compared the new payments with this customer’s own habit: the day, the hour, the place, the phone and the speed. Every question in this course starts the same way: unusual compared with what?
The payment that did not fit, at Larkfield
Section titled “The payment that did not fit, at Larkfield”Larkfield is the online shop for home and garden products that you may know from other courses. In this course, Larkfield wants to find unusual events before they cost money. Examples are stolen cards, refunds that do not fit, a failing service and a broken data feed. All of its data is made up for the course, and every problem in it is an anomaly we planted on purpose. We keep a list of the planted anomalies, called the answer key, so that we can check every detector.
Larkfield has a customer like the café’s regular: customer C-01234. From June to September 2025, she paid Larkfield 17 times. Every payment was on a Saturday, between 9:00 and 10:00 in the morning, in Canada, from the same phone (device D-6674e29). The amounts were small: 11.15 to 17.98 dollars, for kitchen and home products.
Then, on Friday 3 October 2025, these eight payments arrived:
| Time | Amount (dollars) | Country | Currency | Device | Category |
|---|---|---|---|---|---|
| 02:14:07 | 24.27 | PT | EUR | D-9f3e1a2 | home |
| 02:15:05 | 19.26 | PT | EUR | D-9f3e1a2 | gift_cards |
| 02:16:16 | 20.66 | PT | EUR | D-9f3e1a2 | electronics |
| 02:17:17 | 18.88 | PT | EUR | D-9f3e1a2 | electronics |
| 02:18:18 | 24.39 | PT | EUR | D-9f3e1a2 | home |
| 02:19:13 | 19.16 | PT | EUR | D-9f3e1a2 | electronics |
| 02:20:23 | 24.36 | PT | EUR | D-9f3e1a2 | home |
| 02:21:31 | 19.25 | PT | EUR | D-9f3e1a2 | home |
PT is Portugal. The currency column says euros, but the amount column is always in Canadian dollars, so you can compare amounts directly. The eight payments took 7.4 minutes and added up to 170.23 dollars. This episode is in the answer key: a stolen card.
What changed, and what did not
Section titled “What changed, and what did not”Put the customer’s habit next to that night, one feature at a time:
| Feature | The 17 usual payments | The 8 payments of 3 October |
|---|---|---|
| Day and hour | Saturday, 9:00 to 10:00 | Friday, 2:14 to 2:21 at night |
| Country | Canada | Portugal |
| Device | D-6674e29, always | D-9f3e1a2, never seen for her |
| Time between payments | one week | about one minute |
| Amount | 11.15 to 17.98 dollars | 18.88 to 24.39 dollars |
Four features changed a lot. One changed only a little: the amount. The chart shows all 25 payments in time order:
- Usual Saturday payments in Canada
- 3 October, Portugal (planted)
Text description of the chart
The chart has 25 dots, one for each payment, in time order from left to right. The y-axis is the amount, from 0 to 30 dollars. The first 17 dots are the usual Saturday payments in Canada, between 11.15 and 17.98 dollars. Dots 18 to 25 are the eight planted payments from Portugal, between 18.88 and 24.39 dollars. They sit just above the usual ones. Nothing in the chart jumps out.
This is the first lesson of the course, in one chart. The thief chose small amounts, so a detector that looks only at the amount sees nothing special. The fraud is visible only when you compare the payment with the customer’s past. Look at her day, her hour, her country, her device and her pace.
Why a program cannot just look
Section titled “Why a program cannot just look”The café owner checks a few payments a day. Larkfield receives about 1,875 card payments a day. On 3 October 2025, it received 1,780. If a person looked at each one for 4 minutes, that day alone would need 118.7 hours of work. Over the six months of data, Larkfield has 343,074 payments from 24,038 customers. Nobody can look at all of them.
So a program must do what the café owner does in her head. Priya, Larkfield’s data scientist, builds these programs in this course, and she compares them all in the same way. This is the job of anomaly detection: to find the rows of data that do not fit the normal pattern, among many rows that do. An anomaly is a value or an event that is unusual compared with a clear idea of normal. In everyday words, it is “the thing that does not fit”.
To copy the café owner, a program needs three answers that she never had to say out loud:
- Normal for whom? For all customers, or for this customer? The usual amount of all Larkfield payments is far above 15 dollars, so the eight payments look normal against everybody. Against this customer, they do not.
- Normal in which features? Here the country, the device, the hour and the time between payments carry the signal. The amount carries almost none.
- How far from normal is “too far”? Every customer changes habits sometimes. A customer who travels to Portugal for a holiday pays from Portugal too. Larkfield’s data has 703 such payments on trips, and they are honest.
Point 3 matters a lot. Unusual is not the same as wrong. A customer on holiday, a business that orders 50 garden chairs, and Black Friday are all unusual and all honest. A good detector finds what is both unusual and costly, and leaves the rest alone.
Two mistakes, and what each one costs
Section titled “Two mistakes, and what each one costs”A detector can be wrong in two ways. It can stay silent on a real fraud: a missed fraud. Or it can send an alert on an honest payment: a false alert. Omar, Larkfield’s product manager, asks the useful question: “What does each mistake cost us?”
Text description of the diagram
The diagram is a grid with two columns and two rows. The columns say whether the payment is really fraud or honest. The rows say whether the detector sends an alert or stays silent. Top left, an alert on a fraud: caught, and Sofia’s team stops the card; it costs one review of 4 minutes, about 3 dollars. Top right, an alert on an honest payment: a false alert. It costs one review. If the payment is blocked, it also costs a lost basket and some trust. Bottom left, no alert on a fraud: a missed fraud; it costs the amount plus a 15-dollar chargeback fee for each payment. For the eight payments in Portugal, that is 170.23 + 8 × 15 = 290.23 dollars. Bottom right, no alert on an honest payment: nothing happens. The two mistakes, the false alert and the missed fraud, have a thick border.
The course uses the same costs in every lesson. They are assumptions, chosen to be realistic, and you can change them for your own work:
- A missed fraud costs the stolen amount plus a 15-dollar chargeback fee for each payment. In a chargeback, the card owner’s bank takes the money back from the shop. The bank also charges the shop a fee for it.
- A false alert costs about 3 dollars: 4 minutes of a reviewer’s time.
- A blocked honest payment costs a lost basket and some of the customer’s trust. The course counts these separately and gives them no money value, because a made-up number would hide a real question.
A tiny calculation
Section titled “A tiny calculation”If Larkfield misses the eight payments in Portugal, it loses 170.23 + 8 × 15 = 290.23 dollars. One false alert costs 3 dollars. So 290.23 ÷ 3 ≈ 97: one missed episode like this costs about as much as 97 false alerts.
Does that mean “send many alerts”? No, for two reasons. First, each false alert takes the time of a real person. Second, Larkfield may block honest payments instead of only reviewing them. Then customers like the one on holiday in Portugal cannot pay, and some of them stop shopping at Larkfield. Omar cares about both sides, and so must the detector.
Fifty slots a day
Section titled “Fifty slots a day”Sofia is Larkfield’s payments risk analyst. Her team reviews alerts every day: they look at the payment, call the customer if needed, and stop the card if it is stolen. They have time for 50 alerts a day. This number is the alert budget, and it is also an assumption of the course.
Think of the budget as 50 empty slots that must be filled every morning. Each slot is one alert, and each alert takes 4 minutes. The detector’s job is not “find all the fraud”. Its job is to choose which 50 payments go into the slots.
Text description of the diagram
At the top, the picture says: 3 October 2025, 1,780 payments, of which 25 are planted fraud. Sofia’s team can review 50, and one slot is one alert of 4 minutes. Below are two grids of 50 squares, in five rows of ten. A filled square is a planted fraud; an empty square is an honest payment. In the first grid, only one square is filled: 50 payments chosen at random contained 1 planted fraud. In the second grid, 25 squares are filled. A detector that knew the answers would put all 25 planted payments of the day in the slots. The other 25 slots would still hold honest payments, because there was no more fraud to find.
The picture shows the scale of the problem. On that day, 50 slots cover 2.8% of the payments. A random choice found 1 fraud in 50. The second grid shows a surprise: even a perfect detector cannot fill every slot with fraud on a day with fewer than 50 frauds. The course scores detectors on three evaluation months. There, Larkfield has about 31 planted payments a day, and 78 of the 91 days have fewer than 50.
So the course judges a detector by one main number: of the 50 alerts that Sofia’s team reviews each day, the share that are real anomalies. This is precision at the budget. If 20 of the 50 alerts are planted frauds, the precision at the budget is 20 ÷ 50 = 0.40. The machine learning course called the same idea precision. Here we measure it only on the alerts that people can really review.
From “find fraud” to a detection problem
Section titled “From “find fraud” to a detection problem”“Find fraud” is a wish, not a problem that a program can solve. A program needs four things:
| Part | The question it answers | For Sofia’s payments |
|---|---|---|
| Unit | What does one score belong to? | One card payment |
| Time | When must the score be ready? | Every day, for the payments of that day |
| Cost | What does each mistake cost? | A miss: the amount + 15 dollars. A false alert: 3 dollars |
| Budget | How many alerts can people review? | 50 a day |
With these four parts, “find fraud” becomes: “Each day, give every card payment a score. Put the 50 payments with the highest scores in front of Sofia’s team. Choose the scores so that the missed frauds and the false alerts cost as little as possible.” Now we can measure any detector, and compare two of them fairly.
The unit is not always a payment. Grace, Larkfield’s support lead, looks at refunds per agent per day. Amira, the developer of the help desk’s ticket-api service, looks at its latency per minute. A change of unit changes the whole problem, as the next lesson shows.
When anomaly detection is not the answer
Section titled “When anomaly detection is not the answer”Anomaly detection is the right tool when you do not know in advance what every problem looks like. Two cases need something else:
- You know the exact pattern. If the shop’s policy says “never accept more than 5 payments in 10 minutes from one card”, write that rule. It is fast, easy to explain and needs no data.
- You have many examples with the right answer. Suppose you have thousands of past frauds, each marked by the bank, and new frauds look like old ones. Then a classifier that learns from those examples usually works better. The machine learning course teaches this. Labels are rare, and that changes everything explains how to choose.
Common mistake
Flag the largest payments
Symptom: Tomás, Larkfield’s AI engineer, has a quick idea: “Fraud means big money. Every day, send Sofia the 50 largest payments.” It sounds sensible, and it runs without an error.
Cause: in the course’s evaluation months, this detector finds almost nothing. Only 0.3% of its alerts are planted frauds (precision at the budget 0.003). It costs 10,264 dollars a week. Reviewing nothing at all costs 10,367 dollars a week, almost the same (both measured with the course’s costs). Every one of its reviewed alerts was a business customer’s payment, because business customers buy in larger amounts. Thieves keep their amounts small, as the eight payments in Portugal showed.
Fix: compare each payment with the customer’s own normal, not with every payment. Module 2 moves to a score per customer, and you score this “largest amounts” detector yourself in Meet the Larkfield data.
Consequence in AI work: a detector built on a guess about what fraud looks like can be worse than useless. It fills the review slots with honest customers, and the team learns to ignore the alerts.
The case studies at the end of the course
Section titled “The case studies at the end of the course”Larkfield keeps one set of problems in view while you learn each method. Real work has other data and other traps, so the course ends with four case studies on real, public data. Each one takes a dataset from the raw file to a recommendation.
Case studies in this course
At the end of the course, you use what you learn on a realistic project, from the data to the recommendation. Complete at least one for the certificate; the others are optional.
- Card fraud with a review budgetAnonymised card transactions: compare a rule, Isolation Forest, an autoencoder and a supervised model on the few labels, and choose by precision at a daily review budget.
- Faults in machine or server metricsLabelled time series of real systems: compare seasonal baselines, STL residuals, change points and a multivariate detector, scored by time to detection and alerts per day.
- Unusual readings in public open dataA public open-data series with no labels: plant anomalies, separate real events from data faults, and write the alert policy.
- Control charts on a real production lineSensor readings and pass/fail results from a manufacturing process: control charts and drift checks on the key signals, compared with Isolation Forest.
Read the list as a map of where the course goes. The case studies use the whole course, so most learners start them after Module 9.
Knowledge check
Section titled “Knowledge check”Check your understanding
Explain why “find fraud” is not yet a detection problem. Rewrite it for Larkfield's card payments with a unit, a time, a cost and a budget.
Show the answer
“Find fraud” does not say what one score belongs to, when it is needed, what a mistake costs or how many alerts people can review. So no detector can be measured against it, and two detectors cannot be compared. A rewrite: “Each day (time), give every card payment (unit) a score, and send the 50 highest (budget) to Sofia’s team. A missed fraud costs its amount plus a 15-dollar chargeback fee. A false alert costs about 3 dollars of review time (cost).” A blocked honest payment is counted separately.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: customer C-01234's eight planted payments were all between 18.88 and 24.39 dollars. Would a detector that alerts on each day's 50 largest payments find them? Why or why not?
Show the answer
No. Her usual payments were 11.15 to 17.98 dollars, so the planted ones are only a little larger. They are small compared with many honest payments. The day’s 50 largest payments are business customers’ payments, almost all honest. The fraud is visible in other features. It came from a new country (Portugal) and a new device. It came at 2 a.m. instead of Saturday morning, and once a minute instead of once a week. A good detector compares each payment with the customer’s own past.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Omar says: “A missed fraud costs about 97 times more than a false alert, so let's block every payment that looks a little unusual.” What is wrong with this plan?
Show the answer
The calculation (290.23 ÷ 3 ≈ 97) compares a missed fraud with a reviewed false alert, which costs 4 minutes. Blocking is a different action. A blocked honest payment costs a lost basket and some trust, and many unusual payments are honest: trips, bulk orders, Black Friday. Also, Sofia’s team can review only 50 alerts a day, so more alerts do not mean more reviews. The detector must choose the 50 best, and blocked honest customers must be counted separately.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- An anomaly does not fit a clear idea of normal. Customer C-01234 paid 17 times on Saturday mornings in Canada. Then 8 payments came from Portugal at 2 a.m., on a new device, in 7.4 minutes.
- The amount hid the fraud: 18.88 to 24.39 dollars against 11.15 to 17.98. The country, the device, the hour and the pace showed it. “Unusual compared with what?” is always the first question.
- A missed fraud costs the amount plus a 15-dollar chargeback fee per payment (290.23 dollars here). A false alert costs about 3 dollars. Blocked honest payments are counted separately. These are the course’s assumptions.
- Sofia’s team has an alert budget of 50 a day. A detector is judged by precision at the budget: the share of the 50 reviewed alerts that are real.
- A detection problem needs a unit, a time, a cost and a budget.
You can now say what a detector must do and what its mistakes cost. But this customer’s fraud was only one kind of unusual: some anomalies are a single strange value, and some are normal one by one. Next, you name the kinds of anomalies, with one Larkfield example of each.
Next: Kinds of anomalies