Skip to content
Français

The payment that did not fit

  • Lesson 1
  • 25 min
  • Module 1: What "unusual" means

Where we are

In the machine learning course, we trained a model from many labelled examples and chose a threshold for the team. Some problems give us almost no labels: a stolen card is rare, and nobody marks it in advance. In this lesson, we look at one payment that does not fit, and at what a miss and a false alert each cost.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy
Lesson 1 of 38 · Anomaly Detection: Find What Is UnusualThe payment that did not fit
Course overview

What you will be able to do

See why a program needs a definition of “normal” before it can find what is unusual. Then turn a vague wish, “find fraud”, into a problem with a unit, a time, a cost and a daily budget.

Before you startWhat to know and to have readyAbout 25 minutes

What you need to know

What you need on your computer

Nothing. This lesson has no code. You open the course data in a notebook in Meet the Larkfield data, the last lesson of this module.

Think of a small café. Every Saturday morning at nine, the same woman comes in. She buys a coffee and a pastry, pays about 15 dollars with her phone, and leaves. The owner does not write anything down. After a few months, she knows this customer’s habit by heart.

Now imagine that the café’s card machine shows eight payments on the same woman’s card. They come at two in the morning, from another country, on a phone the café has never seen, all within a few minutes. The owner does not need a rule book. She knows at once that this does not fit. Perhaps the card was stolen.

Notice what the owner did. She did not ask “is this payment large?” She compared the new payments with this customer’s own habit: the day, the hour, the place, the phone and the speed. Every question in this course starts the same way: unusual compared with what?

The payment that did not fit, at Larkfield

Section titled “The payment that did not fit, at Larkfield”

Larkfield is the online shop for home and garden products that you may know from other courses. In this course, Larkfield wants to find unusual events before they cost money. Examples are stolen cards, refunds that do not fit, a failing service and a broken data feed. All of its data is made up for the course, and every problem in it is an anomaly we planted on purpose. We keep a list of the planted anomalies, called the answer key, so that we can check every detector.

Larkfield has a customer like the café’s regular: customer C-01234. From June to September 2025, she paid Larkfield 17 times. Every payment was on a Saturday, between 9:00 and 10:00 in the morning, in Canada, from the same phone (device D-6674e29). The amounts were small: 11.15 to 17.98 dollars, for kitchen and home products.

Then, on Friday 3 October 2025, these eight payments arrived:

Time Amount (dollars) Country Currency Device Category
02:14:07 24.27 PT EUR D-9f3e1a2 home
02:15:05 19.26 PT EUR D-9f3e1a2 gift_cards
02:16:16 20.66 PT EUR D-9f3e1a2 electronics
02:17:17 18.88 PT EUR D-9f3e1a2 electronics
02:18:18 24.39 PT EUR D-9f3e1a2 home
02:19:13 19.16 PT EUR D-9f3e1a2 electronics
02:20:23 24.36 PT EUR D-9f3e1a2 home
02:21:31 19.25 PT EUR D-9f3e1a2 home

PT is Portugal. The currency column says euros, but the amount column is always in Canadian dollars, so you can compare amounts directly. The eight payments took 7.4 minutes and added up to 170.23 dollars. This episode is in the answer key: a stolen card.

Put the customer’s habit next to that night, one feature at a time:

Feature The 17 usual payments The 8 payments of 3 October
Day and hour Saturday, 9:00 to 10:00 Friday, 2:14 to 2:21 at night
Country Canada Portugal
Device D-6674e29, always D-9f3e1a2, never seen for her
Time between payments one week about one minute
Amount 11.15 to 17.98 dollars 18.88 to 24.39 dollars

Four features changed a lot. One changed only a little: the amount. The chart shows all 25 payments in time order:

  • Usual Saturday payments in Canada
  • 3 October, Portugal (planted)
Customer C-01234's 25 payments. The eight planted payments (numbers 18 to 25) are only a little larger than her usual baskets. The amount alone hides the fraud.
Text description of the chart

The chart has 25 dots, one for each payment, in time order from left to right. The y-axis is the amount, from 0 to 30 dollars. The first 17 dots are the usual Saturday payments in Canada, between 11.15 and 17.98 dollars. Dots 18 to 25 are the eight planted payments from Portugal, between 18.88 and 24.39 dollars. They sit just above the usual ones. Nothing in the chart jumps out.

This is the first lesson of the course, in one chart. The thief chose small amounts, so a detector that looks only at the amount sees nothing special. The fraud is visible only when you compare the payment with the customer’s past. Look at her day, her hour, her country, her device and her pace.

The café owner checks a few payments a day. Larkfield receives about 1,875 card payments a day. On 3 October 2025, it received 1,780. If a person looked at each one for 4 minutes, that day alone would need 118.7 hours of work. Over the six months of data, Larkfield has 343,074 payments from 24,038 customers. Nobody can look at all of them.

So a program must do what the café owner does in her head. Priya, Larkfield’s data scientist, builds these programs in this course, and she compares them all in the same way. This is the job of anomaly detection: to find the rows of data that do not fit the normal pattern, among many rows that do. An anomaly is a value or an event that is unusual compared with a clear idea of normal. In everyday words, it is “the thing that does not fit”.

To copy the café owner, a program needs three answers that she never had to say out loud:

  1. Normal for whom? For all customers, or for this customer? The usual amount of all Larkfield payments is far above 15 dollars, so the eight payments look normal against everybody. Against this customer, they do not.
  2. Normal in which features? Here the country, the device, the hour and the time between payments carry the signal. The amount carries almost none.
  3. How far from normal is “too far”? Every customer changes habits sometimes. A customer who travels to Portugal for a holiday pays from Portugal too. Larkfield’s data has 703 such payments on trips, and they are honest.

Point 3 matters a lot. Unusual is not the same as wrong. A customer on holiday, a business that orders 50 garden chairs, and Black Friday are all unusual and all honest. A good detector finds what is both unusual and costly, and leaves the rest alone.

A detector can be wrong in two ways. It can stay silent on a real fraud: a missed fraud. Or it can send an alert on an honest payment: a false alert. Omar, Larkfield’s product manager, asks the useful question: “What does each mistake cost us?”

A grid of two rows and two columns. The columns are: the payment is fraud, the payment is honest. The rows are: the detector sends an alert, the detector stays silent. The four cells are: caught, one review of 4 minutes; false alert, one review of 4 minutes and, if the payment is blocked, a lost basket and some trust; missed fraud, the amount plus a 15-dollar chargeback fee per payment, 290.23 dollars for the eight payments in Portugal; nothing happens. The false alert and the missed fraud have a thick border.
The four outcomes of one alert decision. The two mistakes have a thick border. The costs are the course's assumptions.
Text description of the diagram

The diagram is a grid with two columns and two rows. The columns say whether the payment is really fraud or honest. The rows say whether the detector sends an alert or stays silent. Top left, an alert on a fraud: caught, and Sofia’s team stops the card; it costs one review of 4 minutes, about 3 dollars. Top right, an alert on an honest payment: a false alert. It costs one review. If the payment is blocked, it also costs a lost basket and some trust. Bottom left, no alert on a fraud: a missed fraud; it costs the amount plus a 15-dollar chargeback fee for each payment. For the eight payments in Portugal, that is 170.23 + 8 × 15 = 290.23 dollars. Bottom right, no alert on an honest payment: nothing happens. The two mistakes, the false alert and the missed fraud, have a thick border.

The course uses the same costs in every lesson. They are assumptions, chosen to be realistic, and you can change them for your own work:

  • A missed fraud costs the stolen amount plus a 15-dollar chargeback fee for each payment. In a chargeback, the card owner’s bank takes the money back from the shop. The bank also charges the shop a fee for it.
  • A false alert costs about 3 dollars: 4 minutes of a reviewer’s time.
  • A blocked honest payment costs a lost basket and some of the customer’s trust. The course counts these separately and gives them no money value, because a made-up number would hide a real question.

If Larkfield misses the eight payments in Portugal, it loses 170.23 + 8 × 15 = 290.23 dollars. One false alert costs 3 dollars. So 290.23 ÷ 3 ≈ 97: one missed episode like this costs about as much as 97 false alerts.

Does that mean “send many alerts”? No, for two reasons. First, each false alert takes the time of a real person. Second, Larkfield may block honest payments instead of only reviewing them. Then customers like the one on holiday in Portugal cannot pay, and some of them stop shopping at Larkfield. Omar cares about both sides, and so must the detector.

Sofia is Larkfield’s payments risk analyst. Her team reviews alerts every day: they look at the payment, call the customer if needed, and stop the card if it is stolen. They have time for 50 alerts a day. This number is the alert budget, and it is also an assumption of the course.

Think of the budget as 50 empty slots that must be filled every morning. Each slot is one alert, and each alert takes 4 minutes. The detector’s job is not “find all the fraud”. Its job is to choose which 50 payments go into the slots.

The title says: 2025-10-03, 1,780 payments, 25 of them planted fraud; Sofia's team can review 50, one slot is one alert of 4 minutes. Two grids of 50 squares each. In the first grid, one square is filled: 50 payments chosen at random contain 1 planted fraud. In the second grid, 25 squares are filled: a detector that knew the answers would fill 25 slots with the 25 planted payments of the day, and the other 25 with honest payments.
The alert budget on 3 October 2025. Chosen at random, the 50 slots hold 1 planted fraud. Even a detector that knew the answers could fill only 25 slots with fraud, because the day had only 25 planted payments.
Text description of the diagram

At the top, the picture says: 3 October 2025, 1,780 payments, of which 25 are planted fraud. Sofia’s team can review 50, and one slot is one alert of 4 minutes. Below are two grids of 50 squares, in five rows of ten. A filled square is a planted fraud; an empty square is an honest payment. In the first grid, only one square is filled: 50 payments chosen at random contained 1 planted fraud. In the second grid, 25 squares are filled. A detector that knew the answers would put all 25 planted payments of the day in the slots. The other 25 slots would still hold honest payments, because there was no more fraud to find.

The picture shows the scale of the problem. On that day, 50 slots cover 2.8% of the payments. A random choice found 1 fraud in 50. The second grid shows a surprise: even a perfect detector cannot fill every slot with fraud on a day with fewer than 50 frauds. The course scores detectors on three evaluation months. There, Larkfield has about 31 planted payments a day, and 78 of the 91 days have fewer than 50.

So the course judges a detector by one main number: of the 50 alerts that Sofia’s team reviews each day, the share that are real anomalies. This is precision at the budget. If 20 of the 50 alerts are planted frauds, the precision at the budget is 20 ÷ 50 = 0.40. The machine learning course called the same idea precision. Here we measure it only on the alerts that people can really review.

From “find fraud” to a detection problem

Section titled “From “find fraud” to a detection problem”

“Find fraud” is a wish, not a problem that a program can solve. A program needs four things:

Part The question it answers For Sofia’s payments
Unit What does one score belong to? One card payment
Time When must the score be ready? Every day, for the payments of that day
Cost What does each mistake cost? A miss: the amount + 15 dollars. A false alert: 3 dollars
Budget How many alerts can people review? 50 a day

With these four parts, “find fraud” becomes: “Each day, give every card payment a score. Put the 50 payments with the highest scores in front of Sofia’s team. Choose the scores so that the missed frauds and the false alerts cost as little as possible.” Now we can measure any detector, and compare two of them fairly.

The unit is not always a payment. Grace, Larkfield’s support lead, looks at refunds per agent per day. Amira, the developer of the help desk’s ticket-api service, looks at its latency per minute. A change of unit changes the whole problem, as the next lesson shows.

Anomaly detection is the right tool when you do not know in advance what every problem looks like. Two cases need something else:

  • You know the exact pattern. If the shop’s policy says “never accept more than 5 payments in 10 minutes from one card”, write that rule. It is fast, easy to explain and needs no data.
  • You have many examples with the right answer. Suppose you have thousands of past frauds, each marked by the bank, and new frauds look like old ones. Then a classifier that learns from those examples usually works better. The machine learning course teaches this. Labels are rare, and that changes everything explains how to choose.

Common mistake

Flag the largest payments

Symptom: Tomás, Larkfield’s AI engineer, has a quick idea: “Fraud means big money. Every day, send Sofia the 50 largest payments.” It sounds sensible, and it runs without an error.

Cause: in the course’s evaluation months, this detector finds almost nothing. Only 0.3% of its alerts are planted frauds (precision at the budget 0.003). It costs 10,264 dollars a week. Reviewing nothing at all costs 10,367 dollars a week, almost the same (both measured with the course’s costs). Every one of its reviewed alerts was a business customer’s payment, because business customers buy in larger amounts. Thieves keep their amounts small, as the eight payments in Portugal showed.

Fix: compare each payment with the customer’s own normal, not with every payment. Module 2 moves to a score per customer, and you score this “largest amounts” detector yourself in Meet the Larkfield data.

Consequence in AI work: a detector built on a guess about what fraud looks like can be worse than useless. It fills the review slots with honest customers, and the team learns to ignore the alerts.

Larkfield keeps one set of problems in view while you learn each method. Real work has other data and other traps, so the course ends with four case studies on real, public data. Each one takes a dataset from the raw file to a recommendation.

Case studies in this course

At the end of the course, you use what you learn on a realistic project, from the data to the recommendation. Complete at least one for the certificate; the others are optional.

  • Card fraud with a review budgetAnonymised card transactions: compare a rule, Isolation Forest, an autoencoder and a supervised model on the few labels, and choose by precision at a daily review budget.
  • Faults in machine or server metricsLabelled time series of real systems: compare seasonal baselines, STL residuals, change points and a multivariate detector, scored by time to detection and alerts per day.
  • Unusual readings in public open dataA public open-data series with no labels: plant anomalies, separate real events from data faults, and write the alert policy.
  • Control charts on a real production lineSensor readings and pass/fail results from a manufacturing process: control charts and drift checks on the key signals, compared with Isolation Forest.

Read the list as a map of where the course goes. The case studies use the whole course, so most learners start them after Module 9.

Check your understanding

Explain why “find fraud” is not yet a detection problem. Rewrite it for Larkfield's card payments with a unit, a time, a cost and a budget.

Show the answer

“Find fraud” does not say what one score belongs to, when it is needed, what a mistake costs or how many alerts people can review. So no detector can be measured against it, and two detectors cannot be compared. A rewrite: “Each day (time), give every card payment (unit) a score, and send the 50 highest (budget) to Sofia’s team. A missed fraud costs its amount plus a 15-dollar chargeback fee. A false alert costs about 3 dollars of review time (cost).” A blocked honest payment is counted separately.

Check your understanding

Predict: customer C-01234's eight planted payments were all between 18.88 and 24.39 dollars. Would a detector that alerts on each day's 50 largest payments find them? Why or why not?

Show the answer

No. Her usual payments were 11.15 to 17.98 dollars, so the planted ones are only a little larger. They are small compared with many honest payments. The day’s 50 largest payments are business customers’ payments, almost all honest. The fraud is visible in other features. It came from a new country (Portugal) and a new device. It came at 2 a.m. instead of Saturday morning, and once a minute instead of once a week. A good detector compares each payment with the customer’s own past.

Check your understanding

Omar says: “A missed fraud costs about 97 times more than a false alert, so let's block every payment that looks a little unusual.” What is wrong with this plan?

Show the answer

The calculation (290.23 ÷ 3 ≈ 97) compares a missed fraud with a reviewed false alert, which costs 4 minutes. Blocking is a different action. A blocked honest payment costs a lost basket and some trust, and many unusual payments are honest: trips, bulk orders, Black Friday. Also, Sofia’s team can review only 50 alerts a day, so more alerts do not mean more reviews. The detector must choose the 50 best, and blocked honest customers must be counted separately.

Recap

  • An anomaly does not fit a clear idea of normal. Customer C-01234 paid 17 times on Saturday mornings in Canada. Then 8 payments came from Portugal at 2 a.m., on a new device, in 7.4 minutes.
  • The amount hid the fraud: 18.88 to 24.39 dollars against 11.15 to 17.98. The country, the device, the hour and the pace showed it. “Unusual compared with what?” is always the first question.
  • A missed fraud costs the amount plus a 15-dollar chargeback fee per payment (290.23 dollars here). A false alert costs about 3 dollars. Blocked honest payments are counted separately. These are the course’s assumptions.
  • Sofia’s team has an alert budget of 50 a day. A detector is judged by precision at the budget: the share of the 50 reviewed alerts that are real.
  • A detection problem needs a unit, a time, a cost and a budget.

You can now say what a detector must do and what its mistakes cost. But this customer’s fraud was only one kind of unusual: some anomalies are a single strange value, and some are normal one by one. Next, you name the kinds of anomalies, with one Larkfield example of each.

Next: Kinds of anomalies