Skip to content
Français

Slices and fairness

  • Lesson 32
  • 30 min
  • Module 7: Evaluate with few labels

Where we are

We chose a detector and a threshold from the budget, and we learned to check its precision without a full key. But one average can hide a group of honest customers who get stopped again and again. In this lesson, we check the false alerts by account type, country, device, region and customer age, and we see why that matters for the people behind the payments. A shop's door alarm shows the problem.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy
Lesson 32 of 38 · Anomaly Detection: Find What Is UnusualSlices and fairness
Course overview

What you will be able to do

Check a detector’s false alerts group by group, and decide what to do when one group of honest customers gets far more of them. A good average can hide customers who are stopped again and again.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every code cell, task and check of this lesson. It computes the false alerts of Isolation Forest for each account type, country, device, region and customer age, and counts the customers behind them. Then it compares Isolation Forest and kNN distance for the module practice. Its setup cells install pandas and scikit-learn and download the course data (about 7 MB). About 30 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need on your computer

Nothing: the notebook runs in Colab or Kaggle. Tested with Python 3.12.13 in Jupyter on macOS, with pandas 3.1.1 and scikit-learn 1.9.1; Colab, Kaggle and Windows were not tested.

A door alarm that rings for the wrong people

Section titled “A door alarm that rings for the wrong people”

Think of a shop with an alarm at its door. It rings when a customer walks out with an item that has not been paid for. Sometimes it rings by mistake. The shop counts these mistakes: 1 false ring in every 100 customers. The manager thinks that is fine.

Now look at who hears the false rings. The garden centre next door puts the same kind of tag on its plants. So almost every customer who carries a plant from next door makes the alarm ring. For most customers, the alarm never rings by mistake. For the garden centre’s customers, it rings on almost every visit. Each time, a guard stops them and checks their bags in front of everybody.

The average, 1 in 100, is true. It still hides the real problem: a few customers carry almost all the mistakes, and they pay with their time and their dignity.

A fraud detector can do the same. In the last two lessons, we described Isolation Forest with averages over every payment. Here is one more: the share of honest payments that get an alert. An honest payment is one that is not planted, and a false alert is an alert on an honest payment.

Ten customers each made 100 honest payments. Eight are personal customers and two are business customers. This is a constructed example. The table counts each group’s false alerts:

tiny = pd.DataFrame({
"customer": ["personal"] * 8 + ["business"] * 2,
"honest payments": [100] * 10,
"false alerts": [1, 0, 1, 1, 0, 1, 0, 1, 12, 9],
})
total = tiny[["honest payments", "false alerts"]].sum()
print(f"all customers: {total['false alerts'] / total['honest payments'] * 10_000:.0f} false alerts per 10,000 honest payments")
by_group = tiny.groupby("customer")[["honest payments", "false alerts"]].sum()
by_group["per 10,000"] = by_group["false alerts"] / by_group["honest payments"] * 10_000
by_group

The output:

all customers: 260 false alerts per 10,000 honest payments
honest payments false alerts per 10,000
customer
business 200 21 1050.0
personal 800 5 62.5

All customers together get 26 false alerts in 1,000 payments, so 260 per 10,000. But the business customers get 1,050 per 10,000, and the personal customers get 62.5. The business customers have about 17 times as many. The average of 260 describes neither group.

This lesson counts false alerts per 10,000 honest payments. It is a rate, so groups of different sizes can be compared. It uses only honest payments, so a group with more fraud does not look worse because of its real frauds.

A slice is one group of the data, such as the payments of business customers, or the payments made abroad. To slice the results means to compute the same measure for each group, side by side. It is the door alarm’s second question: not “how often does it ring by mistake?”, but “for whom does it ring by mistake?”.

Suppose the mistakes of a system fall much more often on one group of people. Then the system treats those people unfairly, even when nobody planned it. This is a question of fairness. AI Security, Privacy and Responsible Design teaches it in depth. Bias and accessibility tests a system group by group. Impact and error costs decides who must review a decision that can harm someone. This lesson does not repeat that course. It shows how to find such a group in a detector’s alerts, and what Larkfield does about it.

A false alert is not only 4 minutes of Sofia’s time. It is also a real customer whose payment is held or blocked:

A flow. An honest payment, for example a business customer who orders 12 garden chairs, leads to a false alert: its score is in the top 50 of the day. The false alert leads to two boxes: a reviewer reads it (4 minutes, about 3 dollars), and the payment is held or blocked. The held payment leads to the customer, who loses time and may lose the basket and trust. A dashed arrow, the same group again and again, goes back to the honest payment.
The 3 dollars of review time is the cost that the course counts in money. The customer's cost is counted apart: the course does not invent a money value for it.
Text description of the diagram

The diagram starts with an honest payment, for example a business customer who orders 12 garden chairs. An arrow leads to a false alert: the payment’s score is in the top 50 of the day. Two arrows leave the false alert. One goes to “a reviewer reads it”: 4 minutes, about 3 dollars. The other goes to “the payment is held or blocked”. From there, an arrow leads to the customer, who loses time and may lose the basket and some trust. A dashed arrow labelled “the same group, again and again” goes from the customer back to the honest payment. The same kind of customer makes the same kind of payment next week.

The course’s cost per week counts the 3 dollars of review time. It does not give the customer’s cost a money value: a lost basket, a call to support, a customer who goes to another shop. So the cost per week alone cannot show this harm. A slice table can.

Larkfield: Isolation Forest, slice by slice

Section titled “Larkfield: Isolation Forest, slice by slice”

Priya, Larkfield’s data scientist, takes Isolation Forest at 50 alerts a day, the detector of the last lessons. Over the evaluation months, 2,274 of the 166,986 honest payments got an alert: 136.2 per 10,000. The function false_alerts_by() in the notebook gives the rate for each group of a column, with its counts:

def false_alerts_by(groups, flags=None):
"""False alerts per 10,000 honest payments, for each group; with the counts, so small groups show.
flags: one True/False per evaluation payment (an alert or not); by default, Isolation Forest's top 50 a day."""
flags = alert if flags is None else flags
t = pd.DataFrame({"group": np.asarray(groups)[honest], "alert": flags[honest]})
out = t.groupby("group").alert.agg(false_alerts="sum", honest_payments="size")
out["per 10,000"] = (out.false_alerts / out.honest_payments * 10_000).round(1)
return out.sort_values("per 10,000", ascending=False)
false_alerts_by(evaluation.account_type)

The output:

false_alerts honest_payments per 10,000
group
business 1658 37337 444.1
personal 616 129649 47.5

The business customers made 22% of the honest payments, and they got 73% of the false alerts (1,658 of 2,274). Per 10,000 honest payments, they get 444.1 false alerts, against 47.5 for personal customers: 9.3 times as many.

Isolation Forest at 50 alerts a day, September to November 2025. Business customers get 9.3 times as many false alerts as personal customers. Payments made abroad get the most, but only 272 honest payments were made abroad.
Text description of the chart

A column chart of false alerts per 10,000 honest payments, for four groups. Business customers: 444.1. Personal customers: 47.5. Payments made abroad: 661.8, from only 272 honest payments. Payments in Canada: 135.3. The average over all honest payments is 136.2.

The device matters less: desktop payments get 199.5 false alerts per 10,000, mobile 117.0 and tablet 75.9. This slice partly repeats the last one: 59.4% of business payments come from a desktop, against 25.7% of personal payments.

The behaviour features compare each payment with the customer’s own past and count the payments of the last hour. A business that buys for a project makes many large payments in one hour, from one desktop. To these features, that looks like card testing or a stolen card. Module 2 saw the same with the velocity rule: 1,785 of its 1,788 false alerts were business bulk orders.

Are the business alerts at least useful? Isolation Forest finds 243 of the 268 planted business payments (90.7%), and 2,033 of the 2,549 planted personal payments (79.8%). So it finds business frauds well, and it pays for that with many false alerts on honest businesses.

A rate per payment does not show whether the false alerts are spread over many customers or fall on the same few. So Priya counts the customers behind them:

false = evaluation[alert & honest]
per_customer = false.groupby("customer_id").size()
account = evaluation.groupby("customer_id").account_type.first()
pd.DataFrame({"customers": account.value_counts(),
"with a false alert": account[per_customer.index].value_counts(),
"with 3 or more": account[per_customer[per_customer >= 3].index].value_counts(),
"most for one customer": per_customer.groupby(account[per_customer.index]).max()})

The output:

customers with a false alert with 3 or more most for one customer
account_type
business 2105 420 245 19
personal 21289 558 7 3

In 13 weeks, 420 of the 2,105 business customers got at least one false alert: one business in five. 245 of them got three or more, and one business got 19. Among 21,289 personal customers, only 7 got three or more. Like the garden centre’s customers, the same honest businesses are stopped again and again.

Payments made abroad have the highest rate of all: 661.8 per 10,000. But look at the counts first. Only 272 honest payments were made abroad in three months, and 18 of them got an alert. A small slice moves a lot by chance, so Priya adds the 95% interval of the last lesson:

false_alerts honest_payments per 10,000 95% interval per 10,000
group
abroad 18 272 661.8 366 to 957
in Canada 2256 166714 135.3 130 to 141

The rate abroad is somewhere between 366 and 957 per 10,000. Even its low end is well above Canada’s 135.3, so the difference is real. But “661.8” has a false precision: it rests on 18 alerts. Report the counts with the rate, and an interval when a slice is small. Customers who travel are honest, and the detector stops them often, because a new country is a strong sign of a stolen card.

Priya also expected two groups to be stopped more: new customers, who have little past, and the new region east, live since 1 September 2025.

region
false_alerts honest_payments per 10,000
group
north 950 64227 147.9
coast 470 32677 143.8
south 610 46778 130.4
east 244 23304 104.7
customer age
false_alerts honest_payments per 10,000
group
older 2056 146872 140.0
first 30 days 218 20114 108.4

Both are lower than average. Customers in their first 30 days get 108.4 false alerts per 10,000, against 140.0 for older customers. The east region gets 104.7, the lowest of the four regions. A slice table that finds no problem is useful too: it replaces a guess with evidence.

What Larkfield does about the business slice

Section titled “What Larkfield does about the business slice”

Priya, Sofia and Omar discuss the business slice. Several fixes are possible, and each one has a cost:

  • Do not block on the score alone. An alert sends the payment to a reviewer, not to an automatic block. For a business alert, the first step is a call or a message to the customer. This costs review time, and it protects the customer.
  • Give the detector the missing context. The features treat a bulk order like card testing. A feature such as “this business usually orders in bulk” could help. It must be measured again in the comparison table, and it can also hide a real takeover of a business account.
  • Report the slice with the other numbers. Every report of the detector shows precision at the budget and the cost per week. It also shows the false alerts per 10,000 honest payments for business and for personal customers.

A separate quota of alerts for each group is risky: it can hide real fraud in one group to make the numbers look equal. The AI security course discusses such trade-offs.

Slice every detector whose alerts touch customers, and slice by the groups that the business can act on: account type, country, device, region, customer age. Slice again after every change of the detector or its features.

Slicing is cheap: the alerts and one column for each group. The traps are in the reading. A small slice moves by chance, so always show its count and an interval. Many slices mean many comparisons, and some will look high by chance, as in the drift tests of Module 3. And some groups need data that you should not collect without a good reason, such as a person’s age or origin. Ask before you add such a column; the AI security course covers this.

  1. Open the notebook from the box Do this lesson in a notebook at the top of this page.
  2. Run the setup cells. The data cell ends with the Ready: line.
  3. In section 1, predict whether the business customers are close to the average. Then run the tiny example. You see 1050.0 and 62.5.
  4. In section 2, run the cell that fits Isolation Forest. You see 136.2 per 10,000.
  5. In section 3, run the account-type slice. You see 444.1 for business customers.
  6. In section 4, run the abroad slice. You see 366 to 957.
  7. In section 5, run the device, region and age slices. You see 108.4 for the first 30 days.
  8. Run the customer count. You see 245 business customers with 3 or more false alerts.

The notebook’s section 6 has two tasks. First, slice the false alerts by merchant_category, and write the category with the highest rate as worst_category. Second, compute the false alerts per 10,000 honest night payments (from 00:00 to 05:59, hour from 0 to 5) as night_per_10000. Before you run them, predict which category is the worst.

Show the answers
by_category = false_alerts_by(evaluation.merchant_category)
worst_category = by_category.index[0]
night = np.where(evaluation.hour.between(0, 5), "night", "day")
night_per_10000 = float(false_alerts_by(night).loc["night", "per 10,000"])

worst_category is furniture, with 350.2 false alerts per 10,000 honest payments, then tools with 234.7. Electronics has 40.5, the lowest. Large orders of furniture and tools are what business customers buy. night_per_10000 is 841.5, against 112.6 by day: honest customers who pay at night are stopped about 7.5 times as often. The night slice has 5,395 honest payments, so it is not a tiny slice.

Common mistake

A good average, and one group that pays

Symptom: Tomás reports the detector to Omar: “Only 136 false alerts per 10,000 honest payments, about 1.4%. Customers will hardly notice.” Two weeks later, support has many calls from business customers whose orders were held. Some had three or four orders held.

Cause: the average mixes 129,649 honest personal payments, with 47.5 false alerts per 10,000, and 37,337 honest business payments, with 444.1. 245 businesses got three or more false alerts in 13 weeks. The average is right, and it describes nobody.

Fix: slice the false alerts by the groups that matter, with their counts, before a detector goes live. Report the worst slice beside the average. Then change the action for that slice, such as a call before any block, or change the features, and measure again.

Consequence in AI work: a system that is fine on average can treat one group badly, again and again, without any error in a log. The harm shows up late, as complaints or lost customers, and it can also be a legal and reputation problem.

Module practice: defend a detector to Sofia

Section titled “Module practice: defend a detector to Sofia”

This practice joins the four lessons of the module. The task: choose between Isolation Forest and kNN distance for Sofia’s team, at 50 alerts a day. Defend the choice in three sentences, with precision at the budget, the cost per week and one slice result.

In the notebook’s section 7, Priya scores both detectors with compare.score_payments() and slices kNN’s false alerts by account type:

from sklearn.neighbors import NearestNeighbors
X_ref = scaler.transform(behaviour.loc[reference.index])
sample = X_ref[np.random.default_rng(0).choice(len(X_ref), 20_000, replace=False)] # kNN, as in Module 4
dist, _ = NearestNeighbors(n_neighbors=10).fit(sample).kneighbors(scaler.transform(behaviour.loc[evaluation.index]))
knn = dist[:, -1]
table = compare.Table()
table.add(compare.score_payments(evaluation, score, "Isolation Forest", "behaviour features, 50 a day"))
table.add(compare.score_payments(evaluation, knn, "kNN distance", "behaviour features, 50 a day"))
table.show()
knn_alert = compare.daily_alerts(evaluation, knn, 50).set_index("payment_id").reviewed.reindex(evaluation.payment_id).to_numpy()
false_alerts_by(evaluation.account_type, knn_alert)

The output:

detector setting alerts per day precision at budget recall cost per week ($) explain one alert (min)
Isolation Forest behaviour features, 50 a day 50.0 0.500 0.808 3218 None
kNN distance behaviour features, 50 a day 50.0 0.505 0.816 2739 None
false_alerts honest_payments per 10,000
group
business 1056 37337 282.8
personal 1196 129649 92.2

kNN distance has almost the same precision (0.505 against 0.500), and it costs 479 dollars a week less (2,739 against 3,218). Its false alerts fall less on businesses: 282.8 per 10,000 honest business payments, against 444.1 for Isolation Forest. They fall more on personal customers: 92.2 against 47.5. So kNN spreads its false alerts more evenly: business customers get 3.1 times the personal rate, not 9.3.

Priya’s three sentences for Sofia:

We recommend kNN distance at 50 alerts a day: 50.5% of its alerts are planted frauds. It costs 2,739 dollars a week, against 3,218 for Isolation Forest (missed frauds plus 3 dollars per false alert, our assumptions). Business customers still get 282.8 false alerts per 10,000 honest payments, three times the personal rate, so every business alert starts with a call, not a block. The precision comes from planted frauds; in the first month, we review a random sample of 200 alerts to check it on real payments.

Write the same three sentences for a fraud lead who cares most about customers who travel. Use the notebook to slice kNN’s false alerts by “paid abroad” (np.where(evaluation.country == "CA", "in Canada", "abroad"), with knn_alert as the flags), and add the 95% interval as in section 4. Say whether the slice is large enough to trust, and what Larkfield should do for a payment made abroad. There is no single right answer: a good answer uses the numbers that you computed, with their counts.

Defend a detector choice to a fraud lead, as in the module practice. Your answer must include the detector, its budget and its precision at the budget. It must include its cost per week, with the costs that you assumed. And it must include one slice result with its counts, and what you will do about it. Say which number comes from planted cases, and how you will check it on real payments.

Check your understanding

Isolation Forest sends 136.2 false alerts per 10,000 honest payments. Business customers get 444.1 and personal customers 47.5. Explain how the average can be low while one group gets so many, and why the slice matters more than the average to a business customer.

Show the answer

The average weights each group by its number of payments. Personal customers made 129,649 of the 166,986 honest payments, so their low rate pulls the average down. The business customers’ high rate counts for only 37,337 payments. To one business customer, the average is not their experience. One business in five got a false alert in 13 weeks, 245 got three or more, and one got 19. Each false alert can hold a real order, so the harm falls again and again on the same customers.

Check your understanding

Payments made abroad get 661.8 false alerts per 10,000 honest payments, from 18 false alerts among 272 honest payments. How should you report this number, and what does it tell you?

Show the answer

Report it with its counts and its 95% interval: 18 of 272, about 366 to 957 per 10,000. The slice is small, so the exact rate is uncertain. But even the low end of the interval is well above Canada’s 135.3, so the difference is real. The detector stops honest customers who pay abroad much more often. A new country is a strong sign of a stolen card, so the detector treats travellers as suspects.

Check your understanding

Tomás expected new customers (first 30 days) and the new region east to get the most false alerts. The slices show 108.4 and 104.7 per 10,000, below the average of 136.2. What do you learn from this result?

Show the answer

A slice that shows no problem is still evidence: it replaces a guess with a measurement. Here the guess was wrong, and the groups at risk were others: business customers, customers who pay abroad, night payments. So measure every group that the business can act on, and do not decide in advance which ones are at risk. Slice again after every change of the detector, because a new feature can move the false alerts onto another group.

Recap

  • One average of false alerts can hide a group that gets many more. Count false alerts per 10,000 honest payments, slice by slice: business customers get 444.1, personal customers 47.5.
  • Count the customers too: 245 of 2,105 business customers got three or more false alerts in 13 weeks; one got 19.
  • Report every slice with its counts, and a 95% interval when it is small: payments abroad, 18 of 272, 366 to 957 per 10,000.
  • Some expected slices are fine: new customers (108.4) and the new region (104.7) are below average. Measure, do not assume.
  • A false alert can block an honest customer. Count that apart from the money, and change the action for the slice: a call before any block. The fairness of such decisions is the topic of the AI security course.

In this module, you measured a detector by its precision at the budget, its cost per week and its slices, with and without a full answer key. But every review that Sofia’s team makes is a label, and the detector ignores all of them. Next, you use the reviewers’ labels to re-rank the alerts.

Next: From unsupervised to semi-supervised