Skip to content
Français

Local density: LOF

  • Lesson 16
  • 30 min
  • Module 4: Distance, density and isolation

Where we are

Distance to the neighbours works when all normal data is equally dense. Business accounts spend more and spread wider than personal accounts, so one distance does not fit both. In this lesson, we use the Local Outlier Factor, which compares a point with the density of its own area.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy
Lesson 16 of 38 · Anomaly Detection: Find What Is UnusualLocal density: LOF
Course overview

What you will be able to do

Score a point against the density of its own area, so that “far” means “far for this kind of customer”. And learn to check, with numbers, whether a clever method really fits your data: on Larkfield’s payments, it does not.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every code cell, task and check of this lesson, with the instructions. Its setup cells install pandas, NumPy, SciPy, scikit-learn and matplotlib, and download the payments (about 7 MB). The whole notebook runs in about a minute.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need on your computer

A browser and a free Google account (Colab) or Kaggle account. Nothing to install.

Tested with Python 3.12.13 in Jupyter on macOS, with scikit-learn 1.9.1 and pandas 3.1.1. Colab, Kaggle and Windows were not tested.

In a city centre, houses stand about 10 metres apart. In the countryside, farms stand about 500 metres apart. Now imagine two buildings:

  • A house that stands 200 metres from the nearest house in the city. That is strange: there is a big empty gap in a crowded street.
  • A farm that stands 200 metres from the nearest farm. That is normal, even a little close.

Both buildings have the same distance to their nearest neighbour: 200 metres. One is strange and one is normal. The difference is the area: what is normal spacing there.

The problem: one distance for two kinds of area

Section titled “The problem: one distance for two kinds of area”

kNN distance from the last lesson uses one ruler for everybody. That works when normal data is equally crowded everywhere. Larkfield’s payments are not.

Personal accounts buy small baskets. Business accounts buy for a shop or an office. In the reference months, June to August 2025:

Account Payments Median amount 90% are below 99% are below 400 dollars or more
Personal 131,016 34.32 $ 81.56 $ 165.90 $ 0.01%
Business 42,255 257.50 $ 727.60 $ 1,693.58 $ 29.17%

A payment of 400 dollars is almost never seen on a personal account: about 1 in 10,000. On a business account, almost 1 payment in 3 is that large. So a 400-dollar payment is strange for a personal account and normal for a business account.

Business payments are also spread wider. Their normal neighbours stand far apart, like farms. A plain distance then says that many honest business payments are “far”. And it misses a strange payment that sits in a gap among the dense personal ones.

The idea: compare with the neighbours’ own spacing

Section titled “The idea: compare with the neighbours’ own spacing”

The fix is a relative question. Is this point farther from its neighbours than its neighbours are from their neighbours?

  • If a point is as crowded as its neighbours, the answer is about 1: normal for this area.
  • If the point is much farther from its neighbours than they are from theirs, the answer is much more than 1. The point is strange for this area.

This is the Local Outlier Factor, or LOF. “Local” because it compares a point only with its own area. “Factor” because the score is a ratio: how many times sparser the point is than its neighbours. The word density means how crowded an area is: many points close together is a high density.

A tiny example: a dense group and a spread-out group

Section titled “A tiny example: a dense group and a spread-out group”

Take two groups of nine points. The dense group, d1 to d9, has points 0.2 apart: think of personal accounts. The spread-out group, s1 to s9, has points 1.5 apart: think of business accounts. Then add two test points:

  • P stands 0.8 from the edge of the dense group.
  • Q stands 1.5 from the edge of the spread-out group: the same spacing as its group.

We score every point with k = 3, first with kNN distance and then with LOF.

A plane with two groups of nine points. Bottom left, a dense group of points 0.2 apart, labelled personal accounts. Right, a spread-out group of points 1.5 apart, labelled business accounts. A blue point P stands 0.8 to the right of the dense group: kNN 0.82, LOF 3.46. A dark point Q stands 1.5 to the right of the spread-out group: kNN 2.12, LOF 1.08. The corner s1 of the spread-out group has kNN 2.12, LOF 0.97. The corner d1 of the dense group has kNN 0.28, LOF 0.97. The text says: kNN distance gives its highest score, 2.12, to the 4 corners of the spread-out group and Q. LOF gives its highest score, 3.46, to P; all the others are 0.78 to 1.17.
The same 20 points scored two ways, with k = 3 (constructed). kNN distance flags the spread-out group's corners. LOF flags P, the point that is far for its own area.
Text description of the diagram

The picture shows a plane with two groups of nine points. At the bottom left, a dense group of points 0.2 apart, labelled personal accounts. On the right, a spread-out group of points 1.5 apart, labelled business accounts. A blue point, P, stands 0.8 to the right of the dense group. Its kNN distance is 0.82 and its LOF is 3.46. A dark point, Q, stands 1.5 to the right of the spread-out group. Its kNN distance is 2.12 and its LOF is 1.08. A corner of the spread-out group, s1, has a kNN distance of 2.12 and an LOF of 0.97. A corner of the dense group, d1, has 0.28 and 0.97. Below, the text says that kNN distance gives its highest score, 2.12, to the four corners of the spread-out group and to Q. LOF gives its highest score, 3.46, to P, and 0.78 to 1.17 to all the other points.

Look at the two scores of P:

  • kNN distance gives P 0.82. But it gives the corners of the spread-out group 2.12. So P is not even in kNN’s top five. The honest “farms” fill the alerts.
  • LOF gives P 3.46: P is about three and a half times sparser than its neighbours. Every other point, including Q, gets 0.78 to 1.17.

Here is the idea in numbers, in a simple form. P’s 3 nearest neighbours are d7, d8 and d9, on average 0.82 away. Their own 3 nearest neighbours are on average 0.22 away. So P is about 0.82 ÷ 0.22 ≈ 3.7 times sparser than its area. Q’s 3 nearest neighbours are on average 1.91 away, and theirs are 1.64 away: 1.91 ÷ 1.64 ≈ 1.2. scikit-learn’s exact formula smooths the distances a little. It calls them reachability distances. So it gives 3.46 and 1.08. The idea is the same.

Priya runs LOF on the payments exactly as she ran kNN distance. The same scaling. The same sample of 20,000 reference payments. 35 neighbours, a common setting for LOF. Then she adds the rows to the comparison table.

  1. Open the notebook from the box at the top of this page. Run the three setup cells.
  2. Run section 1, the tiny example. Check that P has an LOF of 3.46.
  3. Run section 2. Check the medians: 34.32 dollars for personal accounts, 257.50 for business accounts.
  4. Before you run section 3, predict: will LOF beat kNN distance (0.219 raw, 0.505 behaviour)?
  5. Run section 3. It takes about 15 seconds. Check the two new rows.

The scoring function is almost the same as the kNN one:

def lof_scores(X, k=35):
"""LOF of each evaluation payment against a sample of 20,000 reference payments. Higher = more unusual."""
ref, ev = scaled(X)
model = LocalOutlierFactor(n_neighbors=k, novelty=True).fit(ref) # novelty=True: score new points
return -model.score_samples(ev)

novelty=True tells scikit-learn that you will score new points, not the points that you trained on. Without it, LocalOutlierFactor can only score its own training data. score_samples returns the LOF with a minus sign, so the code turns it around: a higher score means more unusual.

The output:

detector setting alerts per day precision at budget recall cost per week ($) explain one alert (min)
LOF raw features, 35 neighbours 50.0 0.168 0.271 7857 3
LOF behaviour features, 35 neighbours 50.0 0.082 0.132 9248 3
kNN distance (k = 10) and LOF (35 neighbours) on the same scaled features and the same 20,000 reference payments, measured on the evaluation months. beh. = the behaviour features.
Text description of the chart

A column chart with four columns. kNN distance on the raw features: 0.219. LOF on the raw features: 0.168. kNN distance on the behaviour features: 0.505. LOF on the behaviour features: 0.082. LOF is lower in both cases, and much lower with the behaviour features.

LOF is worse than kNN distance with both sets of features. With the behaviour features, it is six times worse: 0.082 against 0.505. That is 4 planted payments in 50 alerts, against 25. The idea worked perfectly on the tiny example. On the real data, it fails. Why?

Priya does not stop at the number. She looks at the recall of each kind of fraud, with the behaviour features (measured):

Kind of fraud kNN distance LOF
Stolen card 0.896 0.142
Card testing 0.720 0.060
Account takeover 0.603 0.361

Card testing is almost invisible to LOF: it finds 6% of the card-test payments. Then she looks at the neighbours. For each evaluation payment, she finds its 35 nearest neighbours in the reference sample. Then she counts how many of them are planted payments:

For each evaluation payment, the share of planted payments among its 35 nearest reference neighbours, by kind (measured, behaviour features). A card test's neighbours are almost all old frauds.
Text description of the chart

A column chart with four columns. For a card-testing payment, 94.7% of its 35 nearest neighbours are planted payments. For a stolen-card payment, 73.3%. For an account takeover, 13.6%. For an honest payment, 0.2%.

Now the failure is clear. The reference months contain old card tests too: 123 of them in the sample of 20,000. A card test looks very much like other card tests: tiny amounts, a device that pays with many cards in an hour, often at night. So the old card tests form a crowd of their own, far from the honest payments. A new card test lands in that crowd. It is as far from its neighbours as they are from their own neighbours, so its LOF is about 1. The median LOF of the card-test payments is 1.01, the same as the median of the honest payments.

LOF answered its own question correctly: “this card test is normal for its area”. But its area is a crowd of frauds. The question that we needed was kNN’s question: is it far from its neighbours, in plain distance? A card test’s 10th nearest neighbour is a median 3.46 away, against 0.05 for an honest payment. The old frauds look close to each other only when you compare them with each other.

A bigger k looks past a small crowd. Priya measures three sizes, with the behaviour features:

Neighbours 10 35 100
Precision at the budget 0.077 0.082 0.154

100 neighbours almost doubles the result, because the neighbourhood is now larger than many groups of old frauds. It is still far below kNN distance (0.505), and it is slower. The method does not fit this data.

Use it when normal data has areas of very different density. The anomalies must be single points in the gaps. Examples: sensor readings from machines that run at different speeds, or customers of very different sizes, where each anomaly is a lone event.

Avoid it, or check it carefully, when anomalies come in groups that look alike. Examples are card tests, bot sign-ups, or a burst of errors after a release. Such a group forms an area of its own, and LOF calls it normal. This is a property of the method, not a bug.

What it costs: like kNN, LOF stores the reference points and searches neighbours for each new point. Scoring the evaluation months with the behaviour features took 5.3 seconds against 20,000 stored payments (measured). That is a little slower than kNN (4.1 seconds). It also needs scaling, for the same reasons as kNN.

The best lesson here is not about LOF. It is a habit. When a method looks clever, measure it in the same table as the simple ones. When it fails, look at the recall by kind and at a few neighbours before you tune anything. One table and one chart told Priya more than an afternoon of tuning would.

Common mistake

A local method that calls a fraud ring normal

Symptom: Tomás runs LOF with the behaviour features. Sofia’s team gets 50 alerts a day, but only about 4 are real. Card tests from one device go through without an alert, day after day.

Cause: old card tests in the reference data form a crowd of their own. A new card test sits inside that crowd, so its LOF is about 1. LOF compares each point with its own area, and here the area is made of frauds.

Diagnosis: compute the recall by kind (compare.recall_by_kind): card testing has 0.060. Then count the planted payments among the neighbours of the missed frauds: 94.7%.

Fix: use a method that measures plain distance or isolation: kNN distance (0.505) or Isolation Forest (next lesson). If you keep LOF, use a much larger neighbourhood, and measure again.

Consequence in AI work: a “local” or “contextual” method can learn that a repeated attack is normal. Attacks that repeat are exactly the ones that matter most.

In the notebook, section 5 has three tasks:

  1. In the tiny example, write P’s LOF as lof_p.
  2. Find the kind of planted fraud with the lowest recall for LOF on the behaviour features. Write its name in worst_kind.
  3. Score the behaviour features with 100 neighbours. Put the precision at the budget in lof_100. Predict first: better or worse than with 35 neighbours?
Show the expected results
  1. lof_p = 3.46.
  2. worst_kind = "card_testing" (recall 0.060).
  3. lof_100 = 0.154: better than 0.082 with 35 neighbours, because a larger neighbourhood looks past the small crowds of old frauds. Still far below kNN distance (0.505).

Check your understanding

Explain the Local Outlier Factor in plain words. What does an LOF of about 1 mean, and what does an LOF of 3.5 mean?

Show the answer

LOF compares how far a point is from its neighbours with how far those neighbours are from their own neighbours. About 1 means the point is as crowded as its area: normal there. 3.5 means the point is about three and a half times sparser than its neighbours. It is strange for its area, even if its plain distance is small.

Check your understanding

Predict: a 400-dollar payment on a personal account and a 400-dollar payment on a business account. Which one should a density-based method flag, and why does one global distance struggle here?

Show the answer

The personal one. In the reference months, about 0.01% of personal payments are 400 dollars or more, against 29.17% of business payments. Business payments are spread wide, so a global distance calls many honest business payments “far”, and those fill the alerts. A local density compares each payment with its own area, so the personal 400-dollar payment stands out and the business one does not.

Check your understanding

LOF finds 6% of the card-test payments with the behaviour features, while kNN distance finds 72%. Explain why, with the evidence from the neighbours.

Show the answer

94.7% of a card test’s 35 nearest reference neighbours are planted payments, mostly old card tests. They form a crowd of their own, far from the honest payments. A new card test is as far from its neighbours as they are from theirs, so its LOF is about 1. The median is 1.01, the same as for honest payments. kNN distance asks how far the point is from its k-th neighbour, in plain distance. Here, that neighbour is still far away (median 3.46, against 0.05 for honest payments), so kNN flags the card test.

Recap

  • LOF compares a point’s distance to its neighbours with the neighbours’ own distances. About 1: normal for the area. Much more than 1: strange for the area.
  • It beats a plain distance when areas have different densities: in the tiny example, it flags P (3.46) and not the spread-out group.
  • On Larkfield’s payments, it fails: 0.168 raw and 0.082 behaviour, against 0.219 and 0.505 for kNN distance.
  • The reason: frauds that repeat form a crowd of their own, so LOF calls them normal. Card testing recall: 0.060.
  • The habit: measure every method in the same table, and read the recall by kind and the neighbours before you tune.

You can now judge a density method on real evidence. But kNN and LOF both store thousands of points and search their neighbours for every new payment, which is slow on many rows. Next, you turn the question around: cut the data at random, and count how quickly a point ends up alone, with Isolation Forest.

Next: Isolation Forest