One-Class SVM and the PyOD toolbox
- Lesson 18
- 35 min
- Module 4: Distance, density and isolation
Where we are
We have three detectors with three different interfaces. A team wants to try several and to combine them. In this lesson, we learn a boundary around normal data with a One-Class SVM, run many detectors with one interface in PyOD, and see why a combined score is often steadier.
Lesson 18 of 38 · Anomaly Detection: Find What Is UnusualOne-Class SVM and the PyOD toolbox
1. What "unusual" means
2. Simple baselines that are hard to beat
3. Statistical process control and distribution drift
4. Distance, density and isolation
5. Anomalies in time
6. Deep and embedding methods
7. Evaluate with few labels
8. Learn from the reviewers
9. Run a detector in production
What you will be able to do
Run many detectors with one interface, so that trying a new method costs one line of code. Then combine their scores safely: a combination protects you from choosing the wrong detector, but one bad member can ruin it.
NotebookDo this lesson in a notebook
The notebook has every code cell, task and check of this lesson, with the instructions. Its setup cells install pandas, NumPy, SciPy, scikit-learn, matplotlib and PyOD, and download the payments (about 7 MB). The whole notebook runs in about a minute.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- kNN distance, LOF and Isolation Forest, from the first three lessons of this module: Far from the neighbours: kNN distance, Local density: LOF and Isolation Forest.
- The semi-supervised setting (learn from data that you trust to be normal), from Labels are rare, and that changes everything.
What you need on your computer
A browser and a free Google account (Colab) or Kaggle account. Nothing to install. The setup cell installs PyOD 3.6.7, which also installs numba, a helper that makes some of its code fast.
Tested with Python 3.12.13 in Jupyter on macOS, with PyOD 3.6.7 and scikit-learn 1.9.1. Colab, Kaggle and Windows were not tested.
A fence around the field
Section titled “A fence around the field”A farmer keeps sheep in a large field with no fence. Over the summer, he watches where the sheep usually graze. Then he builds a fence around that area: not a square, but a smooth line that follows the shape of where they go. Now, a sheep outside the fence is worth a look.
Notice two things. First, the farmer never needed to see a lost sheep to build the fence. He only watched normal behaviour. Second, he chose how tight the fence is. A wide fence holds every sheep that ever wandered a little. A tight fence leaves some normal wanderers outside, but it catches a lost sheep sooner.
The problem: three detectors, three interfaces, one choice
Section titled “The problem: three detectors, three interfaces, one choice”After three lessons, Priya has three detectors. Each one has its own code:
- kNN distance used
NearestNeighborsandkneighbors, and took the last column. - LOF used
LocalOutlierFactorwithnovelty=Trueand turned the sign ofscore_samples. - Isolation Forest used
IsolationForestand also turned the sign ofscore_samples.
Each new method means new code and new chances to get the sign of a score wrong. And Priya must choose one detector before she knows which is best for next month’s frauds. Tomás asks: “Why not run them all and combine them? Surely four detectors find more than one.”
This lesson answers both problems: one interface for every detector, and an honest test of combining them.
The idea: a boundary around normal data
Section titled “The idea: a boundary around normal data”A One-Class SVM is the farmer’s fence. It learns from normal data only, and draws a smooth boundary around it. A new point inside the boundary is normal; a point outside is unusual. “One class” because it learns only one kind of data: normal. SVM stands for support vector machine, a classic method that draws boundaries. The boundary rests on a few training points at its edge, the support vectors.
This is the semi-supervised setting of Module 1: you train on data that you trust to be mostly normal, here the reference months.
The fence can bend around the shape of the data because of a kernel. The usual one is RBF (radial basis function). It measures how similar two points are by their distance: close points are very similar, and far points are not. You do not need its formula. You need its two settings:
- nu (the Greek letter, said “new”) is the share of training points that may stay outside the fence. nu = 0.1 means “leave about 10% of the training points outside”.
- gamma decides how far the influence of one training point reaches. A large gamma gives a wiggly boundary that hugs each point. A small gamma gives a smooth one.
A tiny example: 60 sheep and 2 strays
Section titled “A tiny example: 60 sheep and 2 strays”Take 60 normal points around the centre (5, 5), and 2 strange points far away. The One-Class SVM learns from the 60 normal points only. Here is its boundary with two values of nu:
Text description of the diagram
The picture has two panels. Each shows 60 grey points in a cloud around the centre of a square. Two blue points are near the top left and top right corners. In the left panel, nu is 0.1. A wide, smooth boundary goes around most of the grey points. The text says that 7 of the 60 normal points are outside, and that both blue points are outside. In the right panel, nu is 0.3. The boundary is tighter and goes around the middle of the cloud only. The text says that 17 of the 60 normal points are outside, and both blue points are outside.
| nu | Normal points outside (of 60) | Strange points outside (of 2) |
|---|---|---|
| 0.02 | 5 | 2 |
| 0.1 | 7 | 2 |
| 0.3 | 17 | 2 |
With nu = 0.3, 17 of 60 normal points (28%) are outside: close to the 30% that nu allows. The two strange points are outside in every case. On real data, the strange points are not so clear, and a tight fence means more false alerts. The course uses a small nu (0.02) and ranks by score, so the alert budget, not nu, decides the number of alerts.
One interface: PyOD
Section titled “One interface: PyOD”PyOD (Python Outlier Detection) is an open-source library with more than 40 detectors. They all have the same two calls:
fit(X): learn what normal looks like.decision_function(X): score new points. Higher always means more unusual, for every detector.
No more turning signs around, and no more special code for each method. Priya puts the four detectors of this module in one loop:
detectors = { "kNN distance": (KNN(n_neighbors=10, method="largest"), X_ref[sample]), "LOF": (LOF(n_neighbors=35, novelty=True), X_ref[sample]), "Isolation Forest": (IForest(n_estimators=200, random_state=0, n_jobs=-1), X_ref), "One-Class SVM": (OCSVM(kernel="rbf", nu=0.02, gamma="auto"), X_ref[sample[:10_000]]),}for name, (model, train) in detectors.items(): model.fit(train) # the same two calls for every detector scores[name] = model.decision_function(X_ev) # higher = more unusual, for every PyOD detector table.add(compare.score_payments(evaluation, scores[name], name, "behaviour features", explain_minutes=3))Each detector learns from what suits it. Isolation Forest takes all 173,271 reference payments. kNN and LOF take the sample of 20,000. The One-Class SVM takes 10,000, because its training time grows fast with the number of rows. gamma="auto" sets gamma to 1 divided by the number of features: 1/8 here.
- Open the notebook from the box at the top of this page. Run the three setup cells. Check that the first one prints
pyod 3.6.7. - Run section 1, the tiny example. Check: 7 normal points outside with nu = 0.1, 17 with nu = 0.3.
- Run the cell that loads the payments. It prints
(173271, 8) (169803, 8). - Run the loop of four detectors. It takes about 20 seconds. Check the four rows below.
detector setting alerts per day precision at budget recall cost per week ($) explain one alert (min) kNN distance behaviour features 50.0 0.505 0.816 2739 3 LOF behaviour features 50.0 0.082 0.132 9248 3Isolation Forest behaviour features 50.0 0.500 0.808 3218 3 One-Class SVM behaviour features 50.0 0.480 0.776 3145 3The first three rows are the same numbers as in the earlier lessons: PyOD runs the same methods. The new row, the One-Class SVM, reaches 0.480: close to kNN distance and Isolation Forest. With the raw features, it reaches 0.195 (measured).
Combine the scores
Section titled “Combine the scores”Tomás’s idea is common, and it has a name: an ensemble, a group of detectors whose scores are combined. Often it is steadier than one detector. Each detector makes different mistakes, so an average can cancel some of them.
There is a practical problem first. The four scores are on different scales: a distance, a ratio near 1, a score near 0.5, an SVM value. You cannot average a distance of 4.9 with a ratio of 1.2. So each score becomes its rank first: 0 for the most usual payment, 1 for the most unusual. Then the ranks are averaged.
Text description of the diagram
The diagram is a flow from left to right. On the left, the same scaled features of the reference months. Arrows go to four detectors: kNN distance, LOF, Isolation Forest and One-Class SVM. Each one is fitted, then scores new payments with decision_function. The four scores go into one box: rank each score, from 0 for the most usual to 1 for the most unusual. The next box averages the ranks into one combined score. The last box takes the top 50 a day into the comparison table.
Predict before you read on. Three of the four detectors have about 0.5; LOF has 0.082. What will the average of all four reach?
Text description of the chart
A column chart with six columns. kNN distance: 0.505. LOF: 0.082. Isolation Forest: 0.500. One-Class SVM: 0.480. The average of all four: 0.211. The average of three, without LOF: 0.504.
The average of all four falls to 0.211, worse than any detector except LOF. One bad member ruined the group. Remember from the LOF lesson: LOF gives card tests a normal score. In the average, LOF’s low rank pulls every card test down. So the combination finds 26.5% of the card-test payments, against 68.5 to 72% for the three good detectors.
Leave LOF out, and the average of three reaches 0.504: as good as the best single detector, kNN distance (0.505).
Is the combination steadier?
Section titled “Is the combination steadier?”“Steadier” should mean fewer bad days. Priya checks it. For each of the 91 evaluation days, she computes the precision of that day’s 50 alerts:
| Detector | Mean of the days | Spread (standard deviation) | Days below 0.3 |
|---|---|---|---|
| kNN distance | 0.505 | 0.315 | 32 |
| Isolation Forest | 0.500 | 0.321 | 32 |
| One-Class SVM | 0.480 | 0.304 | 32 |
| Average of 3 | 0.504 | 0.317 | 31 |
| Average of 4 | 0.211 | 0.196 | 71 |
Here, the average of three is not measurably steadier than its members: 31 bad days against 32. Every detector, and every combination, has at least one day with 0 planted alerts, for example 2025-09-19. On some days, very few payments are planted. The detectors agree so much that there is little to cancel.
So what does the combination buy? Look at the recall by kind (measured):
| Kind | kNN | Forest | SVM | Average of 3 |
|---|---|---|---|---|
| Stolen card | 0.896 | 0.905 | 0.850 | 0.902 |
| Card testing | 0.720 | 0.718 | 0.685 | 0.707 |
| Account takeover | 0.603 | 0.443 | 0.589 | 0.589 |
No single detector is the best on every kind, and the best one can change next month. The average of three is never far from the best on any kind. That is its real value: you do not have to guess the winner in advance. Its price: one bad member can drag it down, as LOF did.
When to use them, and what they cost
Section titled “When to use them, and what they cost”One-Class SVM. Use it when you have clean normal data and a moderate number of rows. It gives a smooth boundary and works well with a few numeric features. Its cost: training time grows quickly with the number of rows, so it learns from 10,000 payments here. It needs scaling, and its settings (nu, gamma) are not easy to choose without labels.
PyOD. Use it whenever you compare detectors. One interface means a fair comparison: the same data, the same scaling, the same scoring. Its cost: one more library, with numba as a helper. And a common interface hides the differences: PyOD will happily run LOF on data where LOF fails. You still measure.
A combination. Use it when several detectors are good and you do not know which will be best next month. Check each member first, and leave out a member that fails. A combination also needs a little more time to explain an alert. The course assumes 4 minutes instead of 3, because a reviewer reads several scores.
Common mistake
The ensemble that is worse than its members
Symptom: Tomás combines four detectors by the average of their ranks. The precision at the budget falls from about 0.5 to 0.211. Card tests that the single detectors caught now pass.
Cause: one member, LOF, scores card tests as normal (its card-testing recall is 0.060). In a plain average, one very low rank pulls the combined rank down for exactly the frauds that LOF misses.
Diagnosis: score each member alone in the comparison table before you combine. Compare the recall by kind of the combination with that of each member: the combination’s card-testing recall is 0.265.
Fix: leave out the member that fails: the average of three gives 0.504. Then measure the combination again in the same table.
Consequence in AI work: “more models” is not “more safety”. An ensemble is only as good as the members you let in, and a weak member can hide inside a group whose average looks reasonable.
Your turn
Section titled “Your turn”In the notebook, section 5 has three tasks:
- In the tiny example, write how many normal points are outside with nu = 0.3, as
outside_03. - Average the ranks of kNN distance, Isolation Forest and the One-Class SVM only, with
average_rank(). Put the precision at the budget inprecision_3. - Write the name of the detector that you left out in
left_out, and say in one sentence why that helped.
Show the expected results
outside_03 = 17.precision_3 = 0.504.left_out = "LOF". LOF scores card tests as normal, so in the average it pulls every card test down. Without it, the three good detectors agree, and the combination is as good as the best one.
Knowledge check
Section titled “Knowledge check”Check your understanding
What does a One-Class SVM learn, and what does the setting nu control? Use the tiny example.
Show the answer
It learns a smooth boundary around normal data, from normal data only (the semi-supervised setting). A point outside the boundary is unusual. nu is the share of training points that may stay outside. With nu = 0.1, 7 of 60 normal points were outside. With nu = 0.3, 17 of 60 were outside. A larger nu gives a tighter boundary, which catches strange points sooner but sends more false alerts.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Why must you turn each detector's score into a rank before you average the scores of several detectors?
Show the answer
The scores are on different scales. A kNN distance can be 4.9, and an LOF ratio is near 1. An Isolation Forest score is near 0.5, and an SVM value has its own scale. A plain average would be decided by the score with the biggest numbers. A rank (0 for the most usual, 1 for the most unusual) puts every detector on the same scale, so each one counts equally.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás says: “Four detectors always find more than one.” Answer him with the comparison table.
Show the answer
Not always. The average of all four detectors reached 0.211, worse than three of its members (0.480 to 0.505). LOF scores card tests as normal and pulls them down. Without LOF, the average of three reached 0.504: as good as the best single detector, not better. A combination protects you from choosing the wrong good detector, but only if you leave out the members that fail.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A One-Class SVM learns a smooth boundary around normal data only. nu sets how many training points may stay outside. On the payments: 0.480 with the behaviour features.
- PyOD runs more than 40 detectors with the same
fitanddecision_function, where higher always means more unusual. - To combine scores, rank each one, then average the ranks.
- A combination is only as good as its members: with LOF, 0.211; without LOF, 0.504, as good as the best single detector.
- Here it is not measurably steadier from day to day. Its value: you do not have to guess the best detector in advance.
You can now run and combine any detector in one table. But look at the numbers of this module again. The good detectors sit near 0.2 with the raw features. With the behaviour features, they sit near 0.5. Next, you build those features one by one, and measure what helped more: the better algorithm or the better features.