Skip to content
Français

Change points

  • Lesson 23
  • 35 min
  • Module 5: Anomalies in time

Where we are

Our detectors find days that differ from normal. Some changes do not go away: a memory leak that grows for days, or a product launch that moves normal for good. In this lesson, we find those change points with ruptures, and compare them with the CUSUM chart from the control-chart module.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy
Lesson 23 of 38 · Anomaly Detection: Find What Is UnusualChange points
Course overview

What you will be able to do

Find the moments when a series moves to a new level and stays there. Then decide what each one means: an incident to fix, or a new normal to learn. A detector that does not learn the new normal alarms forever.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every code cell, task and check of this lesson. Its setup cells install pandas, NumPy, SciPy, scikit-learn, matplotlib, statsmodels and ruptures. They download the ticket API’s metrics and events, and the payments (about 8 MB). About 25 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 35 minutes

What you need to know

What you need on your computer

The notebook route (Colab or Kaggle) or the local route, as in the first lesson of this module.

Tested with Python 3.12.13 in Jupyter on macOS, with pandas 3.1.1 and ruptures 1.1.10. The Windows commands and Colab were not tested.

On a road, the sign says 50 km/h. Then a new sign says 80. The cars’ speeds go up after the sign, and they stay up. No single car is strange. The sign is a change point: a moment after which the normal level is different.

A police officer who still uses the old limit gives a ticket to every car after the sign. The officer is not wrong about the cars. He is wrong about normal.

The problem: alerts for ever after a change

Section titled “The problem: alerts for ever after a change”

In Module 3, a CUSUM chart found a small lasting shift in the refund rate. It adds up small deviations above normal, and alarms when the sum is too large. After each alarm, it starts again from zero.

Now look at what happens when the change stays. On the ticket API, the planted configuration change on 20 October 2025, 10:00, made the latency higher for good. The cache pool went from 64 to 16 connections. We planted a rise of 12%; the measured hourly mean went from 193.8 to 221.1 ms, 14% higher. A CUSUM that still compares with the old normal fills up again after every alarm:

alarms before the leak: 6
first alarm after the leak starts: 2025-09-22 09:00:00
alarms in total: 235 | after the configuration change: 113

113 alarms in ten days: one every two hours, for the same change. The detector is right that the latency is higher. It is wrong about what to do: it should say once “normal moved here”, and then learn the new normal.

  • CUSUM statistic (k 0.5)
  • Alarm line h = 5
  • Planted leak start, configuration change
The CUSUM statistic on the hourly latency, with the daily pattern removed and normal learned on the first 14 days. It alarms each time it reaches 5, then starts again from 0. Inside the leak and after the configuration change, it climbs to 5 again and again: 235 alarms in total.
Text description of the chart

A line of the CUSUM statistic from 1 September to 30 October 2025, mostly between 0 and 5, with a dashed alarm line at 5. In some hours it jumps above 5 at once, when the latency rises fast. In the first three weeks, the line stays near 0 and touches 5 a few times (6 alarms before the leak). From the start of the leak on 22 September, it climbs to 5 within hours, drops to 0, and climbs again. That happens 99 times until the restart. After the restart, it alarms less often: 17 times in three weeks. From the configuration change on 20 October, it climbs to 5 again every two hours or so until the end of the data.

The idea: cut the series where the level changes

Section titled “The idea: cut the series where the level changes”

Here are 12 values. Where would you cut them into two parts, each with a steady level?

10, 11, 9, 10, 12, 10, 15, 14, 16, 15, 14, 16

You see it at once: after the sixth value. A program needs a number for “steady”. For each possible cut, it computes the cost: how far each value is from the mean of its own part, squared, and added up. A good cut has a low cost.

Cut before value Left mean Right mean Cost
5 10.00 14.00 32.00
6 10.40 14.29 30.63
7 10.33 15.00 9.33
8 11.00 15.00 28.00
9 11.38 15.25 34.62

With no cut, the cost is 74.67. The cut before value 7 brings it down to 9.33. Every other cut is worse.

The cost of every possible cut of the 12 values. The cut before value 7 has the lowest cost, 9.33. With no cut, the cost is 74.67.
Text description of the chart

Nine columns, one for each place to cut, from before value 3 to before value 11. The costs are 63.40, 46.22, 32.00, 30.63, 9.33, 28.00, 34.62, 52.89 and 61.60. The column for the cut before value 7, at 9.33, is in another colour and is far lower than all the others.

This is a change-point method. On a long series, it can cut more than once. But every extra cut lowers the cost a little, even in pure noise: with a cut between every two values, the cost is 0. So the method adds a penalty for each cut. It keeps a cut only if the cut lowers the cost by more than the penalty. The penalty is a choice, like a threshold: a large penalty gives few, big changes.

Two methods do this in the ruptures package:

  • PELT (Pruned Exact Linear Time) finds the best set of cuts for a given penalty. “Exact” means it finds the lowest cost; “pruned” means it skips cuts that cannot win, so it is fast.
  • Binary segmentation cuts once at the best place, then cuts each part again, until it has the number of cuts you ask for. It is faster still, but not always the best set.

Change points in the ticket API’s latency

Section titled “Change points in the ticket API’s latency”

The notebook takes the hourly mean of the p95 latency (1,440 hours) and uses its log, so a rise of 10% has the same size everywhere. Then it runs both methods. PELT gets a penalty of 2 × log(1,440) × the variance of the series. Each segment must have at least 12 hours.

Predict. We planted a slow memory leak from 22 September, 03:00, until the restart on 27 September, 04:30. We also planted the configuration change on 20 October, 10:00. Where will PELT cut?

PELT: ['2025-09-22 06:00:00', '2025-09-24 08:00:00', '2025-09-27 06:00:00', '2025-10-20 09:00:00']
binary segmentation: ['2025-09-22 06:00:00', '2025-09-23 07:00:00', '2025-09-24 08:00:00', '2025-09-25 09:00:00', '2025-09-27 06:00:00', '2025-10-20 09:00:00']
2025-09-01 00:00:00 to 2025-09-22 05:00:00: 193.1 ms
2025-09-22 06:00:00 to 2025-09-24 07:00:00: 230.7 ms
2025-09-24 08:00:00 to 2025-09-27 05:00:00: 298.0 ms
2025-09-27 06:00:00 to 2025-10-20 08:00:00: 193.8 ms
2025-10-20 09:00:00 to 2025-10-30 23:00:00: 221.1 ms
  • p95 latency, hourly mean
  • PELT segments (mean of each)
  • Planted changes
The ticket API's hourly latency, the five PELT segments and the planted changes. PELT draws the slow leak as a staircase: 193.1 ms, then 230.7, then 298.0, then back to 193.8 after the restart. The configuration change is one step, to 221.1 ms.
Text description of the chart

The hourly latency line stays between about 180 and 220 milliseconds for three weeks. Then it rises slowly to near 380 by 27 September, drops at the restart, and rises a little again from 20 October. A step line shows the PELT segments. It is flat at 193.1 milliseconds until 22 September, 06:00, and at 230.7 until 24 September, 08:00. Then it is at 298.0 until 27 September, 06:00, at 193.8 until 20 October, 09:00, and at 221.1 to the end. Three dashed vertical lines mark the planted leak start, the restart and the configuration change. PELT’s cuts are within three hours of each of them, plus one extra cut in the middle of the leak.

PELT cuts four times. Its first cut (22 September, 06:00) comes 3 hours after the leak starts. The restart cut (27 September, 06:00) is 1.5 hours after the restart. The configuration cut (20 October, 09:00) is one hour before the change, because that hour’s mean already includes its first minutes.

The extra cut in the middle of the leak teaches the most. A slow rise is not one step. A method that looks for steady levels draws it as a staircase: each step is “a new level”. Binary segmentation, asked for 6 cuts, puts two more steps in the same rise. So a change point does not always mean “something happened at this hour”. Inside a slow change, it means “the level is still moving”.

These methods look at the whole series. Like STL, they say what changed in the past. They are tools to review a week or a month, not live alerts.

The two tools answer different questions:

CUSUM PELT
Question “Has the level moved now?” “Where did the level move, in this stretch of history?”
Data it needs The past only: it runs live The whole stretch, before and after
On the leak First alarm 6 hours after the start A cut 3 hours after the start, found afterwards
After a lasting change Alarms every few hours (113 after the configuration change) One cut, then a new segment
False alarms 6 in the three weeks before the leak None in the first three weeks

So use them together. CUSUM sends the first alert. Then a change-point method on the last days says where the level moved, and how far: from 193.8 to 221.1 ms. That is what Amira needs to find the configuration change in the release log.

A change point, the level moved and stays, goes to a question: is the new level a problem? If yes: fix it, an incident, such as the leak or the cache pool. If no: a new normal, such as the app launch or a new carrier. Both paths end at the same box: learn it again, a new baseline, retrain the detector.
What to do after a change point. Fix it if it is a problem. In both cases, learn the normal again, or the detector alarms forever.
Text description of the diagram

A box on the left: a change point, the level moved and stays. An arrow goes to a question: is the new level a problem? If yes, the arrow goes to “fix it”: an incident, such as the memory leak or the small cache pool. If no, the arrow goes to “a new normal”: the app launch, or the new delivery carrier from Module 3. Both paths end at the same box: learn it again, with a new baseline, or retrain the detector.

A change point is not an alert. It is a question for a person: is the new level a problem?

  • The memory leak and the small cache pool are problems. Amira fixes them. After the fix, the old normal returns: the restart brought the latency back to 193.8 ms.
  • A new carrier, a new region, a new app are not problems. They are the new normal. The detector must learn them.

In both cases, the detector learns normal again. Otherwise, it either alarms for ever, as CUSUM did, or it treats the next real problem as “the usual”.

The product launch: a new normal in the payments

Section titled “The product launch: a new normal in the payments”

On 15 October 2025, Larkfield launched its app. Many customers started to pay on a phone that the shop had never seen for them. The notebook computes, for each day, the share of payments from a device that is new for the customer. It uses the same rule as the course’s features: a device first seen today, for a customer who paid before.

2025-10-12 5.25
2025-10-13 3.77
2025-10-14 7.58
2025-10-15 10.58
2025-10-16 20.59
2025-10-17 19.05
2025-10-18 23.90

PELT on this daily series (penalty: 2 × log(days) × the variance of July and August; at least 7 days per segment) finds:

2025-07-01 to 2025-10-13: 3.82% new device
2025-10-14 to 2025-10-23: 21.79% new device
2025-10-24 to 2025-11-02: 24.21% new device
2025-11-03 to 2025-11-12: 16.37% new device
2025-11-13 to 2025-11-22: 11.07% new device
2025-11-23 to 2025-11-30: 6.73% new device
  • New-device share
  • PELT segments
  • App launch
The daily share of payments from a device that is new for the customer, and the PELT segments. Before the app, about 3.8% a day. The first change point is on 14 October, one day before the launch. Then the share rises to about 24% and falls back in steps, as customers use the app again and their phone is no longer new.
Text description of the chart

Daily dots from 1 July to 30 November 2025. Until mid-October, they stay between about 2% and 6%, and the PELT step line is flat at 3.82%. A dashed vertical line marks the app launch on 15 October. From 14 October, the dots rise fast, to above 20% by 16 October. The step line jumps to 21.79%, then to 24.21% from 24 October. From November, the dots fall again. The step line comes down in steps: 16.37%, 11.07%, and 6.73% in the last week of November.

Two lessons are in this chart.

First, the change point is on 14 October, one day before the launch. On that day the share was already 7.58%, about twice the usual. A change-point method finds where the level changes in the data, not the date in the release plan. The data may show a change a day early (a test group, a soft launch) or late.

Second, the new normal does not stay still. A phone is “new” only on its first day. So after the launch, the share rises, then falls in steps, as more customers have already used the app once. PELT draws the fall as a staircase again. “Normal” moved, and then it kept moving. The detection job in Module 9 meets this exactly: its drift check finds the new device feature, and the job retrains.

When to use change points, and what they cost

Section titled “When to use change points, and what they cost”

Use them to review a stretch of history. After an incident, ask: “when did it start, and how far did the level move?” Before you learn a new baseline, ask: “which days belong to the new normal?” And check that a fix worked: the restart brought back 193.8 ms.

Do not use them as the live alert. They need the days after the change. Use CUSUM, a seasonal baseline or a forecast for the alert, and a change-point method to explain it.

What they cost: one choice, the penalty, which sets how many changes you get. PELT on 1,440 hours takes well under a second (measured). A slow change becomes several cuts; read them together.

Common mistake

A penalty that is too small, or too large

Symptom: you change the penalty and get a very different answer. With a small penalty, PELT cuts every few hours. With a large one, it misses a change that you know is there.

Diagnose: run PELT on the same latency with other penalties, and compare the cuts with the planted changes. The notebook’s first task does this with a penalty 4 times larger. You will see fewer cuts, and the leak’s first cut comes later. A larger penalty waits until the level has moved far enough to pay for the cut.

Recover: choose the penalty on a stretch where you know the answer: here, three weeks with no planted change should get no cut. Then write the penalty down with the reason, as you do for a threshold. Read the segments’ means, not only the cut times: a cut that moves the mean by 1% is not worth an alert.

The notebook’s section 5 has two tasks.

  1. Run PELT on the hourly latency with a penalty 4 times larger: pen=4 * penalty.
  2. List its change points as strings like "2025-09-22 06:00:00" in strict_points, then run its check cell.
  3. Find the day on which the first change point of the app-launch series starts. A change-point index is the first value of the new segment.
  4. Write it as a string "YYYY-MM-DD" in first_app_change, then run its check cell.

Check your understanding

PELT cuts the five days of the memory leak into two segments (230.7 ms, then 298.0 ms). Did two things happen on the 24th of September? Explain.

Show the answer

No. One thing happened: a slow leak, rising for five days. PELT looks for parts with a steady level, so it draws a slow rise as a staircase of steps. The extra cut means “the level is still moving”, not “a second event”. Read neighbouring cuts together, and look at the shape of the series.

Check your understanding

After the configuration change, CUSUM alarmed 113 times in ten days. Amira says the alerts were right, because the latency really was higher. What is wrong with them?

Show the answer

They all report the same change. After the first alarm, the useful news is “the level moved on 20 October and stays about 14% higher (193.8 to 221.1 ms)”. Every alarm after that costs the team time and adds nothing. A lasting change needs one decision. Fix it (here, the cache pool from 64 to 16 connections), or accept it as the new normal. In both cases, the detector learns normal again and starts from the new level.

Check your understanding

The first change point of the new-device share is on 14 October, but the app launched on 15 October. Is the change-point method wrong?

Show the answer

No. It finds where the data changed. On 14 October, the share was already 7.58%, about twice the usual 3.8%, so the new segment starts there. A change point is a measured fact about the data; the launch date is a plan. When they differ, look at the data around both dates before you decide which days are the new normal.

Recap

  • A change point is a moment after which the normal level is different, and stays different.
  • PELT and binary segmentation (in ruptures) cut a finished series where the level changes. The penalty decides how many cuts.
  • A slow change becomes a staircase of cuts. Read them together.
  • CUSUM runs live and finds a change fast, but it keeps alarming until normal is learned again: 113 alarms after one configuration change.
  • A change point is a question for a person: a problem to fix, or a new normal to learn. In both cases, the detector must learn normal again.

You can now find when the level of one series moved. But the ticket API has three metrics, and one incident moves all three at once. Next, you compare one detector per metric with one for all, and you group the alerts so that one incident gives one alert, not forty.

Next: Many metrics at once