Charts for rates and counts
- Lesson 10
- 35 min
- Module 3: Statistical process control and distribution drift
Where we are
Our charts so far assume that every day is the same size. But Larkfield has about 100 orders on a quiet day and more than 1,300 on a busy one, so a rate is less certain on a small day. In this lesson, we build p- and u-charts, whose limits change with the size of the day.
Lesson 10 of 38 · Anomaly Detection: Find What Is UnusualCharts for rates and counts
1. What "unusual" means
2. Simple baselines that are hard to beat
3. Statistical process control and distribution drift
4. Distance, density and isolation
5. Anomalies in time
6. Deep and embedding methods
7. Evaluate with few labels
8. Learn from the reviewers
9. Run a detector in production
What you will be able to do
Watch a rate or a count when the size of each day changes. A chart with one limit for every day is too strict on small days and too relaxed on big ones. So it misses a bad day of 1,200 orders.
NotebookDo this lesson in a notebook
The notebook has every code cell, task and check of this lesson. It builds the p-chart and the Laney p′-chart of the refunds, a c-chart, and the u-chart of tickets per order. Its setup cells install the packages and download the data (less than 1 MB). About 35 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- The I-MR chart, phase I and the moving range, from Control charts: is this variation normal?.
- The average run length and the price of a chart, from Rules for patterns, and the price of each rule.
What you need on your computer
Nothing. The notebook runs in Colab, Kaggle or VS Code.
Tested with Python 3.12.13 in Jupyter on macOS, with pandas 3.1.1 and NumPy 2.5.4. Colab and Kaggle were not tested.
Three shots or thirty
Section titled “Three shots or thirty”A basketball player scores 3 of 4 shots in a game: 75%. Another player scores 30 of 40: also 75%. Do you trust both numbers the same?
You do not. With 4 shots, one lucky shot moves the rate by 25 points. With 40 shots, one shot moves it by 2.5 points. The same rate means more when it comes from more tries. To call a game “unusual”, you need a bigger move from the first player than from the second.
Larkfield’s refund rate has the same property. Each day has a different number of orders, and a rate from a small day is less certain than a rate from a big day.
The problem: one limit for every day
Section titled “The problem: one limit for every day”The I-MR chart of the last two lessons has one upper limit for every day: 4.00%. Look at two real days:
| Day | Orders | Refunds | Refund rate | Above the I-MR limit (4.00%)? |
|---|---|---|---|---|
| 2024-12-25 (Christmas) | 109 | 2 | 1.84% | no |
| 2025-11-28 (Black Friday) | 1,224 | 46 | 3.76% | no |
On Christmas Day, two refunds out of 109 orders give 1.84%. One more refund would make it 2.75%, and three more would make it 4.59%. Such moves are pure chance on a day this small. On Black Friday, 1,224 orders give a much more precise rate. A rate of 3.76% on that day is far from the usual 2.14%, yet the I-MR chart says nothing. Its limit is too wide for a big day.
The p-chart: limits from the size of the day
Section titled “The p-chart: limits from the size of the day”The p-chart is a control chart for a proportion (p): refunds per order, defects per item, or calls dropped per call. It assumes that each order is refunded or not by chance, with the same chance on every day. That chance is p-bar, the refund rate of phase I. Then one sigma for a day with n orders is:
sigma of the day = √( p-bar × (1 − p-bar) / n )
The limits are p-bar ± 3 sigma of the day. More orders give a smaller sigma, so narrower limits. A small day gets wide limits; a big day gets narrow ones.
Four days by hand
Section titled “Four days by hand”Take p-bar = 2.14% and four days. Before you read the last column, predict: which days are above their upper limit?
| Day | Orders | Refunds | Rate | Sigma of the day | Upper limit | Above? |
|---|---|---|---|---|---|---|
| A | 300 | 9 | 3.00% | 0.84 points | 4.65% | no |
| B | 1,200 | 36 | 3.00% | 0.42 points | 3.39% | no |
| C | 1,200 | 45 | 3.75% | 0.42 points | 3.39% | yes |
| D | 300 | 13 | 4.33% | 0.84 points | 4.65% | no |
Day D has the highest rate, 4.33%, but it is inside its limit: on a day of 300 orders, that is still normal noise. Day C has a lower rate, 3.75%, but on a day of 1,200 orders its limit is 3.39%. Only day C is a signal.
- Upper limit for a day of this size
- p-bar, 2.14%
Text description of the chart
The chart shows a dashed curve for the upper limit, against the number of orders on the day, from 100 to 1,400. The curve starts high, about 6.5% at 100 orders, and falls quickly, then slowly: 4.65% at 300 orders, 3.51% at 1,000 and 3.39% at 1,200. A solid line at 2.14% is p-bar. Four points are marked. A, 3.00% at 300 orders, and D, 4.33% at 300 orders, are both below the curve. B, 3.00% at 1,200 orders, is below the curve. C, 3.75% at 1,200 orders, is above it.
Larkfield’s p-chart
Section titled “Larkfield’s p-chart”The notebook computes p-bar from phase I: all the refunds of April to December 2024 divided by all their orders. Then it computes one sigma for each day:
x = daily.dropna() # every day with orders (2025-10-19 has none)p1 = x.loc["2024-04-01":"2024-12-31"] # phase Ipbar = p1.refunds.sum() / p1.orders.sum() # all refunds of phase I / all its orderssigma_day = np.sqrt(pbar * (1 - pbar) / x.orders) # one sigma per dayucl, lcl = pbar + 3 * sigma_day, (pbar - 3 * sigma_day).clip(lower=0)The output:
p-bar 0.02141; UCL at 300 orders 0.0465, at 1,000 orders 0.0351p-chart, days outside: ['2024-07-11', '2025-07-14', '2025-10-30', '2025-11-10', '2025-11-28', '2025-12-04', '2025-12-29']On a day of 300 orders, the upper limit is 4.65%. On a day of 1,000 orders, it is 3.51%. The chart marks seven days. Here are the last three months of 2025, with the limits of each day:
- Refund rate of the day
- Limits of each day
Text description of the chart
The chart shows the refund rate of each day from 1 October to 31 December 2025 as a line. Two dashed lines show the limits of each day. The limits are not straight: they come closer together on busy days and move apart on quiet days. Around Black Friday, 28 November, the upper limit falls to 3.38%, and the rate of 3.76% is above it. Other marked days are 30 October and 4 and 29 December, above the upper limit. On 10 November, the rate is 0%, below the lower limit: the feed sent zeros.
The notebook also lists the days where the p-chart and the I-MR chart disagree:
only the p-chart: ['2024-07-11', '2025-11-28', '2025-12-29']only the I-MR chart: []The p-chart finds Black Friday, which the I-MR chart missed. It also marks 2024-07-11, a day in phase I where nothing was planted. That alarm is a false alarm, and it is a sign of the next problem.
When to use a p-chart
Section titled “When to use a p-chart”Use a p-chart for a rate whose daily volume changes a lot: refunds per order, failed logins per login, late parcels per parcel. When the volume is nearly the same every day, the p-chart and the I-MR chart give almost the same limits. Then the simpler I-MR chart is enough.
When the data vary more than the model says
Section titled “When the data vary more than the model says”The p-chart assumes that every order has the same chance of a refund, on every day. Then z, the distance of a day in its own sigmas, has a standard deviation of 1. Real processes also have day effects: a storm delays the parcels, or one warehouse has a bad shift. Then z varies more than 1. This is called overdispersion: more spread than the simple model expects. With overdispersion, a p-chart gives too many alarms.
phase I: standard deviation of z 1.104 (1.000 if the simple model were right)Laney sigma_z 1.121Laney p', days outside: ['2025-07-14', '2025-10-30', '2025-11-10', '2025-11-28']In phase I, z has a standard deviation of 1.104, not 1. The Laney p′-chart (say “p-prime”) corrects this. David Laney proposed it in 2002. It measures the real spread of z from its moving range, as the I-MR chart does, and calls it sigma_z. Then it multiplies the limits of the p-chart by that number. Here sigma_z is 1.121, so the limits are 12% wider. The Laney chart keeps 4 of the 7 days. The false alarm of phase I is gone, and Black Friday stays.
The Laney chart has a price too. Its limits are wider, so it is slower on a small shift. The last lesson used the course’s simulation of 2,000 runs. For the Laney chart, it gives a false alarm every 354.5 days, and a 0.3-point shift caught after 78.0 days. The plain p-chart gives 155.1 and 40.3 days.
Counts: the c-chart and the u-chart
Section titled “Counts: the c-chart and the u-chart”Some measures are counts, not rates: complaints, errors, tickets. Each count has its own chart.
The c-chart is for a count when each day has the same chance to have events. An example is complaints at a help desk with the same opening hours every day. For such a count, the variance equals the mean, so one sigma is the square root of the mean count, c-bar. Here are ten constructed days of complaints:
| Day | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Complaints | 3 | 5 | 2 | 4 | 6 | 3 | 4 | 11 | 2 | 4 |
The mean c-bar is 4.4, and √4.4 = 2.10. The upper limit is 4.4 + 3 × 2.10 = 10.69. The lower limit would be below 0, so it is 0. Day 8, with 11 complaints, is above the limit.
The u-chart is for a count per unit, when the number of units changes: tickets per order, errors per 1,000 requests. One sigma for a day with n units is √(u-bar / n), with u-bar the tickets per order of phase I. Larkfield’s help desk gets tickets, and the number of orders changes every day. So the notebook builds the u-chart of tickets per order, from the daily orders and tickets of the shared series:
u-bar 0.4785; 727 days; u-chart alarm days 377; Laney sigma_z 3.57heat-wave week: u-chart alarms 7 of 7 days, Laney u' 1The plain u-chart alarms on 377 of 727 days, more than half of them. Tickets vary from day to day far more than a count model says: sigma_z is 3.57. The Laney u′-chart widens the limits 3.57 times, and it marks 13 days. Seven of them are Christmas Eve, Christmas Day, New Year’s Day and Black Friday. The other six are single days spread over the two years, one of them in the heat wave.
Text description of the chart
A column chart with two columns. The plain u-chart of tickets per order alarms on 377 of 727 days. The Laney u′-chart alarms on 13 days. The first column is about 29 times as tall as the second.
In the heat-wave week of 2025, tickets rose a lot. But orders rose too, so tickets per order stayed almost normal: the Laney chart marks only 1 day of the 7. A chart of a ratio answers a different question from a chart of a count. “Are there more tickets?” is not the same as “are there more tickets per order?” Choose the one that matches the decision.
Which chart for which data
Section titled “Which chart for which data”| You measure | Example | Chart |
|---|---|---|
| One number per day | delivery time of the day | I-MR |
| A few items measured together | packing time of 5 orders | X-bar and R |
| A rate, with a changing number of tries | refunds per order | p-chart (Laney p′ with overdispersion) |
| A count, with the same chance every day | complaints a day | c-chart |
| A count per unit, with a changing number of units | tickets per order | u-chart (Laney u′ with overdispersion) |
Guided practice: the charts in the notebook
Section titled “Guided practice: the charts in the notebook”- Open the notebook from the box Do this lesson in a notebook at the top of this page.
- Run the setup cells. The data cell ends with the
Ready:line. - In section 1, write your prediction for days A to D. Then run the cell. Only day C shows
True. - In section 2, run the p-chart cell. You see
p-bar 0.02141and seven days outside. - Run the chart cell. Black Friday is red.
- In section 3, run the comparison. You see three days that only the p-chart marks.
- In section 4, run the overdispersion cell. You see
Laney sigma_z 1.121and four days. - In sections 5 and 6, run the c-chart and the u-chart cells. You see
377alarm days for the plain u-chart. - In section 7, write the three answers and run each check cell. Each prints
Check passed.
Your turn: a day of 500 orders
Section titled “Your turn: a day of 500 orders”Change one thing: the size of the day. Before you compute it, predict the p-chart’s upper limit for a day of 500 orders. Is it closer to 3.51% (1,000 orders) or to 4.65% (300 orders)? In the notebook, compute ucl_500 from pbar, rounded to 4 decimals.
Show the answer
ucl_500 = round(pbar + 3 * np.sqrt(pbar * (1 - pbar) / 500), 4)Check passed: ucl_500 is 0.0408.The limit is 4.08%. It is closer to 4.65% than you may expect, because sigma falls with the square root of n. Doubling the orders from 500 to 1,000 makes sigma only √2 = 1.41 times smaller, not 2 times. So the limits narrow fast on small days and slowly on big ones.
Common mistake
A u-chart that alarms every second day
Symptom: Grace asks for a daily chart of tickets per order. Amira builds a plain u-chart. It alarms on 377 of 727 days. After a week, nobody reads it.
Cause: the u-chart assumes that tickets arrive by pure chance, with a variance equal to the mean. Real tickets come in waves: a delayed delivery makes many customers write on the same day. The spread of z is 3.57 times larger than the model says (overdispersion), so the limits are far too narrow.
Diagnose: compute z for phase I and its spread. A standard deviation of z well above 1, or a sigma_z from the moving range well above 1, means overdispersion. A chart that alarms on a large share of days in a stable period has the same cause.
Fix: use the Laney u′-chart (or the Laney p′-chart for a rate). It widens the limits by sigma_z. Here it alarms on 13 days, and 7 of them are holidays or Black Friday.
Consequence in AI work: a monitor of a count with the wrong model creates alert fatigue. People learn to ignore it, and they miss the day that matters. Check the spread of your data before you trust a formula’s limits.
Knowledge check
Section titled “Knowledge check”Check your understanding
Explain in your own words why the p-chart's limits are wider on a small day. Use the basketball player or Larkfield as your example.
Show the answer
A rate from few tries moves a lot by chance. One more refund on a day of 109 orders moves the rate by almost a point. On a day of 1,224 orders, it moves the rate by less than 0.1 point. So the p-chart computes one sigma for each day from its size, √(p-bar × (1 − p-bar) / n). A small day has a larger sigma and wider limits, and a big day has narrower ones. The basketball player with 3 of 4 shots needs a bigger move than the one with 30 of 40 before we call a game unusual.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: Larkfield's I-MR chart did not mark Black Friday (3.76% on 1,224 orders). Will the p-chart mark it, and why?
Show the answer
Yes. The I-MR chart has one upper limit for every day, 4.00%, so 3.76% is inside. The p-chart’s upper limit for a day of 1,224 orders is 3.38%, because a big day gives a precise rate. So 3.76% is above it, and the p-chart marks Black Friday. The Laney p′-chart, with limits 1.121 times wider, marks it too.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
What is overdispersion, how do you see it in phase I, and what does the Laney chart do about it?
Show the answer
Overdispersion means that the data vary more than the simple model of the chart expects, for example because of day effects. You see it in phase I when z, the distance of each day in its own sigmas, has a spread well above 1. It is 1.104 for Larkfield’s refunds, and sigma_z is 3.57 for tickets per order. The Laney chart measures sigma_z from the moving range of z and multiplies the limits by it. This removes most false alarms, at the price of a slower alarm on a small shift.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A p-chart gives each day its own limits from its size: Larkfield’s upper limit is 4.65% at 300 orders and 3.51% at 1,000. It finds Black Friday, which the I-MR chart missed.
- The c-chart is for counts with the same chance every day; the u-chart is for counts per unit, such as tickets per order.
- Overdispersion makes simple charts alarm too often. The Laney charts widen the limits by sigma_z: 1.121 for refunds, 3.57 for tickets (from 377 alarm days to 13).
You can now choose a chart for any rate or count. But every chart so far looks at one day at a time, and the planted 0.3-point rise stayed hidden for months. Next, you build charts that add up small differences over many days.