Control charts: is this variation normal?
- Lesson 8
- 40 min
- Module 3: Statistical process control and distribution drift
Where we are
Our baselines score single payments. Grace asks a different question: the refund rate was 2.1% last week and 2.4% this week, so is that a problem or noise? In this lesson, we meet the control chart, which factories and call centres have used for a hundred years to answer exactly this.
Lesson 8 of 38 · Anomaly Detection: Find What Is UnusualControl charts: is this variation normal?
1. What "unusual" means
2. Simple baselines that are hard to beat
3. Statistical process control and distribution drift
4. Distance, density and isolation
5. Anomalies in time
6. Deep and embedding methods
7. Evaluate with few labels
8. Learn from the reviewers
9. Run a detector in production
What you will be able to do
Tell normal day-to-day noise from a real change, with a chart that sets its limits from a stable period. Without it, a team reacts to every small move and still misses the change that matters.
NotebookDo this lesson in a notebook
The notebook has every code cell, task and check of this lesson. It builds the I-MR chart of Larkfield’s daily refund rate by hand, with pandas and NumPy. Its setup cells install the packages and download the data (less than 1 MB), so it needs no local setup. About 40 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- The mean, the standard deviation, and the distance of a value from the mean in standard deviations, from Z-scores, and why the mean is fooled.
- The refunds tables of Larkfield, from Meet the Larkfield data.
What you need on your computer
Nothing. The notebook runs in Colab, Kaggle or VS Code.
Tested with Python 3.12.13 in Jupyter on macOS, with pandas 3.1.1 and NumPy 2.5.4. Colab and Kaggle were not tested.
The bus that is a little late
Section titled “The bus that is a little late”Think of the bus that you take every morning. It is almost never on time. Most days it is 1 to 4 minutes late, because of traffic lights, a slow passenger or rain. You do not call the bus company when it is 3 minutes late. You know that this is how the bus is.
Then one day it is 15 minutes late, and the next day too. Now something is different: a road is closed, or the timetable changed. You change your plans.
You make this judgement without numbers. You learned what “normal late” looks like, and you react only when a day falls far outside it. A control chart does the same thing, with numbers, for a business process.
Grace’s question
Section titled “Grace’s question”Grace, the support lead, looks at Larkfield’s refund rate every Monday. The rate is the week’s refunds divided by its orders. Here are six weeks of autumn 2024. Nothing changed at Larkfield in these weeks: no new carrier, no new product, no new agents.
Text description of the chart
A column chart with six columns, one per week, labelled by the last day of the week. The rates are 2.19% (6 October), 2.21% (13 October), 2.24% (20 October), 1.76% (27 October), 1.65% (3 November) and 2.11% (10 November). The axis goes from 0 to 2.5%. The fourth and fifth columns are clearly lower than the others.
In the week of 3 November, the rate fell to 1.65%. A manager who reacts to every move would thank the team. Two weeks before, at 2.24%, the same manager would ask them what went wrong. Both reactions are mistakes, because nothing happened. The rate moves by chance: some weeks have a few more broken parcels than others.
So Grace’s real question is this: how big must a move be before it means something? A rate of 2.4% after 2.1% can be noise, or the first sign of a real problem. Without a method, people guess, and they guess differently.
Two kinds of variation
Section titled “Two kinds of variation”In the 1920s, Walter Shewhart, an engineer at a telephone company, gave the two cases names.
- Common-cause variation is the normal noise of a process. Many small causes add up: traffic, weather, one careless parcel. It is always there. You cannot remove it by reacting to single days.
- Special-cause variation comes from something new: a new carrier that breaks more parcels, a broken data feed, an agent who refunds too much. It has a cause that you can find and fix.
A control chart separates the two. It draws the measure for each day or week, with a centre line and two control limits. A point between the limits is common-cause noise, so you do nothing. A point outside them is a signal: look for a special cause. Watching a process this way is called statistical process control (SPC). Factories, call centres, banks and hospitals use it every day.
The I-MR chart on ten days
Section titled “The I-MR chart on ten days”The simplest control chart is the individuals and moving range chart (I-MR). It needs one number per day and nothing else. Here are ten days of refund rates, in percent. The moving range of a day is the size of its change from the day before.
| Day | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Rate (%) | 2.0 | 2.4 | 1.8 | 2.2 | 2.1 | 1.9 | 2.5 | 2.0 | 2.3 | 1.8 |
| Moving range | 0.4 | 0.6 | 0.4 | 0.1 | 0.2 | 0.6 | 0.5 | 0.3 | 0.5 |
We compute the chart in four steps:
- The centre line is the mean of the ten rates: 21.0 / 10 = 2.10%.
- The mean moving range is 3.6 / 9 = 0.40 points.
- One sigma (the usual size of the noise) is the mean moving range divided by 1.128: 0.40 / 1.128 = 0.355 points. The number 1.128 is a constant. It turns the mean range of two values into a standard deviation.
- The limits are 3 sigmas from the centre: 2.10 − 1.06 = 1.04% and 2.10 + 1.06 = 3.16%.
- Refund rate
- Centre line
- Control limits
Text description of the chart
The chart shows ten dots joined by a line. They are the rates of days 1 to 10: 2.0, 2.4, 1.8, 2.2, 2.1, 1.9, 2.5, 2.0, 2.3 and 1.8%. A solid centre line is at 2.10%. Two dashed lines show the limits at 1.04% and 3.16%. All ten dots are between the dashed lines. The highest dot, day 7 at 2.5%, is still far from the upper limit.
On these ten days, the highest rate, 2.5%, is still far inside the limits. A day at 3.3% would be outside them, and worth a look.
Why the moving range, and not the usual standard deviation of all the days? Because a real shift inside the data makes the standard deviation large. Then the limits become too wide to show that same shift. The changes from one day to the next stay small, even when the level moves. So the moving range measures the noise, not the shift.
Why 3 sigmas? If the noise has a normal (bell-shaped) distribution, a point is beyond 3 sigmas by chance in about 0.27% of cases. That is roughly one false alarm in 370 points. Wider limits give fewer false alarms and catch real changes later. The next lesson measures this price.
The X-bar and R chart, when you measure several items
Section titled “The X-bar and R chart, when you measure several items”Sometimes you measure several items at each time, not one. Suppose Larkfield times the packing of 5 orders every day. This is a constructed example with made-up minutes. The X-bar and R chart draws two things for each day. X-bar is the mean of the 5 times. R is their range: the longest time minus the shortest.
| Day | Packing times (minutes) | Mean (X-bar) | Range (R) |
|---|---|---|---|
| 1 | 6, 8, 7, 9, 5 | 7.0 | 4 |
| 2 | 7, 7, 8, 6, 7 | 7.0 | 2 |
| 3 | 9, 6, 8, 7, 5 | 7.0 | 4 |
| 4 | 8, 9, 7, 8, 8 | 8.0 | 2 |
The centre of the X-bar chart is the mean of the four means, 7.25 minutes. The mean range is 3.0 minutes. For groups of 5, statistics tables give the constants 0.577 and 2.114. So the X-bar limits are 7.25 ± 0.577 × 3.0, which is 5.52 to 8.98 minutes. The R chart’s upper limit is 2.114 × 3.0 = 6.34 minutes. Day 4 has the highest mean, 8.0, and it is still inside. Use this chart when you measure a small group of items at each time. Larkfield’s refund rate is one number per day, so the rest of this module uses the I-MR chart and its relatives.
Phase I and phase II
Section titled “Phase I and phase II”Where do the limits come from? Not from a target, and not from all the data. They come from a reference period when the process was stable. This is phase I. You check that period first: no known problem, and no day far outside. Then you compute the limits, and you keep them fixed. In phase II, you draw each new day against these fixed limits.
Text description of the diagram
The diagram is a flow from top to bottom. Phase I is a stable period: Larkfield uses April to December 2024, 275 days, checked for known problems. From it come the centre line and the limits, 0.28% to 4.00% for the I-MR chart. In phase II, you draw each new day against the same limits and ask: is the day outside the limits? If not, it is common-cause variation, and you do nothing. If it is, it is a signal, and you look for a special cause, such as a new carrier or a broken feed. A dashed arrow goes back from the signal to phase I: if the process changed for good, you compute new limits from a new stable period.
Larkfield’s phase I is April to December 2024: 275 days with no planted change. Section 1 of the notebook reads refunds_daily.csv into a table called daily, with one row per day. Section 3 then computes the limits in a few lines:
daily = pd.read_csv("data/refunds_daily.csv", parse_dates=["date"]).set_index("date") # from section 1phase1 = daily.loc["2024-04-01":"2024-12-31", "refund_rate"].dropna() # the stable reference periodcenter = phase1.mean()sigma = phase1.diff().abs().mean() / 1.128lcl, ucl = center - 3 * sigma, center + 3 * sigmaprint(f"phase I: {len(phase1)} days; centre {center:.4f}, sigma {sigma:.4f}, limits {lcl:.4f} to {ucl:.4f}")The output:
phase I: 275 days; centre 0.0214, sigma 0.0062, limits 0.0028 to 0.0400On a normal day, the refund rate is between 0.28% and 4.00%. The centre is 2.14%, and one sigma is 0.62 points. That sigma is the answer to Grace’s question. One day’s noise is about 0.6 points, so a move from 2.1% to 2.4% on one day means nothing by itself.
Larkfield’s refund rate, day by day
Section titled “Larkfield’s refund rate, day by day”In the data, we planted one lasting change: from 2025-04-07, a new delivery carrier damages more parcels. Larkfield’s refund rate rises by 0.3 points, from about 2.1% to 2.4%, and stays there. This is an anomaly we planted. In real work, you would not know the day.
Before you look at the chart, predict: how many days after the shift does the I-MR chart first go above its upper limit?
- Refund rate of the day
- Centre line
- Control limits from phase I
Text description of the chart
The chart shows the refund rate of each day from April 2024 to December 2025 as a thin, noisy line, mostly between 1% and 3.5%. A solid centre line is at 2.14%, and dashed limits are at 0.28% and 4.00%. In the first nine months, phase I, no day crosses a limit. After April 2025, the line is a little higher on average, but it still stays inside for months. The first point above the upper limit is 14 July 2025, marked on the chart. Three more days cross later. On 30 October and 4 December, the rate is above the limit. On 10 November, it is 0%, below the lower limit: the data feed sent zeros.
The notebook lists the days outside the limits and the first alarm after the shift:
days outside the limits: ['2025-07-14', '2025-10-30', '2025-11-10', '2025-12-04']{'first alarm': '2025-07-14', 'days to alarm': 98, 'false alarms per month': np.float64(0.0)}mean rate before the shift 0.0211, after 0.0244The chart gave no false alarm in the stable months of phase II (January to early April 2025). But it found the shift only after 98 days, more than three months. The mean daily rate rose from 2.11% (January to 6 April 2025) to 2.44%. One day’s noise, 0.62 points, is twice the shift. So a single day rarely crosses the limit, even after the change.
The low point on 2025-11-10 is a signal too. A refund rate of 0% is not good news: that day, the daily feed sent zeros (an anomaly we planted). A control chart finds broken data as well as broken processes.
A chart that compares one day with 3-sigma limits is a Shewhart chart. It is good at big, sudden changes, and slow at small, lasting ones. The rest of this module gives it better tools for that.
Control limits are not targets
Section titled “Control limits are not targets”People often mix up three kinds of lines:
- Control limits describe what the process does when nothing special happens. They come from the data of phase I.
- A target is what the business wants: “a refund rate of 2%”. It comes from a decision.
- A specification limit is what the business accepts: “never above 3%”. It comes from a contract or a cost.
The limits of Larkfield’s chart are 0.28% and 4.00%. They do not say that 3.9% is acceptable. They say that 3.9% can happen by chance in this process. If the business needs a rate below 3%, the process must change. A control chart cannot change a process by being drawn.
When to use a control chart, and what it costs
Section titled “When to use a control chart, and what it costs”Use a control chart for a process that repeats every day or every week, such as refunds, complaints, delivery times or error counts. It needs no model training. It explains itself: “this day is outside the usual range”. And you choose its false-alarm rate on purpose, with the width of the limits.
It costs a stable phase I, which you must check, and it costs patience on small changes. Do not use it for single payments, as in Module 2. Do not use it when “normal” changes with the hour and the season, unless you remove those patterns first. Module 5 shows how.
Guided practice: the chart in the notebook
Section titled “Guided practice: the chart in the notebook”- Open the notebook from the box Do this lesson in a notebook at the top of this page.
- Run the setup cells. The data cell ends with the
Ready:line. - In section 1, run the cell. You see seven weekly rates from 1.65 to 2.24. The notebook shows one more week than the chart above, with 2.24%.
- In section 2, run the ten-day cell. You see a centre line of
2.1, a mean moving range of0.4and limits from1.04to3.16. - In section 3, run the cell. You see
275 daysand limits0.0028 to 0.0400. - In section 4, run the chart cell. Four days are red.
- Write your prediction for the days to the first alarm. Then run the next cell. You see
'days to alarm': 98. - In section 7, write the three answers and run each check cell. Each prints
Check passed.
Your turn: narrower limits
Section titled “Your turn: narrower limits”Change one thing: the width of the limits. With 2-sigma limits instead of 3, the chart alarms sooner. Before you run section 5 of the notebook, predict: how many alarms will 2-sigma limits give in the 96 stable days of phase II? Nothing changed in those days.
Show the answer
limits at 3 sigma: 0 alarms in 96 stable days (0.0 a month)limits at 2 sigma: 7 alarms in 96 stable days (2.2 a month)With 2-sigma limits, the chart alarms on 7 of the 96 stable days, about 2.2 times a month. Every one of these alarms is false, because nothing changed. If each alarm costs Grace an hour of checking, narrower limits cost her two hours a month for nothing. Wider limits cost late alarms instead. The width is a choice between two costs, and the next lesson measures both.
Common mistake
Limits from every day
Symptom: Tomás computes the limits from all 639 days, “to use all the data”. His chart has fewer alarms, and he is pleased.
limits from every day: 0.0040 to 0.0411limits from phase I: 0.0028 to 0.0400Cause: the data of every day includes the problems you want to find: the higher rate after the shift and the broken-feed day at 0%. They move the centre line and widen the limits. The problems are now part of “normal”.
Fix: compute the limits from a phase I that you checked. Keep them fixed while you watch new days. Compute new limits only after a lasting change that you understand, from a new stable period.
Consequence in AI work: a detector that learns “normal” from data that already contains the anomalies learns to accept them. The same mistake comes back in every module of this course, under other names.
Knowledge check
Section titled “Knowledge check”Check your understanding
Grace sees a refund rate of 2.4% this week after 2.1% last week. In your own words, explain common-cause and special-cause variation, and what a control chart would tell her about this move.
Show the answer
Common-cause variation is the normal noise of the process: many small causes, always there. Special-cause variation comes from something new, with a cause to find. A control chart draws the rate with limits from a stable period. Larkfield’s daily noise is about 0.62 points (one sigma), and the limits are 0.28% to 4.00%. A move of 0.3 points is well inside the limits, so the chart says: no signal, do nothing yet. A lasting change of that size needs other charts, later in this module.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Why does the I-MR chart estimate sigma from the moving range, and not from the standard deviation of all the days?
Show the answer
A real shift inside the data makes the standard deviation of all the days larger, so the limits get wider and hide the shift. The moving range measures the change from one day to the next. It stays small even when the level moves, so it measures the noise, not the shift. The mean moving range divided by 1.128 gives one sigma.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Omar says: the chart's upper limit is 4.00%, so a refund rate of 3.9% is fine for the business. What do you answer?
Show the answer
Control limits are not targets and not specification limits. They describe what the process does by chance, from the data of phase I. A rate of 3.9% can happen by chance in this process, but that does not make it acceptable. Whether 3.9% is acceptable is a business decision, a specification limit. If the business needs less, the process itself must change.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Common-cause variation is the usual noise; a special cause is something new. A control chart separates them with limits from a stable period, phase I.
- The I-MR chart takes its sigma from the moving range: Larkfield’s limits are 0.28% to 4.00%, with one sigma of 0.62 points. The X-bar and R chart is for small groups of items measured together.
- Keep the limits fixed in phase II. Limits from every day hide the problems.
- The planted 0.3-point rise was found only after 98 days, and the broken feed showed as a day at 0%.
You can now tell noise from a signal for one point. But the refund rate rose and stayed inside the limits for months: a pattern of many days, with no single day outside. Next, you add rules that see such patterns, and you measure what each rule costs in false alarms.