Skip to content
Français

Control charts: is this variation normal?

  • Lesson 8
  • 40 min
  • Module 3: Statistical process control and distribution drift

Where we are

Our baselines score single payments. Grace asks a different question: the refund rate was 2.1% last week and 2.4% this week, so is that a problem or noise? In this lesson, we meet the control chart, which factories and call centres have used for a hundred years to answer exactly this.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy
Lesson 8 of 38 · Anomaly Detection: Find What Is UnusualControl charts: is this variation normal?
Course overview

What you will be able to do

Tell normal day-to-day noise from a real change, with a chart that sets its limits from a stable period. Without it, a team reacts to every small move and still misses the change that matters.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every code cell, task and check of this lesson. It builds the I-MR chart of Larkfield’s daily refund rate by hand, with pandas and NumPy. Its setup cells install the packages and download the data (less than 1 MB), so it needs no local setup. About 40 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

What you need on your computer

Nothing. The notebook runs in Colab, Kaggle or VS Code.

Tested with Python 3.12.13 in Jupyter on macOS, with pandas 3.1.1 and NumPy 2.5.4. Colab and Kaggle were not tested.

Think of the bus that you take every morning. It is almost never on time. Most days it is 1 to 4 minutes late, because of traffic lights, a slow passenger or rain. You do not call the bus company when it is 3 minutes late. You know that this is how the bus is.

Then one day it is 15 minutes late, and the next day too. Now something is different: a road is closed, or the timetable changed. You change your plans.

You make this judgement without numbers. You learned what “normal late” looks like, and you react only when a day falls far outside it. A control chart does the same thing, with numbers, for a business process.

Grace, the support lead, looks at Larkfield’s refund rate every Monday. The rate is the week’s refunds divided by its orders. Here are six weeks of autumn 2024. Nothing changed at Larkfield in these weeks: no new carrier, no new product, no new agents.

The weekly refund rate for six weeks in October and November 2024 (week ending on the date shown). Nothing changed in these weeks, and the rate still moved from 1.65% to 2.24%.
Text description of the chart

A column chart with six columns, one per week, labelled by the last day of the week. The rates are 2.19% (6 October), 2.21% (13 October), 2.24% (20 October), 1.76% (27 October), 1.65% (3 November) and 2.11% (10 November). The axis goes from 0 to 2.5%. The fourth and fifth columns are clearly lower than the others.

In the week of 3 November, the rate fell to 1.65%. A manager who reacts to every move would thank the team. Two weeks before, at 2.24%, the same manager would ask them what went wrong. Both reactions are mistakes, because nothing happened. The rate moves by chance: some weeks have a few more broken parcels than others.

So Grace’s real question is this: how big must a move be before it means something? A rate of 2.4% after 2.1% can be noise, or the first sign of a real problem. Without a method, people guess, and they guess differently.

In the 1920s, Walter Shewhart, an engineer at a telephone company, gave the two cases names.

  • Common-cause variation is the normal noise of a process. Many small causes add up: traffic, weather, one careless parcel. It is always there. You cannot remove it by reacting to single days.
  • Special-cause variation comes from something new: a new carrier that breaks more parcels, a broken data feed, an agent who refunds too much. It has a cause that you can find and fix.

A control chart separates the two. It draws the measure for each day or week, with a centre line and two control limits. A point between the limits is common-cause noise, so you do nothing. A point outside them is a signal: look for a special cause. Watching a process this way is called statistical process control (SPC). Factories, call centres, banks and hospitals use it every day.

The simplest control chart is the individuals and moving range chart (I-MR). It needs one number per day and nothing else. Here are ten days of refund rates, in percent. The moving range of a day is the size of its change from the day before.

Day 1 2 3 4 5 6 7 8 9 10
Rate (%) 2.0 2.4 1.8 2.2 2.1 1.9 2.5 2.0 2.3 1.8
Moving range 0.4 0.6 0.4 0.1 0.2 0.6 0.5 0.3 0.5

We compute the chart in four steps:

  1. The centre line is the mean of the ten rates: 21.0 / 10 = 2.10%.
  2. The mean moving range is 3.6 / 9 = 0.40 points.
  3. One sigma (the usual size of the noise) is the mean moving range divided by 1.128: 0.40 / 1.128 = 0.355 points. The number 1.128 is a constant. It turns the mean range of two values into a standard deviation.
  4. The limits are 3 sigmas from the centre: 2.10 − 1.06 = 1.04% and 2.10 + 1.06 = 3.16%.
  • Refund rate
  • Centre line
  • Control limits
The I-MR chart of ten days. Every day is between the limits of 1.04% and 3.16%, so every move is common-cause noise.
Text description of the chart

The chart shows ten dots joined by a line. They are the rates of days 1 to 10: 2.0, 2.4, 1.8, 2.2, 2.1, 1.9, 2.5, 2.0, 2.3 and 1.8%. A solid centre line is at 2.10%. Two dashed lines show the limits at 1.04% and 3.16%. All ten dots are between the dashed lines. The highest dot, day 7 at 2.5%, is still far from the upper limit.

On these ten days, the highest rate, 2.5%, is still far inside the limits. A day at 3.3% would be outside them, and worth a look.

Why the moving range, and not the usual standard deviation of all the days? Because a real shift inside the data makes the standard deviation large. Then the limits become too wide to show that same shift. The changes from one day to the next stay small, even when the level moves. So the moving range measures the noise, not the shift.

Why 3 sigmas? If the noise has a normal (bell-shaped) distribution, a point is beyond 3 sigmas by chance in about 0.27% of cases. That is roughly one false alarm in 370 points. Wider limits give fewer false alarms and catch real changes later. The next lesson measures this price.

The X-bar and R chart, when you measure several items

Section titled “The X-bar and R chart, when you measure several items”

Sometimes you measure several items at each time, not one. Suppose Larkfield times the packing of 5 orders every day. This is a constructed example with made-up minutes. The X-bar and R chart draws two things for each day. X-bar is the mean of the 5 times. R is their range: the longest time minus the shortest.

Day Packing times (minutes) Mean (X-bar) Range (R)
1 6, 8, 7, 9, 5 7.0 4
2 7, 7, 8, 6, 7 7.0 2
3 9, 6, 8, 7, 5 7.0 4
4 8, 9, 7, 8, 8 8.0 2

The centre of the X-bar chart is the mean of the four means, 7.25 minutes. The mean range is 3.0 minutes. For groups of 5, statistics tables give the constants 0.577 and 2.114. So the X-bar limits are 7.25 ± 0.577 × 3.0, which is 5.52 to 8.98 minutes. The R chart’s upper limit is 2.114 × 3.0 = 6.34 minutes. Day 4 has the highest mean, 8.0, and it is still inside. Use this chart when you measure a small group of items at each time. Larkfield’s refund rate is one number per day, so the rest of this module uses the I-MR chart and its relatives.

Where do the limits come from? Not from a target, and not from all the data. They come from a reference period when the process was stable. This is phase I. You check that period first: no known problem, and no day far outside. Then you compute the limits, and you keep them fixed. In phase II, you draw each new day against these fixed limits.

A flow from top to bottom. Phase I: a stable period, April to December 2024, 275 days, checked for known problems. Then the centre line and the limits, for the I-MR chart 0.28% to 4.00%. Phase II: draw each new day against the same limits. A question: is the day outside the limits? If no: common-cause variation, do nothing. If yes: a signal, look for a special cause such as a new carrier or a broken feed. A dashed arrow goes from the signal back to phase I, labelled: the process changed for good, new limits from a new stable period.
Phase I sets the limits from a stable period. Phase II watches new days against them. A lasting change of the process needs new limits from a new stable period.
Text description of the diagram

The diagram is a flow from top to bottom. Phase I is a stable period: Larkfield uses April to December 2024, 275 days, checked for known problems. From it come the centre line and the limits, 0.28% to 4.00% for the I-MR chart. In phase II, you draw each new day against the same limits and ask: is the day outside the limits? If not, it is common-cause variation, and you do nothing. If it is, it is a signal, and you look for a special cause, such as a new carrier or a broken feed. A dashed arrow goes back from the signal to phase I: if the process changed for good, you compute new limits from a new stable period.

Larkfield’s phase I is April to December 2024: 275 days with no planted change. Section 1 of the notebook reads refunds_daily.csv into a table called daily, with one row per day. Section 3 then computes the limits in a few lines:

daily = pd.read_csv("data/refunds_daily.csv", parse_dates=["date"]).set_index("date") # from section 1
phase1 = daily.loc["2024-04-01":"2024-12-31", "refund_rate"].dropna() # the stable reference period
center = phase1.mean()
sigma = phase1.diff().abs().mean() / 1.128
lcl, ucl = center - 3 * sigma, center + 3 * sigma
print(f"phase I: {len(phase1)} days; centre {center:.4f}, sigma {sigma:.4f}, limits {lcl:.4f} to {ucl:.4f}")

The output:

phase I: 275 days; centre 0.0214, sigma 0.0062, limits 0.0028 to 0.0400

On a normal day, the refund rate is between 0.28% and 4.00%. The centre is 2.14%, and one sigma is 0.62 points. That sigma is the answer to Grace’s question. One day’s noise is about 0.6 points, so a move from 2.1% to 2.4% on one day means nothing by itself.

In the data, we planted one lasting change: from 2025-04-07, a new delivery carrier damages more parcels. Larkfield’s refund rate rises by 0.3 points, from about 2.1% to 2.4%, and stays there. This is an anomaly we planted. In real work, you would not know the day.

Before you look at the chart, predict: how many days after the shift does the I-MR chart first go above its upper limit?

  • Refund rate of the day
  • Centre line
  • Control limits from phase I
The I-MR chart of Larkfield's daily refund rate. The planted shift starts on 7 April 2025; the first day above the upper limit is 14 July 2025, 98 days later. The day at 0 is the broken feed.
Text description of the chart

The chart shows the refund rate of each day from April 2024 to December 2025 as a thin, noisy line, mostly between 1% and 3.5%. A solid centre line is at 2.14%, and dashed limits are at 0.28% and 4.00%. In the first nine months, phase I, no day crosses a limit. After April 2025, the line is a little higher on average, but it still stays inside for months. The first point above the upper limit is 14 July 2025, marked on the chart. Three more days cross later. On 30 October and 4 December, the rate is above the limit. On 10 November, it is 0%, below the lower limit: the data feed sent zeros.

The notebook lists the days outside the limits and the first alarm after the shift:

days outside the limits: ['2025-07-14', '2025-10-30', '2025-11-10', '2025-12-04']
{'first alarm': '2025-07-14', 'days to alarm': 98, 'false alarms per month': np.float64(0.0)}
mean rate before the shift 0.0211, after 0.0244

The chart gave no false alarm in the stable months of phase II (January to early April 2025). But it found the shift only after 98 days, more than three months. The mean daily rate rose from 2.11% (January to 6 April 2025) to 2.44%. One day’s noise, 0.62 points, is twice the shift. So a single day rarely crosses the limit, even after the change.

The low point on 2025-11-10 is a signal too. A refund rate of 0% is not good news: that day, the daily feed sent zeros (an anomaly we planted). A control chart finds broken data as well as broken processes.

A chart that compares one day with 3-sigma limits is a Shewhart chart. It is good at big, sudden changes, and slow at small, lasting ones. The rest of this module gives it better tools for that.

People often mix up three kinds of lines:

  • Control limits describe what the process does when nothing special happens. They come from the data of phase I.
  • A target is what the business wants: “a refund rate of 2%”. It comes from a decision.
  • A specification limit is what the business accepts: “never above 3%”. It comes from a contract or a cost.

The limits of Larkfield’s chart are 0.28% and 4.00%. They do not say that 3.9% is acceptable. They say that 3.9% can happen by chance in this process. If the business needs a rate below 3%, the process must change. A control chart cannot change a process by being drawn.

When to use a control chart, and what it costs

Section titled “When to use a control chart, and what it costs”

Use a control chart for a process that repeats every day or every week, such as refunds, complaints, delivery times or error counts. It needs no model training. It explains itself: “this day is outside the usual range”. And you choose its false-alarm rate on purpose, with the width of the limits.

It costs a stable phase I, which you must check, and it costs patience on small changes. Do not use it for single payments, as in Module 2. Do not use it when “normal” changes with the hour and the season, unless you remove those patterns first. Module 5 shows how.

Guided practice: the chart in the notebook

Section titled “Guided practice: the chart in the notebook”
  1. Open the notebook from the box Do this lesson in a notebook at the top of this page.
  2. Run the setup cells. The data cell ends with the Ready: line.
  3. In section 1, run the cell. You see seven weekly rates from 1.65 to 2.24. The notebook shows one more week than the chart above, with 2.24%.
  4. In section 2, run the ten-day cell. You see a centre line of 2.1, a mean moving range of 0.4 and limits from 1.04 to 3.16.
  5. In section 3, run the cell. You see 275 days and limits 0.0028 to 0.0400.
  6. In section 4, run the chart cell. Four days are red.
  7. Write your prediction for the days to the first alarm. Then run the next cell. You see 'days to alarm': 98.
  8. In section 7, write the three answers and run each check cell. Each prints Check passed.

Change one thing: the width of the limits. With 2-sigma limits instead of 3, the chart alarms sooner. Before you run section 5 of the notebook, predict: how many alarms will 2-sigma limits give in the 96 stable days of phase II? Nothing changed in those days.

Show the answer
limits at 3 sigma: 0 alarms in 96 stable days (0.0 a month)
limits at 2 sigma: 7 alarms in 96 stable days (2.2 a month)

With 2-sigma limits, the chart alarms on 7 of the 96 stable days, about 2.2 times a month. Every one of these alarms is false, because nothing changed. If each alarm costs Grace an hour of checking, narrower limits cost her two hours a month for nothing. Wider limits cost late alarms instead. The width is a choice between two costs, and the next lesson measures both.

Common mistake

Limits from every day

Symptom: Tomás computes the limits from all 639 days, “to use all the data”. His chart has fewer alarms, and he is pleased.

limits from every day: 0.0040 to 0.0411
limits from phase I: 0.0028 to 0.0400

Cause: the data of every day includes the problems you want to find: the higher rate after the shift and the broken-feed day at 0%. They move the centre line and widen the limits. The problems are now part of “normal”.

Fix: compute the limits from a phase I that you checked. Keep them fixed while you watch new days. Compute new limits only after a lasting change that you understand, from a new stable period.

Consequence in AI work: a detector that learns “normal” from data that already contains the anomalies learns to accept them. The same mistake comes back in every module of this course, under other names.

Check your understanding

Grace sees a refund rate of 2.4% this week after 2.1% last week. In your own words, explain common-cause and special-cause variation, and what a control chart would tell her about this move.

Show the answer

Common-cause variation is the normal noise of the process: many small causes, always there. Special-cause variation comes from something new, with a cause to find. A control chart draws the rate with limits from a stable period. Larkfield’s daily noise is about 0.62 points (one sigma), and the limits are 0.28% to 4.00%. A move of 0.3 points is well inside the limits, so the chart says: no signal, do nothing yet. A lasting change of that size needs other charts, later in this module.

Check your understanding

Why does the I-MR chart estimate sigma from the moving range, and not from the standard deviation of all the days?

Show the answer

A real shift inside the data makes the standard deviation of all the days larger, so the limits get wider and hide the shift. The moving range measures the change from one day to the next. It stays small even when the level moves, so it measures the noise, not the shift. The mean moving range divided by 1.128 gives one sigma.

Check your understanding

Omar says: the chart's upper limit is 4.00%, so a refund rate of 3.9% is fine for the business. What do you answer?

Show the answer

Control limits are not targets and not specification limits. They describe what the process does by chance, from the data of phase I. A rate of 3.9% can happen by chance in this process, but that does not make it acceptable. Whether 3.9% is acceptable is a business decision, a specification limit. If the business needs less, the process itself must change.

Recap

  • Common-cause variation is the usual noise; a special cause is something new. A control chart separates them with limits from a stable period, phase I.
  • The I-MR chart takes its sigma from the moving range: Larkfield’s limits are 0.28% to 4.00%, with one sigma of 0.62 points. The X-bar and R chart is for small groups of items measured together.
  • Keep the limits fixed in phase II. Limits from every day hide the problems.
  • The planted 0.3-point rise was found only after 98 days, and the broken feed showed as a day at 0%.

You can now tell noise from a signal for one point. But the refund rate rose and stayed inside the limits for months: a pattern of many days, with no single day outside. Next, you add rules that see such patterns, and you measure what each rule costs in false alarms.

Next: Rules for patterns, and the price of each rule