Skip to content
Français

A forecast is a range

  • Lesson 22
  • 35 min
  • Module 7: Uncertainty and the decision

Where we are

In Module 6, we forecast hard series, but every order still came from one number per day. Kenji needs to know how high demand could go. In Module 4, we saw that ETS and ARIMA also give a range. In this lesson, we name its parts, quantiles, and check that the range keeps its promise.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Turn a forecast into a range that says how low and how high demand could go, and check that the range keeps its promise. One number per day cannot tell Kenji how much more stock to order to be safe.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every code cell, task and check of this lesson, with the instructions. It makes the parasol’s range before the heat wave and draws the fan chart. Then it counts the coverage of the ETS ranges over the course backtest. Its setup cells install the packages and download the Larkfield data, so it needs no local setup. About 35 minutes, with about 1 minute of waiting.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 35 minutes

What you need to know

What you need on your computer

Nothing: the notebook runs in Colab or Kaggle. To run it in VS Code instead, use the folder and the environment from Meet the Larkfield data.

Tested with Python 3.12, pandas 2.3.3 and statsforecast 2.1.1 on macOS. The Larkfield data is synthetic: we made it for this course.

Think of a bus trip to the city centre. You have taken it many times. Usually it takes about 30 minutes. On a good day it takes 25, and on a bad day, with traffic, it takes 40.

Now you have two trips this week. On Monday you meet a friend for coffee. On Friday you have a job interview. For the coffee, you leave 30 minutes before. If you are 5 minutes late, your friend waits a little. For the interview, you leave 40 minutes before, or even more. Being late there costs much more than waiting 10 minutes in a café.

The usual time, “about 30 minutes”, is the same for both trips. What changes your plan is the spread: 25 on a good day, 40 on a bad day. You use a different part of it for each trip, because being late costs a different amount.

Every order in this course so far came from one number per day: the point forecast. Kenji, the stock planner, orders parasols every week. Each order must cover 14 days: one week until the next order, and one week for delivery (a course assumption). On Sunday 8 June 2025, the ETS forecast for the next 14 days added up to 92.4 parasols. Suppose Kenji orders 93: the forecast, rounded up.

In the next 14 days, Larkfield sold 101 parasols. Kenji ran short by 8. Was the forecast bad? Not really. A forecast of 92.4 says “about 92, maybe more, maybe less”. It cannot say how much more.

This is not bad luck. In the lesson notebook, Kenji orders the forecast added up for the 50 products of the course backtest. He does it at each of the 13 cut-offs: 650 orders. Half of them, 50%, were too small. The point forecast sits in the middle, so real sales are above it about half of the time, with every method.

Omar, the product manager, asks: “So should we always order more than the forecast?” Kenji answers: “Yes, but how much more? 5 parasols? 30? I need to know how far demand can go.”

A forecast range gives that answer. Instead of one number per day, the forecaster gives a low end and a high end. It adds a promise: “the real value falls inside this range 8 days out of 10”. This is an 80% prediction interval: an interval (the numbers between two ends) for a value that we predict. You met it as the “80% range” of AutoARIMA in ARIMA and SARIMA.

The two ends have their own name. The low end of an 80% interval is the 10% quantile, written q10: the real value is below it 10% of the time. The high end is the 90% quantile, q90: the real value is below it 90% of the time, and above it 10% of the time. Between q10 and q90, there is 80% of the chance. The median, q50, is the middle: the real value is below it half of the time.

A quantile is a cut point. You met quantiles of data in the mathematics course; a quantile forecast is the same idea for a value that has not happened yet.

So the bus has a forecast range too. “30 minutes” is the point forecast. “25 to 40 minutes” is a range, and 40 minutes is about the q90 of the trip. For the job interview, you plan with the q90, or higher.

A forecast says 20 units a day. Its 80% range goes from 14 to 26. Here are five real days:

Day Real value Inside 14 to 26?
1 18 yes
2 27 no: above the q90
3 21 yes
4 13 no: below the q10
5 24 yes

Three of the five values are inside: 60%. Is the range too narrow? Five days are far too few to tell: a good 80% range often holds 3 or 5 of 5 values, by chance.

With many days, the share of real values inside the range is its coverage. For an honest 80% range, it should be close to 80%. Much lower means the range is too narrow: it promises more safety than it gives. Much higher means it is too wide: it makes us plan for risks that are not there.

ETS, like the other statistical methods of this course, learns how far the real values usually fall from its fitted values. From that spread, statsforecast computes the ends of the range for every day ahead. You ask for the ranges with level=[80, 90]. The forecast then has extra columns: AutoETS-lo-80 and AutoETS-hi-80 (the q10 and the q90), and AutoETS-lo-90 and AutoETS-hi-90 (the q5 and the q95).

from statsforecast import StatsForecast
from statsforecast.models import AutoETS
fc = StatsForecast(models=[AutoETS(season_length=7)], freq="D").forecast(df=train, h=14, level=[80, 90])

The range gets wider for days further ahead, because the errors of the days add up, as ARIMA and SARIMA showed. So a chart of the ranges opens like a fan: a fan chart. ETS’s range widens in the same way.

The parasol's 80% range from the cut-off of 22 June 2025 (ETS). It is 10.7 parasols wide for day 1 and 18.7 parasols wide for day 14.
Text description of the chart

The chart has one line with 14 dots, one for each day ahead after the cut-off of 22 June 2025. The vertical axis is the width of the 80% range: the q90 minus the q10, in parasols. The line rises steadily from 10.7 on day 1 to 18.7 on day 14. It rises a little less each day.

Larkfield: the parasol before the heat wave

Section titled “Larkfield: the parasol before the heat wave”

Priya, the data scientist, makes the range for the parasol (P025) from the cut-off of Sunday 22 June 2025. ETS sees the sales up to that day. The heat wave starts on Monday 30 June, in the second week of the forecast. The chart shows the last four weeks of history, the forecast, its two ranges, and the real sales.

  • history
  • forecast
  • 80% range (darker), 90% range
  • real
The parasol, 14 days ahead from the cut-off of 22 June 2025 (ETS with its 80% and 90% ranges). The real sales are inside the 90% range on 11 of 14 days. On Sunday 6 July, 48 parasols sold, far above the 90% range.
Text description of the chart

The chart shows parasols sold per day, from 26 May to 6 July 2025. A grey line shows the history up to the cut-off on 22 June, between 3 and 20 a day. After the cut-off, a blue line shows the ETS forecast, between about 11 and 16 a day. Around it are two bands: the darker 80% range and the lighter 90% range. The 90% range widens from about 5 to 18 on the first day to about 1 to 24 on the last. Dots show the real sales: between 4 and 13 in the first week. After the heat wave starts on 30 June (a dashed line), they are 13, 11, 18, 16, 17, 24 and 48 on Sunday 6 July. That last dot is orange: it is above the 90% range, whose top is 24.5 that day.

The forecast continues the recent level with the weekly pattern, and it knows nothing about the heat wave. Its range is wide, because the parasol’s sales are noisy.

Now look at the real sales. 9 of the 14 real values are inside the 80% range, and 11 of 14 are inside the 90% range. Only one day is above the 90% range: Sunday 6 July, with 48 parasols sold. The two other days outside the 90% range are below it, in the first week.

Two lessons come from this chart. First, the range did its job on most days, even in the heat wave week. The middle was too low all week, but the high end was close to the real sales until the last day. Second, a 90% range is wrong about 1 day in 10, and the day it misses is often the day that matters most. A range does not predict a heat wave. It says how large the usual surprises are.

One product and 14 days are an example, not a test. In the notebook, Priya runs ETS with ranges over the whole course backtest: 50 products, 13 cut-offs, 14 days ahead, so 9,100 forecast days. Then she counts:

Range Promise Coverage (ETS, measured)
80% (q10 to q90) 80% of the real values inside 83.5%
90% (q5 to q95) 90% of the real values inside 90.5%

ETS keeps its promise, and a little more: its 80% range is a little too wide, which is the safe side. Not every method does as well. Chronos-Bolt had the best point forecasts of the course (Deep and foundation models). But its 80% range holds only 78.9% of the real values. It predicts only the quantiles 0.1 to 0.9, so it has no real 90% range: its “90% range” is its 80% range. The next lesson compares the ranges of every method.

Why the 90% range matters more than the point forecast for stock

Section titled “Why the 90% range matters more than the point forecast for stock”

Back to Kenji’s question: how much more than the forecast? A range answers it in his own words: “If I order up to the q90, I run short only 1 time in 10.” The point forecast carries no information about the spread. Two products can have the same forecast of 92 and very different ranges: one from 85 to 99, another from 60 to 125. They need very different orders.

For stock, the high end matters most. A shop that runs out loses the sale and sometimes the customer. A shop with a few extra items sells them later, at a discount.

The diagram shows where this module goes. The range is the new input, and the costs of a lost sale and of a left-over item choose which quantile to order. From forecast to order and roster makes that choice.

A flow from left to right. History leads to two boxes: Forecast, one number a day (the middle), and Range, q10 to q90, a quantile for each level. A dashed arrow from Forecast goes to a pale box, Order the forecast, labelled: no risk, half the orders run short. From Range an arrow goes to Costs, a lost sale and a left-over item, then to Choose the quantile, then to Order and roster.
The point forecast alone leads to orders that run short half of the time. The range, with the costs, leads to a chosen quantile for the order and the roster.
Text description of the diagram

The diagram starts with a box called History: the sales up to the cut-off. Two arrows leave it. The first goes to Forecast, one number a day, and on to a pale, dashed box: Order the forecast. It takes no risk into account, so half of the orders run short. The second arrow goes to Range: the quantiles from q10 to q90. From Range, arrows go to Costs (a lost sale and a left-over item), to Choose the quantile, and to Order and roster.

When to use a range, when not, and what it costs

Section titled “When to use a range, when not, and what it costs”

Use a range whenever a decision depends on the forecast: a stock order, a roster, a budget. For the statistical methods of this course, the range costs nothing extra. You pass level=[80, 90], and the fit time does not change.

Other methods need more work. A regression or LightGBM model gives one number. To get a range, you can train extra models for the quantiles, or build the range from past errors, as the next lesson shows. Chronos-Bolt gives quantiles directly, but only from 0.1 to 0.9.

A range describes the usual surprises that the history contains. It cannot know about an event that the history never showed, such as a heat wave stronger than any before. And it is only as honest as the model’s view of its own errors. So check its coverage on a backtest before you trust it.

Guided practice: the parasol’s range in the notebook

Section titled “Guided practice: the parasol’s range in the notebook”
  1. Open the notebook from the box Do this lesson in a notebook at the top of this page.
  2. Run the setup cells. The data cell ends with the Ready: line.
  3. In section 1, predict how many of the five real values fall inside 14 to 26. Run the cell. You see 3 of 5 inside.
  4. In section 2, predict whether the heat wave stays inside the 90% range. Run the cell. You see the fan chart and days above the 90% range: ['Sun 06 Jul'].
  5. In section 3, run the backtest cell. It takes about a minute. You see coverage of the 80% range: 0.835 and a pinball value of 0.11 (the next lesson explains this score).
  6. In the Your turn cell, compute coverage_90, as described below.
  7. Run the check cell. It prints Check passed.
  8. In section 4, run the last cell. You see orders too small: 50% of 650 orders.

Change one thing: the range. In the notebook, the backtest result fc_range has the columns q5 and q95, the ends of the 90% range. Before you run it, predict: will the coverage be closer to 90% than the 80% range was to 80%? Compute the share of real values inside the 90% range with bt.coverage, and put it in coverage_90.

Show the answer
coverage_90 = bt.coverage(fc_range, "q5", "q95")
print(coverage_90)
0.905

The 90% range holds 90.5% of the real values: very close to its promise. The 80% range held 83.5%, so it was a little wider than it needed to be. The ETS ranges are honest at both levels, and a little cautious at 80%.

Common mistake

Adding up the daily high ends

Symptom: Tomás wants a safe 14-day order for the parasol from the cut-off of 8 June 2025. He takes the q90 of each of the 14 days and adds them up. The sum is 196.9, so he orders 197 parasols. The forecast added up is 92.4, so his “safe” order is more than twice the forecast. In the 14 days, Larkfield sold 101. 96 parasols are left over. At 25% of the price for each left-over parasol (the course’s assumption), that costs about 1,920 dollars.

Cause: the q90 of each day is a high day: one day in ten is higher. Tomás’s sum assumes that all 14 days are high at the same time. That almost never happens: high and low days partly cancel in the total.

Fix: add up the daily forecasts to get the middle of the total (92.4). For the spread, add up the daily variances (the squared standard deviations) and take the square root: 20.4 parasols. This treats the days as independent, as the course’s forecasting job does. On a normal (bell-shaped) curve, the q90 is 1.28 standard deviations above the middle. So the q90 of the 14-day total is 92.4 + 1.28 × 20.4 = 118.5, not 196.9. Quantiles do not add up; means and variances do.

Consequence in AI work: a “safe” plan made from the high ends of its parts is far too large. The extra stock or staff costs real money. The same mistake appears in budgets, cloud capacity and project plans. The honest limit: days are not fully independent. A heat wave lifts all of them together, so the real range of the total is a little wider than 20.4.

Check your understanding

In your own words: what does an 80% prediction interval of 14 to 26 promise? What are 14 and 26 called?

Show the answer

It promises that the real value falls between 14 and 26 about 8 times in 10. 14 is the 10% quantile (q10): the real value is below it about 1 time in 10. 26 is the 90% quantile (q90): the real value is above it about 1 time in 10. The promise is about many forecasts, not one: with many days, the share inside (the coverage) should be close to 80%.

Check your understanding

Predict and explain: Kenji orders the forecast added up, for 650 orders. About how many are too small? Why does a better forecasting method not change this much?

Show the answer

About half: in the lesson notebook, 50% of the 650 ETS orders were too small. The point forecast is the middle of the range, so real sales are above it about half of the time, whatever the method. A better method makes the range narrower, so each shortage is smaller. But only an order above the middle, at a higher quantile, makes shortages less frequent.

Check your understanding

Tomás says: 'Chronos-Bolt has the best point forecasts, so its 90% range must be the best too.' What do you check, and what do you find on Larkfield's backtest?

Show the answer

Check the coverage of each range on the backtest, not the point error. Chronos-Bolt’s 80% range holds only 78.9% of the real values: a little too narrow. It predicts only the quantiles from 0.1 to 0.9, so it has no real 90% range. Its “90% range” also holds 78.9%. ETS’s ranges hold 83.5% and 90.5%. A good point forecast does not make a good range: score both.

Recap

  • A point forecast is the middle. Orders at the middle run short about half of the time: 50% of the 650 ETS orders in the course backtest.
  • A prediction interval gives a low end and a high end with a promise. The ends are quantiles: the q10 and the q90 hold an 80% range between them.
  • The range widens for days further ahead, like a fan. The parasol’s 80% range grows from 10.7 parasols wide on day 1 to 18.7 on day 14.
  • Coverage is the share of real values inside the range. ETS: 83.5% for its 80% range and 90.5% for its 90% range. Chronos-Bolt: 78.9% for both, because it has no real 90% level.
  • Quantiles of days do not add up. The q90 of a 14-day total is 118.5 parasols, not the 196.9 of the daily q90 values added up.

You can now make a range and count its coverage. But a range can be too narrow or too wide, and coverage alone cannot say which range is better. Next, you score ranges with the pinball loss, and you test a simple fix for a range that is too narrow.

Next: Score the range