Electricity load for many customers
- Case study 2
- 150 min
- Optional · 1 case study needed for the certificate
Where we are
In eight modules, we forecast Larkfield's series and their ranges. In this case study, we use real open data: the electricity load of many customers. We compare global and per-series models with quantile forecasts, and price buying too little or too much power.
Case study 2 of 3 · Forecasting and Time Series: Predict What Comes NextElectricity load for many customers
1. Forecasts serve decisions
2. Read a time series
3. Baselines and honest testing
4. Statistical forecasters
5. Machine learning forecasters
6. Hard series and many series
7. Uncertainty and the decision
8. Run a forecast in production
What you will be able to do
Take 80 real customers’ meter data to a day-ahead energy purchase with its cost. You will fix the clock-change hours. You will compare one model per customer (MSTL), one global model for all customers (LightGBM) and a pretrained model (Chronos-Bolt) in the course’s backtest, and score their 90% quantiles. Then you buy for all customers together, test the choice once in December, and say where it failed.
The notebook runs every step of this page, with the same code and the same output, plus three tasks with checks. Its setup cells install the packages and download the data (3.7 MB) and the course’s backtest.py. Chronos-Bolt downloads its model once (about 190 MB). The notebook runs in about 4 minutes after the installs.
Before you startWhat to know and to have ready
What you need to know
This case study uses the whole course. These lessons matter most:
- The rolling-origin backtest and
backtest.py, from Test like the future: backtesting, and MASE, WAPE and bias, from Measure the error that matters. - MSTL with a daily and a weekly season, from Several seasons and long series.
- One model for many series, from Forecasting as regression and Gradient boosting with LightGBM.
- Chronos-Bolt, from Deep and foundation models.
- Quantiles, the critical ratio, the pinball loss and coverage, from A forecast is a range, Score the range and From forecast to order and roster.
What you need on your computer
Nothing: the notebook runs in Colab or Kaggle. To run it locally, use a virtual environment with Python 3.12, pandas 2.3.3, statsforecast 2.1.1, mlforecast 1.1.0, LightGBM 4.7.0, scikit-learn 1.9.1, chronos-forecasting 2.3.2 and PyTorch 2.15.0 (CPU). No GPU. The setup cell makes a new folder electricity-forecasting. To start again, delete that folder.
Tested on macOS with Python 3.12, in Jupyter from a fresh start. Colab, Kaggle, Windows and Linux are not tested.
The situation
Section titled “The situation”A small electricity supplier in Portugal sells power to 80 customers. The data does not say which customers are businesses and which are homes. The supplier does not make electricity. It buys it on the wholesale market, where suppliers buy power in bulk, and sells it on to its customers.
Most of the energy is bought the day before. Every morning, before the market closes at noon, the supplier says how much energy it wants for each hour of tomorrow. Tomorrow, the customers use what they need, and the supplier must settle the difference:
- A kWh short: the customers used more than the supplier bought. It must buy the missing energy at the last minute, at a much higher price.
- A kWh too much: the supplier bought energy that nobody used. It sells it back, at a lower price than it paid.
Inês plans the purchase. Rui, a data analyst, builds the forecasts. We invented both people and the supplier for this case study. The customers and their electricity use are real.
Inês asks Rui: “How much energy should I buy for each hour of tomorrow? And how much does a bad forecast cost us?”
What you will build
Section titled “What you will build”You will build day-ahead forecasts for 80 customers, every hour, with a 90% quantile. They end in an energy purchase for each hour of tomorrow, with its cost, tested once on six December days that no step has seen.
The data
Section titled “The data”The data is ElectricityLoadDiagrams20112014 from the UCI Machine Learning Repository. It has the electricity use of 370 customers in Portugal, every 15 minutes, from 2011 to 2014.
The original file is 678 MB, too large for a notebook. So the course keeps a fixed subset next to the notebook (3.7 MB). It has the customers with almost no zero hours (292 of 370), and 80 of them drawn at random. One row is one hour. Each customer column, MT_001 to MT_368, holds that customer’s use in the hour, in kWh (kilowatt-hours). A 1,000-watt heater that runs for one hour uses 1 kWh.
We also looked at two other datasets. The Monash “Electricity Hourly” set is a third party’s copy of this data. The London smart-meter data has single homes, which are very noisy.
First checks
Section titled “First checks”wide = pd.read_csv("electricity_hourly_80.csv.gz", index_col="hour", parse_dates=True)size = wide.mean() # each customer's mean kWh per hourprint(wide.shape, wide.index.min(), "to", wide.index.max())print(f"zero hours: {(wide == 0).sum().sum():,} of {wide.size:,}")print("mean kWh per hour: smallest", round(size.min(), 1), "| median", round(size.median(), 1), "| largest", round(size.max(), 1))(17520, 80) 2013-01-01 00:00:00 to 2014-12-31 23:00:00zero hours: 51 of 1,401,600mean kWh per hour: smallest 5.2 | median 130.6 | largest 6311.4What this tells us. There are two years of hours with almost no gaps. The customers are very different in size. The largest uses about 1,200 times as much as the smallest, and the 5 largest customers use 45.7% of all the energy. One model for all customers must deal with that. A WAPE over all customers mostly measures the large ones.
The clock changes
Section titled “The clock changes”Portugal moves its clocks one hour forward on the last Sunday of March and one hour back on the last Sunday of October. The source keeps a 24-hour day and says what that does to the data. Here is the total of all 80 customers at 00:00, 01:00 and 02:00 on those days:
2013-03-31 [14005, 4880, 15556]2013-10-27 [22321, 26885, 14715]2014-03-30 [12933, 3757, 14072]2014-10-26 [22677, 28047, 15483]What this tells us. In March, the hour 01:00 is mostly zero: that hour did not exist. In October, it holds two hours of use. Both are errors of the clock, not of the customers. Rui fixes them in the simplest way: in March, the mean of 00:00 and 02:00; in October, half of the value. After the fix, 26 October 2014 reads 22,677, 14,024 and 15,483. The failure case near the end of the page shows what happens without the fix.
Two seasons, and one is weak
Section titled “Two seasons, and one is weak”Think of a street at night: most lights are off, and the street is dark at the same hours every day. Electricity use is the same. It has a daily season: low at night, high in the morning and the evening. Offices add a weekly season: less use at the weekend.
The autocorrelation says how much the load at one hour looks like the load some hours before (1 means exactly alike). For the total, it is 0.954 for 1 hour before, 0.982 for 24 hours and 0.97 for 168 hours (one week). So the same hour yesterday and the same hour last week both tell a lot about the next hour.
- All 80 customers
- MT_368: weekdays 2.61 × the weekend
- MT_083: weekdays 1.00 × the weekend
Text description of the chart
The chart shows one week in May 2014, hour by hour. Each line is divided by its own mean, so 1 means an average hour. The total of all 80 customers goes from about 0.55 at night to about 1.3 in the day, every day. Its weekend looks almost like its weekdays. Customer MT_368 goes from about 0.3 at night to about 2.6 on weekday afternoons. On Saturday and Sunday, it stays below about 0.75. Customer MT_083 goes from about 0.6 to 1.6 every day, and its weekend looks like its weekdays.
What this tells us. The daily season is strong everywhere. The weekly season is strong only for some customers: just 8 of the 80 use more than 1.2 times as much on weekdays as at the weekend. In the total, the weekday hour is 26,891 kWh and the weekend hour 26,545. So a model that learns from all customers together will see many different weekly shapes.
Frame the forecast
Section titled “Frame the forecast”Frame the forecast asks a few questions before any model. Rui’s answers:
- What: kWh used per customer and hour, after the clock fix.
- Unit of time: one hour.
- Level: per customer first, because the supplier’s contracts and bills are per customer. Later, the total.
- Decision and costs (assumptions): a kWh short costs €0.09 more, and a kWh too much loses €0.01. These are our numbers, not market prices.
- Horizon: tomorrow’s 24 hours, forecast at 11:00 today.
The forecast is made before the market closes
Section titled “The forecast is made before the market closes”A day-ahead forecast is not “the next 24 hours”. The supplier buys at noon, so Rui’s forecast runs at 11:00. Tomorrow’s first hour, 00:00, is 13 hours after the forecast. Tomorrow’s last hour, 23:00, is 36 hours after it.
Text description of the diagram
A time line from today 00:00 to tomorrow 23:59. A blue box marks today 11:00, the forecast. An orange box marks today 12:00, when the market closes and tomorrow’s energy is bought. A green band covers tomorrow from 00:00 to 23:00, steps 13 to 36 of the forecast, 13 to 36 hours ahead. A note below says that every input must be known at 11:00 today. The same hour two days before (lag 48 hours) and the same hour last week (lag 168 hours) are known. The same hour yesterday (lag 24 hours) is not known for tomorrow’s evening.
This changes the model, not only the score. A model that uses “the same hour yesterday” (lag 24) would use a value that does not exist yet for tomorrow’s evening. So Rui’s LightGBM uses only lags of 48 hours or more, as in the leakage rule of Forecasting as regression.
How much to buy
Section titled “How much to buy”A kWh short costs nine times as much as a kWh too much. So Inês should buy more than the expected use. The newsvendor idea from From forecast to order and roster gives the level. Buy the quantile at the critical ratio, 0.09 ÷ (0.09 + 0.01) = 0.9. That is the 90% quantile, q90. The use stays below it in 90% of the hours.
So the forecast that matters is q90. Two measures from Score the range say how good it is:
- The pinball loss at 0.9: each kWh of real use above the q90 costs 0.9, and each kWh of q90 above the real use costs 0.1. That is the cost rule of the purchase, scaled down ten times. We divide it by the mean load, so that we can compare.
- Coverage: the share of hours below q90. It should be 0.9. Above means the q90 is too high, and the supplier buys too much. Below means it is too low.
Baseline and alternatives in one backtest
Section titled “Baseline and alternatives in one backtest”The backtest
Section titled “The backtest”Rui uses the course’s backtest.py, with hours. There are 12 origins, 26 days apart, from 10 February to 23 November 2014, at 11:00. With 26 days, the weekday of “tomorrow” changes from one origin to the next. Each forecast covers 36 hours, from 12:00 today, and only tomorrow’s 24 hours are scored. December is kept aside for the final test.
Four methods, each with its reason:
- Same hour last week: the baseline. Its range comes from its own errors.
- MSTL with a daily (24) and a weekly (168) season, as in Several seasons and long series. It is a per-series method: 80 separate models. It is fitted on the last 4 weeks, to keep the notebook fast.
- LightGBM, global: one model for all 80 customers, plus two quantile models for q10 and q90. Each customer is first scaled to its own mean and spread, so that the small customers count as much as the large ones. It is fitted on 26 weeks, and again at every fourth origin.
- Chronos-Bolt small: the pretrained model of Deep and foundation models, with no training. It reads up to 2,048 hours of each customer and returns q10, q50 and q90. It runs in its own Python process, because PyTorch and LightGBM in one process crashed on macOS.
def run_all(df, origins, tag): out = {"same hour last week": bt.backtest(df, stats(SeasonalNaive(season_length=168), "SeasonalNaive"), h=H, origins=origins), "MSTL (24, 168)": bt.backtest(df, stats(MSTL(season_length=[24, 168]), "MSTL"), h=H, origins=origins), "LightGBM, global": bt.backtest(df, GlobalLightGBM(), h=H, origins=origins, refit_every=4)} chronos, seconds = run_chronos(df, origins, tag) out["Chronos-Bolt small"] = bt.backtest(df, chronos, h=H, origins=origins) ...The cost uses the amount bought for each customer and hour. In this first table, each customer is settled alone. Real markets settle the supplier’s whole portfolio. A later section shows what that changes.
method MASE WAPE % bias % pinball q90 below q90 80% range holds fit s cost at forecast cost at q90same hour last week 0.942 7.8 1.6 0.0217 0.926 0.827 0.34 2117 1400 MSTL (24, 168) 0.855 7.5 1.8 0.0199 0.933 0.85 72.93 1952 1286 LightGBM, global 0.743 6.1 -0.1 0.0157 0.885 0.773 20.54 1985 1015 Chronos-Bolt small 0.657 5.5 0.2 0.0142 0.913 0.823 5.08 1704 915The costs are in euros per day for the 80 customers. Fit times are seconds for all 12 origins, and they depend on the computer.
Text description of the chart
A column chart with four columns. Same hour last week: €1,400 a day. MSTL: €1,286. LightGBM: €1,015. Chronos-Bolt: €915, the lowest, highlighted.
One measure for the choice
Section titled “One measure for the choice”Which column decides? The cost at q90, because it is the decision. Here, the columns agree more than in Larkfield’s backtest table:
- Chronos-Bolt is the best on every measure: MASE 0.657, pinball 0.0142 and €915 a day. Its q90 holds 91.3% of the hours, close to the 90% it should. That is the same honest surprise as in Deep and foundation models, now on real data.
- The global LightGBM is second (€1,015). Its q90 is a little too low: only 88.5% of the hours are below it.
- MSTL’s range is too wide (93.3% below q90), so it buys too much. It also took about 73 seconds, against 5 for Chronos.
- Buying the forecast is expensive. Chronos’s median costs €1,704 a day, almost twice its q90. With these costs, a safety margin pays.
Per customer and by size
Section titled “Per customer and by size”Is one method best for every customer? Rui counts, per customer, which method has the lowest pinball loss at 0.9:
lowest pinball at 0.9, customers per method: {'Chronos-Bolt small': 62, 'LightGBM, global': 17, 'MSTL (24, 168)': 1}same hour last week cost small 168, large 1232 | below q90: small 0.920, large 0.932MSTL (24, 168) cost small 151, large 1135 | below q90: small 0.926, large 0.939LightGBM, global cost small 130, large 885 | below q90: small 0.873, large 0.897Chronos-Bolt small cost small 119, large 797 | below q90: small 0.905, large 0.920What this tells us. Chronos is best for 62 customers, LightGBM for 17 and MSTL for only 1. The 40 largest customers carry about 87% of the cost, so they decide the choice. And the global LightGBM’s q90 is too low mostly for the small customers (87.3% below q90): one model shares its quantile between customers of very different shapes.
- Real use
- Chronos: median (dashed) and q90
- MSTL q90
Text description of the chart
The chart shows customer MT_046 on Monday 24 November 2014, hour by hour. The real use is about 140 kWh at midnight and about 100 kWh in the early morning. It has a short peak of about 150 kWh at noon and its highest hour, about 220 kWh, at 20:00. The Chronos median follows the shape a little below. The Chronos q90 lies above the real use in 20 of the 24 hours. The MSTL q90 is similar, and a little higher in the afternoon, up to about 230 kWh.
One purchase for all customers
Section titled “One purchase for all customers”So far, each customer was settled alone. A real supplier buys for its portfolio: the market settles the total. When one customer uses more than forecast and another less, the two cancel.
Think of a family that packs for a trip. If each of four people packs a spare towel “in case”, the family carries four. One or two would cover the whole family, because not everyone needs a spare at once. Each customer’s q90 is like a spare towel: it has its own safety margin. Add 80 of them, and the margins pile up. A quantile of a sum is not the sum of the quantiles.
Rui compares three ways to buy, all with Chronos-Bolt:
each customer alone: 915 euros a daysum of the 80 q90: 664 euros a day, below q90 0.997the total's own q90: 393 euros a day, below q90 0.934Text description of the chart
A column chart with three columns. Each customer bought and settled alone: €915 a day. The sum of the 80 customers’ q90, settled as one total: €664. The q90 of the total’s own forecast: €393, the lowest, highlighted.
What this tells us. Adding the 80 q90 values buys too much: the total stays below it in 99.7% of the hours, not 90%. Forecasting the total itself gives a q90 close to honest (93.4% of the hours below it) and the lowest cost, €393 a day. MSTL and LightGBM show the same pattern: for the total’s own q90, MSTL costs €510 and LightGBM €528 a day. So Chronos stays the choice for the total too.
Why still forecast each customer? For the work that is per customer: contracts, bills, a new customer’s first weeks, a meter that suddenly reads zero. For the purchase, forecast the total.
The final test, once
Section titled “The final test, once”Rui has chosen: Chronos-Bolt, the total’s own q90, bought at 11:00. Before the final test, he also runs all four methods per customer on six December origins that no step has seen: 4 to 29 December 2014, for the days 5, 10, 15, 20, 25 and 30 December. He changes nothing after.
method MASE WAPE % bias % pinball q90 below q90 80% range holds fit s cost at forecast cost at q90same hour last week 1.521 14.7 7.7 0.0315 0.89 0.731 0.15 2415 1767 MSTL (24, 168) 1.524 14.4 8.4 0.0293 0.927 0.769 34.14 2148 1642 LightGBM, global 1.435 14.1 8.0 0.026 0.857 0.652 11.81 2141 1459 Chronos-Bolt small 1.331 13.3 7.7 0.0242 0.897 0.7 2.17 1993 1354Text description of the chart
A column chart with six columns, the cost of buying Chronos’s q90 for each test day. 5 December: €747. 10 December: €890. 15 December: €861. 20 December: €1,073. 25 December: €3,454, highlighted. 30 December: €1,097.
What this tells us. The choice holds: Chronos-Bolt is again the cheapest (€1,354 a day). But every method is worse than in the comparison. Every MASE is above 1.3, and every method is 7.7% to 8.4% too high. The worst day by far is Christmas Day: it alone is 43% of Chronos’s cost over the six test days. Many businesses close, use falls, and no method here knows the calendar. All four methods cost between €3,454 and €3,707 on that one day.
No comparison test day was a public holiday, so the backtest could not show this. Rui reports the result as it is. A fix belongs in the next cycle: a holiday calendar, as in Calendars, holidays and events, tested on new data.
The purchase for 31 December, on new input
Section titled “The purchase for 31 December, on new input”The last step runs the chosen pipeline at 11:00 on 30 December 2014, for the total. It writes the amount to buy for each hour of 31 December. The data has the real use of that day, so the result can be scored.
buy 633.8 MWh (median forecast 583.8) | real 538.5 MWh | hours short: 0 | cost 953 euros ds y yhat q902014-12-31 00:00:00 17014.0 18319.0 19852.02014-12-31 08:00:00 22395.0 21984.0 23167.02014-12-31 12:00:00 31996.0 32291.0 34548.02014-12-31 19:00:00 21966.0 29551.0 32585.0What this tells us. The pipeline works end to end on a new day. (1 MWh is 1,000 kWh.) No hour was short. But New Year’s Eve used 538.5 MWh, less than even the median forecast (583.8 MWh). So most of the €953 is energy sold back. Look at 19:00: 21,966 kWh used, against 29,551 forecast. On a normal evening, use is high. On this evening, many places had closed. Like Christmas, it is a calendar day that the model does not know. Adding up the 80 customers’ q90 would have bought 673.5 MWh, for €1,350.
Common mistake
The clock change, one week later
Symptom: on Sunday 2 November 2014, the “same hour last week” forecast for 01:00 is 28,047 kWh for all customers. The real use is 15,464 kWh: the forecast is 81% too high.
Cause: the forecast copies 01:00 on 26 October, the night the clocks went back. That hour holds two hours of use. LightGBM’s lags of 168 and 336 hours copy the same wrong value into its inputs, one and two weeks later.
Fix: fix the clock-change hours before any model, as in Clean without hiding demand. With the fix, the copied value is 14,024 kWh.
Consequence in AI work: time zones and clock changes break data quietly. Count the hours per day, and check the two clock-change nights every year.
Recommendation
Section titled “Recommendation”Rui writes to Inês in her words:
- Buy the 90% quantile of the total, from Chronos-Bolt, every day at 11:00. In the comparison, it cost €393 a day, against €915 if each customer were bought and settled alone.
- Do not add up the customers’ safety margins. The total stayed below the sum of the 80 q90 values in 99.7% of the hours: far too much.
- Watch the calendar. On Christmas Day, every method cost about three times a normal day, and New Year’s Eve was much lower than forecast. Until a holiday calendar is tested, Inês should lower the purchase for public holidays by hand, and Rui should log those changes.
- Keep a fallback. If Chronos cannot run, use the global LightGBM (€1,015 a day, each customer settled alone; €528 for the total), not MSTL.
- Check the cost ratio. The 0.9 quantile comes from our assumed costs. If a kWh short costs 19 times a kWh too much, the ratio is 0.95. Chronos-Bolt cannot give a 0.95 quantile: it predicts only 0.1 to 0.9. LightGBM would need a q95 model and a new backtest.
And what not to use it for:
- Not for new customers without a few weeks of history. Chronos needs a history to read. See New products with no history.
- Not for the network. A power line or a transformer needs a forecast of its own peak, not of the supplier’s purchase.
- Not as a promise to a customer. It is a forecast with a known error (WAPE 5.5% per customer in the comparison), not a contract.
Insights and limits
Section titled “Insights and limits”- The foundation model won again. As at Larkfield, Chronos-Bolt had the best accuracy and was the fastest, with no training. Here, it also had the lowest cost. On electricity, many public series look like this data, which may help it. Its costs are real: its own process, 47.7 million parameters, a 190 MB download, and no quantile above 0.9.
- Global beats per-series here. The global LightGBM was cheaper than 80 MSTL models and more than three times faster. But one model shares its quantiles, and its q90 was too low for small customers.
- The portfolio matters more than the model. Going from Chronos per customer to Chronos for the total saved €522 a day. Going from MSTL to Chronos per customer saved €371 a day.
- No calendar. No method knows public holidays. Christmas Day was the most expensive day of the test.
- Simplifications. The market rules here are simple: one price for a kWh short and one for a kWh too much, the same every hour. Real prices change every hour. MSTL saw only 4 weeks, to keep the notebook fast; a longer window may help it.
- Assumed costs, old data. €0.09 and €0.01 are our numbers. The data is from Portugal in 2013 and 2014, before most solar panels, heat pumps and electric cars.
The course’s Watch the forecast shows how a bias alarm would catch the December problem in production.
Responsible use
Section titled “Responsible use”- The data is about real customers. The customer names are codes, and the data has no personal information. Do not try to find out who a customer is. Hourly use can show when a home is empty.
- Credit the source. CC BY 4.0 allows you to share and change the data, with the attribution text above and a note of your changes.
- A purchase affects other people. When a supplier buys too little, the grid operator must find the missing power at short notice, and other people share that cost. State the error and the coverage next to every forecast you hand over.
- Label the assumptions. The costs, the clock fix and the market rules are choices. Say so wherever a number depends on them.
Make it a portfolio piece
Section titled “Make it a portfolio piece”This case study shows an employer that you can forecast many series hour by hour. You turn a quantile into an amount to buy, with its cost. It also shows that you found the bigger saving outside the model, and that you report a test that went worse. Make it yours with one change, for example other costs, a holiday calendar or more customers. Then make it public and easy to read in five minutes.
What to put in the repository:
README.md, the page that people read first (template below).- The notebook, run from top to bottom, with its outputs.
checks.md: the clock-change hours before and after the fix, and the zero hours.backtest_table.csv: the four methods with MASE, WAPE, bias, pinball at 0.9, coverage and cost; andfinal_test.csv, kept apart.- The data source and licence, with the attribution text above. CC BY 4.0 lets you share the subset with credit, but a link and a checksum are enough.
A README template:
# Day-ahead electricity purchase for 80 customers
## ProblemA supplier buys tomorrow's energy at 11:00 today. A kWh short costs 0.09 euros extra,a kWh too much 0.01 (assumptions), so it buys the 90% quantile.
## Data and licenceUCI "ElectricityLoadDiagrams20112014" (Trindade), 80 customers, kWh per hour, 2013-2014,CC BY 4.0, https://doi.org/10.24432/C58C86. Clock-change hours fixed.
## ApproachBacktest: 12 origins Feb-Nov 2014 at 11:00, tomorrow's 24 hours (13-36 hours ahead).Same hour last week, MSTL (24, 168) per customer, one global LightGBM with quantilemodels, Chronos-Bolt small. Scored with pinball loss at 0.9, coverage and cost.
## ResultsChronos-Bolt: lowest cost, each customer settled alone (915 euros a day for all 80 vs1,015 LightGBM), q90 holds 91.3%.Buying the total's own q90: 393 euros a day; the sum of 80 q90s: 664.Final test (December): Chronos still cheapest; Christmas Day three times a normal day.
## LimitsNo holiday calendar; simple market rules; assumed costs; data from 2013-2014.How to describe it in an interview, in three sentences: “I built day-ahead quantile forecasts for 80 real electricity customers. I chose a pretrained model over a global LightGBM and per-customer MSTL by the cost of the energy bought, in a rolling backtest. The largest saving came from forecasting the portfolio instead of adding up the customers’ 90% quantiles, and the final test showed that holidays need a calendar.”
Go further:
- Add a holiday flag to the global LightGBM and backtest it on the 2013 holidays.
- Fit MSTL on 8 weeks instead of 4 and see if it closes the gap.
- Compare with the other case studies, Bike-share demand and fleet planning and Retail demand with a hierarchy: where did the cost ratio move the decision most?
Your turn: the price of buying the median
Section titled “Your turn: the price of buying the median”Before you run it, predict: how much more does it cost to buy Chronos’s median instead of its q90? In the notebook, section 4, compute the cost per day of buying the median (yhat). Use fcs["Chronos-Bolt small"], and put the cost in median_cost.
Show the answer
median_cost = round(cap(tomorrow(fcs["Chronos-Bolt small"]), "yhat"))print(median_cost)1704Buying the median costs €1,704 a day, against €915 for q90: 86% more. The median is short in about half the hours, and each kWh short costs nine times a kWh too much.
Knowledge check
Section titled “Knowledge check”Check your understanding
Rui's forecast runs at 11:00 for tomorrow's 24 hours. Why does his LightGBM use lags of 48 hours and more, and not the same hour yesterday (lag 24)?
Show the answer
At 11:00 today, tomorrow’s hours are 13 to 36 hours ahead. For tomorrow at 20:00, the same hour yesterday is today at 20:00, which has not happened yet at 11:00. A model that uses lag 24 would use a value that does not exist at forecast time, or its own guess of it. Lag 48 (the same hour two days before) and lag 168 (the same hour last week) are known at 11:00 for every hour of tomorrow. So the backtest matches what the supplier can really do.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
A kWh short costs €0.09 and a kWh too much €0.01. Which quantile should Inês buy, and how do the pinball loss and the coverage tell her that a method's q90 is good?
Show the answer
She should buy the quantile at the critical ratio 0.09 ÷ (0.09 + 0.01) = 0.9, the 90% quantile. The pinball loss at 0.9 charges 0.9 for each kWh above the q90 and 0.1 for each kWh below it, so it ranks methods by the same cost rule as the purchase: lower is better. The coverage checks the level: about 90% of the hours should be below the q90. Chronos-Bolt had the lowest pinball loss and 91.3% of hours below q90. MSTL’s 93.3% meant that its q90 was too high. LightGBM’s 88.5% meant that its q90 was too low.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Adding up the 80 customers' q90 values cost €664 a day, and the total's own q90 cost €393. Explain why the sum of the quantiles is too much.
Show the answer
Each customer’s q90 includes a safety margin for that customer’s own high hours. The customers do not all have a high hour at the same time, so in the total, one customer’s extra use is often balanced by another’s lower use. The total therefore varies less, compared with its size, than each customer does. When you add up 80 margins, you prepare for all 80 customers being high at once. That almost never happens: the total stayed below the sum in 99.7% of the hours, not 90%. A forecast of the total itself gives an honest q90 and buys less.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- You fixed the clock-change hours: in March the hour 01:00 is missing, in October it holds two hours.
- You framed a day-ahead forecast. It runs at 11:00, covers tomorrow’s 24 hours (13 to 36 hours ahead) and uses lags of 48 hours or more.
- With a kWh short at nine times a kWh too much, the purchase is the 90% quantile. The pinball loss at 0.9 and the coverage scored it.
- Chronos-Bolt beat the global LightGBM and the per-customer MSTL: €915 a day with each customer settled alone, 91.3% of hours below its q90.
- Buying the total’s own q90 cost €393 a day, less than the sum of the customers’ q90 (€664). The final test held the choice, and Christmas Day showed the missing calendar.
Next, the third case study: an online shop’s sales by country and product, with slow sellers and a stock order.