The forecasting job
- Lesson 25
- 45 min
- Module 8: Run a forecast in production
Where we are
We can now turn a forecast into an order and a roster in a notebook. A notebook does not run itself every Monday. In this lesson, we run a small forecasting job from the command line that gets the data, checks it, forecasts and writes the plan. Think first of a grocery order that you place online every week.
Lesson 25 of 27 · Forecasting and Time Series: Predict What Comes NextThe forecasting job
1. Forecasts serve decisions
2. Read a time series
3. Baselines and honest testing
4. Statistical forecasters
5. Machine learning forecasters
6. Hard series and many series
7. Uncertainty and the decision
8. Run a forecast in production
What you will be able to do
Run the forecast every Monday as a small job that nobody has to watch. It gets the data, checks it, forecasts, and writes and keeps the plan. A second run of the same week changes nothing.
Before you startWhat to know and to have ready
What you need to know
- The stock order at the 58% quantile and the roster at the 72% quantile, from From forecast to order and roster.
- Exponential smoothing (ETS), from Exponential smoothing, and the bottom-up totals, from Hierarchies that must add up.
- How the job fills a missing day and a stock-out, from Clean without hiding demand.
- A terminal:
cd, a virtual environment andpip install. Python for Practical AI Engineering shows them.
What you need on your computer
- Python 3.12, a terminal and VS Code (or another editor). Git is useful, but you can download a ZIP file instead.
- About 400 MB of free disk space for the virtual environment. No account, no key and no money.
Tested with Python 3.12, pandas 2.3.3, NumPy 2.4.6 and statsforecast 2.1.1 on macOS. The Windows (PowerShell) commands are written but not tested.
A grocery order that you place every week
Section titled “A grocery order that you place every week”Think of a family that orders its groceries online every Monday. The app follows the same steps each time: it checks the basket, works out the total, sends the order and keeps a receipt.
Two details make this app safe to use. First, if you press Order twice by mistake, you do not get two vans at your door. The shop sees that it is the same order. Second, the receipts stay. In a month, you can open any week’s receipt and see what you paid and why.
Larkfield needs the same two details. Every Monday, Kenji orders stock and Grace plans the roster from the job’s plan.
From a notebook to a job
Section titled “From a notebook to a job”In Modules 2 to 7, you forecast in notebooks. A notebook is good for learning, but a person must open it, run the cells in order and copy the result. On a busy Monday, someone forgets a cell, or runs an old cell twice.
A job is a program that runs from start to end with no person at the keyboard. It does the same steps in the same order every time. A scheduler (a program that starts other programs at set times) can start it every Monday morning.
Larkfield’s job is a small Python program, forecast_job, that puts together what the notebooks did. For one Monday, it does six steps:
Text description of the diagram
The diagram shows six boxes joined by arrows. Box 1, get the data known on the Monday. Box 2, a decision: is the data safe to use? If not (the feed is late, a row is there twice, a sale is negative), an arrow goes to STOPPED. Nothing is written, and last week’s plan stays in use. If yes, box 3, forecast: the 600 product series for 14 days and the tickets for one week, each with a range. Box 4, reconcile: bottom-up. Box 5, plan: the stock order at the 58% quantile and the roster at the 72% quantile. Box 6, keep: a folder for the Monday with a fingerprint of the inputs. A dotted arrow carries warnings (a missing day filled, many stock-outs, a price change) to the kept plan.
Each step uses a method that the course chose from its backtest table:
- Forecast: ETS with a weekly season for each of the 600 product series (200 products in 3 regions), 14 days ahead. MSTL with a daily and a weekly season for the tickets, one week ahead.
- Reconcile: bottom-up. The job adds the product forecasts up to the regions, the categories and the total, so they always agree.
- Plan: the order is the 58% quantile of each product’s 14-day demand; the roster covers the 72% quantile of each hour’s tickets. Both quantiles come from the course’s assumed costs.
Set up the job
Section titled “Set up the job”The job comes in two snapshots in the labs repository. A snapshot is a copy of the project at one point of the course. start is the job at the end of Module 7, and end-of-m08 at the end of this module. You start from start.
-
Get the files of the course. If you have Git, clone the labs repository once, in
~/projects. If you have the clone already, rungit -C nextia-learning-labs pullinstead.Terminal window cd ~/projectsgit clone https://github.com/Nextia-Intelligence/nextia-learning-labs.gitWithout Git, download the ZIP file of the repository and unzip it in
~/projects. Use its folder name,nextia-learning-labs-main, in the next steps. -
Copy the snapshot
startto a folder of your own,forecast-job:Terminal window cp -R nextia-learning-labs/C35/forecast-job/start forecast-jobcd forecast-jobTerminal window Copy-Item -Recurse nextia-learning-labs\C35\forecast-job\start forecast-jobcd forecast-job -
Make the virtual environment, install the packages and get the data:
Terminal window python3 -m venv .venvsource .venv/bin/activatepython -m pip install -r requirements.txtpython get_data.pyTerminal window python -m venv .venv.venv\Scripts\Activate.ps1python -m pip install -r requirements.txtpython get_data.pyget_data.pydownloads about 2 MB of Larkfield data intodata/, checks each file and ends with:Ready: 10 files in data/, every checksum is right. -
Run the tests:
Terminal window python -m pytest -q14 passed in 17.56sYour time will differ; the first run is slower.
-
Plan one week, the week of Monday 10 March 2025:
Terminal window python -m forecast_job run --week 2025-03-10Week 2025-03-10: 12839 units to order (expected cost 32038 dollars), 397 agent hours (expected cost 3665 dollars), 0 warnings. Plan: plan/2025-03-10It takes about 5 seconds.
Read the line from left to right. The job orders 12,839 units for the next 14 days, for all products and regions. Its expected cost, US$32,038, is the average cost of the units that will be short or left over. It is right only if the forecast range is right. The roster has 397 agent hours for the week (expected cost US$3,665). The data check found no problem.
The job uses only the data known on that Monday: the sales up to Sunday 9 March, and the promotions and prices planned in advance. So you can plan any past Monday as if it were today.
What the job wrote
Section titled “What the job wrote”The folder plan/2025-03-10/ now has four files:
| File | What it holds |
|---|---|
forecast.csv |
One row per product, region and day: the forecast yhat and its range (q5, q10, q90, q95) |
totals.csv |
The forecast added up per region, category and for the whole shop |
orders.csv |
One row per product and region: the 14-day forecast, its 80% range, the order and its expected cost |
roster.csv |
One row per hour: the ticket forecast, its range, the planned tickets and the number of agents |
The roster file shows how a range becomes a decision. Here is Monday, hour by hour:
- Ticket forecast
- 80% range (10% to 90%)
- Agents on the roster × 6 tickets
Text description of the chart
The chart has four lines over the 24 hours of Monday 10 March 2025. The ticket forecast, in blue, is about 2 to 4 tickets an hour at night. It rises from 11 at 07:00 to 33 at 10:00 and 11:00, and falls to about 11 at 17:00. It rises again to about 16 at 19:00 and 20:00, and falls at night. Two dashed grey lines show the 80% range, about 4 to 5 tickets on each side. The orange line is the roster: the agents on duty times 6 tickets. It is 0 outside 07:00 to 21:59, and above the forecast in the open hours. There are 3 agents (18 tickets) at 07:00 and 6 (36 tickets) at 10:00 and 11:00. In the evening, there are 3 or 4.
The orange line sits a little above the forecast, because the job plans for the 72% quantile. An hour one agent short costs more than an extra agent hour.
The problem: a job with no memory
Section titled “The problem: a job with no memory”On Tuesday, Omar asks a question: “What if a left-over item costs us 30% of its price, not 25%?” Tomás tries it at once. He changes LEFT_OVER_SHARE in forecast_job/config.py from 0.25 to 0.30, and runs the same week again:
Week 2025-03-10: 12519 units to order (expected cost 35355 dollars), 397 agent hours (expected cost 3665 dollars), 0 warnings. Plan: plan/2025-03-10The answer is useful: with a higher cost of left-over stock, the job orders 320 units less. But look at the path at the end: the job wrote the new plan into the same folder. Kenji placed his order on Monday for 12,839 units, and the files that explain that order are gone. Nothing warned Tomás.
The same happens with no change of settings. A scheduler often starts a job again after a crash, and Mei sometimes delivers a corrected sales file on Tuesday. Each rerun replaces the Monday plan without a trace. Then nobody can check what the job really planned against what was sold, and the next lesson needs exactly that check.
(Undo Tomás’s change now: set LEFT_OVER_SHARE back to 0.25.)
The idea: same inputs, same plan, every plan kept
Section titled “The idea: same inputs, same plan, every plan kept”The job needs the two details of the grocery app:
- Keep every plan. Each Monday gets its own folder, and the job never deletes one.
- Make a second run harmless. A run is idempotent when running it twice has the same effect as running it once.
But how does the job know that a second run is “the same”? It compares a fingerprint of the inputs: a short code that a hash function computes from data. The same data always gives the same code; a change of one value gives a completely different code. A tiny example with Python’s hashlib:
import hashlibfor text in ["sales: 4, 6, 5", "sales: 4, 6, 5", "sales: 4, 6, 6"]: print(text, "->", hashlib.sha256(text.encode()).hexdigest()[:16])sales: 4, 6, 5 -> 527a118a0b587747sales: 4, 6, 5 -> 527a118a0b587747sales: 4, 6, 6 -> b4900c0c117cecaeIn the third line, one sale changed from 5 to 6, and the code is completely different. The job keeps the first 16 characters.
The job’s fingerprint covers everything that can change the plan. That is the data it could see, every setting in config.py, the planner’s overrides and the job’s version. Then the rule is simple:
Text description of the diagram
The chart starts at run –week, for one Monday. First question: is there a plan for this week? If no, the job plans and keeps the plan in runs/Monday/, with run.json and its fingerprint. If yes: the same fingerprint? If yes, nothing to do, and no file changes. If no: did you pass –force? If no, the job stops with a message. If yes, it replaces the plan and moves the old files to a folder named replaced- and the old fingerprint.
The code is in forecast_job/store.py. This is its heart:
def fingerprint(inputs, overrides=None): h = hashlib.sha256() for df in (inputs.demand, inputs.tickets, inputs.promotions, inputs.price_changes, inputs.products): h.update(pd.util.hash_pandas_object(df, index=False).to_numpy().tobytes()) if overrides is not None and len(overrides): h.update(pd.util.hash_pandas_object(overrides, index=False).to_numpy().tobytes()) settings = {k: v for k, v in vars(config).items() if k.isupper()} h.update(json.dumps(settings, sort_keys=True, default=str).encode()) h.update(__version__.encode()) return h.hexdigest()[:16]hash_pandas_object turns every row of a table into a number, and h.update feeds them into the hash. vars(config) reads every name in capital letters in config.py.
Add this lesson’s files
Section titled “Add this lesson’s files”The snapshot end-of-m08 has the job of this whole module. In this lesson, you copy the code and one test file. You copy the other tests and the planner’s files in the next two lessons.
-
In
forecast-job, with the virtual environment active, copy the new files from the snapshotend-of-m08. Four files are new; three change.Terminal window S=../nextia-learning-labs/C35/forecast-job/end-of-m08for f in forecast_job/store.py forecast_job/monitor.py forecast_job/overrides.py \forecast_job/note.py forecast_job/__main__.py tests/test_job.py README.md; docp "$S/$f" "$f"doneTerminal window $S = "..\nextia-learning-labs\C35\forecast-job\end-of-m08"$files = 'forecast_job\store.py', 'forecast_job\monitor.py', 'forecast_job\overrides.py','forecast_job\note.py', 'forecast_job\__main__.py', 'tests\test_job.py', 'README.md'foreach ($f in $files) { Copy-Item "$S\$f" $f }store.pyis this lesson’s code.__main__.pyalso usesmonitor.py,overrides.pyandnote.py; the next two lessons explain them. -
Run the tests.
test_job.pynow checks four things: a run writes the plan, a second run changes nothing, other inputs need--force, and bad data writes nothing.Terminal window python -m pytest -q17 passed in 3.16s -
Delete the old folder
plan(the job now usesruns). On Windows (PowerShell), useRemove-Item -Recurse plan.Terminal window rm -rf planPlan the same week again, twice:
Terminal window python -m forecast_job run --week 2025-03-10python -m forecast_job run --week 2025-03-10Week 2025-03-10: 12839 units to order (expected cost 32038 dollars), 397 agent hours (expected cost 3665 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-10Week 2025-03-10 is already planned with the same inputs (244b2032b74c7443). Nothing to do.The second run saw the same fingerprint and did nothing.
-
Look at the new folder:
Terminal window ls runs/2025-03-10checks.txt forecast.csv orders.csv roster.csv run.json totals.csvchecks.txtholds the data check’s warnings (no problems).run.jsonholds the fingerprint and the summary:{"week": "2025-03-10","fingerprint": "244b2032b74c7443","job_version": "1.0.0","summary": {"order_units": 12839,"order_value_usd": 596273,"order_expected_cost_usd": 32038,"agent_hours": 397,"roster_expected_cost_usd": 3665,"peak_hour": "2025-03-10 10:00:00","peak_agents": 6,"overrides": 0}} -
Now make Tomás’s change again: set
LEFT_OVER_SHARE = 0.30inforecast_job/config.py. Run the week:Terminal window python -m forecast_job run --week 2025-03-10Week 2025-03-10 was planned with other inputs (244b2032b74c7443, now eb041f889a02059e). Run again with --force to replace the plan (the old one is kept).The setting is part of the fingerprint, so the job refuses to replace Kenji’s plan. It ends with exit code 2, which a scheduler sees as a failure.
-
Set
LEFT_OVER_SHAREback to0.25, and run the week once more. The fingerprint is244b2032b74c7443again, so the job printsNothing to do.
To try a setting without touching the real plans, give the job another folder: --store try-runs. Use --force only when the new plan is the one that people will use, and tell them.
Replay: run the job on past weeks
Section titled “Replay: run the job on past weeks”You tested forecasts with a backtest. The job needs the same test, because it is more than a forecast: it has data checks, a store and, in the next lessons, alarms. A replay runs the whole job week after week on past Mondays, each week seeing only what was known on its Monday.
python -m forecast_job replay --start 2025-03-03 --weeks 3Week 2025-03-03: 12847 units to order (expected cost 32485 dollars), 418 agent hours (expected cost 3734 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-03Week 2025-03-10 is already planned with the same inputs (244b2032b74c7443). Nothing to do.Check 2025-03-03: all categories +5.7%, largest plants-and-seeds -22.8%, 0 alarmsWeek 2025-03-17: 13504 units to order (expected cost 32004 dollars), 382 agent hours (expected cost 3708 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-17Check 2025-03-10: all categories -4.0%, largest plants-and-seeds -56.1%, 1 alarms BIAS ALARM plants-and-seeds: the forecast was too low 2 weeks in a row (-23% and -56%, weeks of 2025-03-03, 2025-03-10). Look for the cause before the next order (a season that starts early, a stock-out, a price change), then refit or change the method.Replayed 3 weeks from 2025-03-03: 1 bias alarms, 0 weeks stopped.It takes about 10 seconds. The replay planned 3 and 17 March and kept the plan of 10 March, whose fingerprint did not change. After each week, it compared the plan of the week before with what was sold. Those Check lines and the alarm are the topic of the next lesson.
The data check stops a week
Section titled “The data check stops a week”Mei’s sales feed dropped one day: Sunday 19 October 2025 has no rows. Plan the Monday after it:
python -m forecast_job run --week 2025-10-20Week 2025-10-20: STOPPED, the data is not safe to use: the sales feed is late: the last day is 2025-10-18, expected 2025-10-19. Nothing was written.The job ends with exit code 2 and writes no folder. Kenji keeps last week’s plan, and Mei fixes the feed. Without the check, the job would forecast from a history that ends one day early, and nobody would see it.
Run it every Monday
Section titled “Run it every Monday”A job is useful only if it runs on time. On Linux and macOS, the simplest scheduler is cron. It reads a table of lines, and each line says when to run a command. The job’s README.md has this line:
0 6 * * 1 cd /path/to/forecast-job && .venv/bin/python -m forecast_job run --week "$(date +\%F)"The first five fields say when: minute 0, hour 6, any day of the month, any month, weekday 1 (Monday). $(date +\%F) writes today’s date, for example 2025-03-10. Add the line with crontab -e, with your folder in place of /path/to/forecast-job. On Windows, Task Scheduler does the same work. Your computer must be on at 06:00 for cron to run.
A real team also needs a message when the job fails, a second try after a crash, and one place to see every run. The MLOps course covers that. Our store makes the second try safe.
When to use a job, and what it costs
Section titled “When to use a job, and what it costs”Make a forecast a job when people use it on a schedule, as Kenji and Grace do. Keep a notebook for questions that you ask once, such as Omar’s 30% cost. A job has costs:
- Disk. Each week’s folder takes about 0.5 MB: a year of plans is about 30 MB. Keep them; the next two lessons read old plans.
- Discipline. A setting change makes a new fingerprint. You must decide, with
--force, which plan people use. - Time. The job refits every model every Monday: about 5 seconds here. For a much bigger shop, that can take hours; then you refit less often and only update the forecast.
Common mistake
The job runs on a Sunday
Symptom: Tomás schedules the job for Sunday at 23:00, to have the plan ready early. On Monday, there is no new plan. The scheduler’s log shows:
2025-03-09 is not a Monday. The job runs for a week that starts on a Monday.Cause: the cron line passes today’s date, and today is Sunday. The job plans a week that starts on a Monday, with the sales up to the Sunday before. At 23:00, Sunday’s sales are not complete yet.
Fix: run the job on Monday morning, after Mei’s feed for Sunday has arrived, with some time for delays. If the feed is late, the data check stops the job; run it again when the feed is there.
Consequence in AI work: a scheduled job that starts before its data is ready forecasts from an incomplete day, and the plan looks normal. Check the data’s last day, and schedule the job after the data.
Your turn: predict the replay
Section titled “Your turn: predict the replay”You have plans for 3, 10 and 17 March in runs/. Before you run the next command, predict which weeks the replay plans, which weeks it keeps, and how many Check lines it prints.
python -m forecast_job replay --start 2025-03-03 --weeks 4Show the answer
The first three weeks already have plans with the same fingerprints (already planned … Nothing to do. three times). It plans only 24 March:
Week 2025-03-24: 15334 units to order (expected cost 33596 dollars), 382 agent hours (expected cost 3667 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-24It prints three Check lines, for 3, 10 and 17 March, with 3 alarms. A check compares a plan’s first 7 days with their sales. So the week of 24 March is checked only when the job runs on 31 March. A replay is safe to run again and again: it plans only the missing weeks.
Knowledge check
Section titled “Knowledge check”Check your understanding
Explain with Larkfield's example why a forecasting job must keep every plan and never replace one silently.
Show the answer
Kenji orders on Monday from the plan of that week: 12,839 units for the week of 10 March. Suppose that a rerun replaces the plan, for example after Tomás changed the left-over cost to 30% (12,519 units). Then the files that explain Kenji’s order are gone. Nobody can compare the forecast that was used with what was sold, and nobody can explain the order later. The job keeps one folder per Monday, refuses to replace a plan made with other inputs, and with --force keeps the old plan.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
What is in the job's fingerprint, and what does a second run with the same fingerprint do?
Show the answer
The fingerprint is a short hash of everything that can change the plan. That is the sales and tickets known on the Monday, the promotions, the price changes and the product list. It is also every setting in config.py, the planner’s overrides and the job’s version. A second run with the same fingerprint does nothing. It prints already planned with the same inputs (244b2032b74c7443). Nothing to do. So a retry after a crash is harmless: the run is idempotent.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: Mei delivers a corrected sales file for the week before 10 March, with 3 rows changed. You run the week of 10 March again. What happens, and what should you do?
Show the answer
The sales are part of the fingerprint, so it changes. The job stops with Week 2025-03-10 was planned with other inputs (…). Run again with --force to replace the plan (the old one is kept). and exit code 2. Decide with Kenji which plan counts. If his order has not gone out, run with --force: the old plan is kept in runs/2025-03-10/replaced-244b2032b74c7443/. If the order is already placed, keep the old plan; it is what happened.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A job runs the same steps every week with no person at the keyboard: get the data, check it, forecast, reconcile, plan, keep. A late feed stops the week, and nothing is written.
- A fingerprint of all the inputs makes runs idempotent: the same inputs give
Nothing to do., and other inputs need--force, which keeps the old plan. - Every plan stays in
runs/<Monday>/. A replay runs the job on past Mondays, as a backtest of the whole job. - cron can start the job every Monday at 06:00.
The job now keeps a plan for every week, but nobody looks at it again. Next, you compare each plan with what was really sold, and let the job raise an alarm when the forecast is off.
Next: Watch the forecast