Skip to content
Français

The forecasting job

  • Lesson 25
  • 45 min
  • Module 8: Run a forecast in production

Where we are

We can now turn a forecast into an order and a roster in a notebook. A notebook does not run itself every Monday. In this lesson, we run a small forecasting job from the command line that gets the data, checks it, forecasts and writes the plan. Think first of a grocery order that you place online every week.

What you will be able to do

Run the forecast every Monday as a small job that nobody has to watch. It gets the data, checks it, forecasts, and writes and keeps the plan. A second run of the same week changes nothing.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

What you need on your computer

  • Python 3.12, a terminal and VS Code (or another editor). Git is useful, but you can download a ZIP file instead.
  • About 400 MB of free disk space for the virtual environment. No account, no key and no money.

Tested with Python 3.12, pandas 2.3.3, NumPy 2.4.6 and statsforecast 2.1.1 on macOS. The Windows (PowerShell) commands are written but not tested.

Think of a family that orders its groceries online every Monday. The app follows the same steps each time: it checks the basket, works out the total, sends the order and keeps a receipt.

Two details make this app safe to use. First, if you press Order twice by mistake, you do not get two vans at your door. The shop sees that it is the same order. Second, the receipts stay. In a month, you can open any week’s receipt and see what you paid and why.

Larkfield needs the same two details. Every Monday, Kenji orders stock and Grace plans the roster from the job’s plan.

In Modules 2 to 7, you forecast in notebooks. A notebook is good for learning, but a person must open it, run the cells in order and copy the result. On a busy Monday, someone forgets a cell, or runs an old cell twice.

A job is a program that runs from start to end with no person at the keyboard. It does the same steps in the same order every time. A scheduler (a program that starts other programs at set times) can start it every Monday morning.

Larkfield’s job is a small Python program, forecast_job, that puts together what the notebooks did. For one Monday, it does six steps:

Six steps from left to right: get the data, check it, forecast, reconcile, plan, keep. From the check, a branch marked no goes to STOPPED, nothing is written. A dotted branch carries warnings to the kept plan.
The forecasting job for one Monday. The data check comes before any forecast. If the data is not safe, the job stops and writes nothing.
Text description of the diagram

The diagram shows six boxes joined by arrows. Box 1, get the data known on the Monday. Box 2, a decision: is the data safe to use? If not (the feed is late, a row is there twice, a sale is negative), an arrow goes to STOPPED. Nothing is written, and last week’s plan stays in use. If yes, box 3, forecast: the 600 product series for 14 days and the tickets for one week, each with a range. Box 4, reconcile: bottom-up. Box 5, plan: the stock order at the 58% quantile and the roster at the 72% quantile. Box 6, keep: a folder for the Monday with a fingerprint of the inputs. A dotted arrow carries warnings (a missing day filled, many stock-outs, a price change) to the kept plan.

Each step uses a method that the course chose from its backtest table:

  • Forecast: ETS with a weekly season for each of the 600 product series (200 products in 3 regions), 14 days ahead. MSTL with a daily and a weekly season for the tickets, one week ahead.
  • Reconcile: bottom-up. The job adds the product forecasts up to the regions, the categories and the total, so they always agree.
  • Plan: the order is the 58% quantile of each product’s 14-day demand; the roster covers the 72% quantile of each hour’s tickets. Both quantiles come from the course’s assumed costs.

The job comes in two snapshots in the labs repository. A snapshot is a copy of the project at one point of the course. start is the job at the end of Module 7, and end-of-m08 at the end of this module. You start from start.

  1. Get the files of the course. If you have Git, clone the labs repository once, in ~/projects. If you have the clone already, run git -C nextia-learning-labs pull instead.

    Terminal window
    cd ~/projects
    git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.git

    Without Git, download the ZIP file of the repository and unzip it in ~/projects. Use its folder name, nextia-learning-labs-main, in the next steps.

  2. Copy the snapshot start to a folder of your own, forecast-job:

    Terminal window
    cp -R nextia-learning-labs/C35/forecast-job/start forecast-job
    cd forecast-job
  3. Make the virtual environment, install the packages and get the data:

    Terminal window
    python3 -m venv .venv
    source .venv/bin/activate
    python -m pip install -r requirements.txt
    python get_data.py

    get_data.py downloads about 2 MB of Larkfield data into data/, checks each file and ends with:

    Ready: 10 files in data/, every checksum is right.
  4. Run the tests:

    Terminal window
    python -m pytest -q
    14 passed in 17.56s

    Your time will differ; the first run is slower.

  5. Plan one week, the week of Monday 10 March 2025:

    Terminal window
    python -m forecast_job run --week 2025-03-10
    Week 2025-03-10: 12839 units to order (expected cost 32038 dollars), 397 agent hours (expected cost 3665 dollars), 0 warnings. Plan: plan/2025-03-10

    It takes about 5 seconds.

Read the line from left to right. The job orders 12,839 units for the next 14 days, for all products and regions. Its expected cost, US$32,038, is the average cost of the units that will be short or left over. It is right only if the forecast range is right. The roster has 397 agent hours for the week (expected cost US$3,665). The data check found no problem.

The job uses only the data known on that Monday: the sales up to Sunday 9 March, and the promotions and prices planned in advance. So you can plan any past Monday as if it were today.

The folder plan/2025-03-10/ now has four files:

File What it holds
forecast.csv One row per product, region and day: the forecast yhat and its range (q5, q10, q90, q95)
totals.csv The forecast added up per region, category and for the whole shop
orders.csv One row per product and region: the 14-day forecast, its 80% range, the order and its expected cost
roster.csv One row per hour: the ticket forecast, its range, the planned tickets and the number of agents

The roster file shows how a range becomes a decision. Here is Monday, hour by hour:

  • Ticket forecast
  • 80% range (10% to 90%)
  • Agents on the roster × 6 tickets
The job's roster for Monday 10 March 2025. Each agent answers about 6 tickets an hour (an assumption). The roster covers the ticket forecast with some room, up to 6 agents at 10:00 and 11:00. The help desk is open from 07:00 to 21:59.
Text description of the chart

The chart has four lines over the 24 hours of Monday 10 March 2025. The ticket forecast, in blue, is about 2 to 4 tickets an hour at night. It rises from 11 at 07:00 to 33 at 10:00 and 11:00, and falls to about 11 at 17:00. It rises again to about 16 at 19:00 and 20:00, and falls at night. Two dashed grey lines show the 80% range, about 4 to 5 tickets on each side. The orange line is the roster: the agents on duty times 6 tickets. It is 0 outside 07:00 to 21:59, and above the forecast in the open hours. There are 3 agents (18 tickets) at 07:00 and 6 (36 tickets) at 10:00 and 11:00. In the evening, there are 3 or 4.

The orange line sits a little above the forecast, because the job plans for the 72% quantile. An hour one agent short costs more than an extra agent hour.

On Tuesday, Omar asks a question: “What if a left-over item costs us 30% of its price, not 25%?” Tomás tries it at once. He changes LEFT_OVER_SHARE in forecast_job/config.py from 0.25 to 0.30, and runs the same week again:

Week 2025-03-10: 12519 units to order (expected cost 35355 dollars), 397 agent hours (expected cost 3665 dollars), 0 warnings. Plan: plan/2025-03-10

The answer is useful: with a higher cost of left-over stock, the job orders 320 units less. But look at the path at the end: the job wrote the new plan into the same folder. Kenji placed his order on Monday for 12,839 units, and the files that explain that order are gone. Nothing warned Tomás.

The same happens with no change of settings. A scheduler often starts a job again after a crash, and Mei sometimes delivers a corrected sales file on Tuesday. Each rerun replaces the Monday plan without a trace. Then nobody can check what the job really planned against what was sold, and the next lesson needs exactly that check.

(Undo Tomás’s change now: set LEFT_OVER_SHARE back to 0.25.)

The idea: same inputs, same plan, every plan kept

Section titled “The idea: same inputs, same plan, every plan kept”

The job needs the two details of the grocery app:

  1. Keep every plan. Each Monday gets its own folder, and the job never deletes one.
  2. Make a second run harmless. A run is idempotent when running it twice has the same effect as running it once.

But how does the job know that a second run is “the same”? It compares a fingerprint of the inputs: a short code that a hash function computes from data. The same data always gives the same code; a change of one value gives a completely different code. A tiny example with Python’s hashlib:

import hashlib
for text in ["sales: 4, 6, 5", "sales: 4, 6, 5", "sales: 4, 6, 6"]:
print(text, "->", hashlib.sha256(text.encode()).hexdigest()[:16])
sales: 4, 6, 5 -> 527a118a0b587747
sales: 4, 6, 5 -> 527a118a0b587747
sales: 4, 6, 6 -> b4900c0c117cecae

In the third line, one sale changed from 5 to 6, and the code is completely different. The job keeps the first 16 characters.

The job’s fingerprint covers everything that can change the plan. That is the data it could see, every setting in config.py, the planner’s overrides and the job’s version. Then the rule is simple:

A decision chart. run --week leads to the question: is there a plan for this week? No: plan and keep it with its fingerprint. Yes: the same fingerprint? Yes: nothing to do. No: --force? No: stop with a message. Yes: replace the plan and move the old files to a folder replaced-fingerprint.
What a run does when the week already has a plan. Only --force replaces a plan, and even then the old plan is kept.
Text description of the diagram

The chart starts at run –week, for one Monday. First question: is there a plan for this week? If no, the job plans and keeps the plan in runs/Monday/, with run.json and its fingerprint. If yes: the same fingerprint? If yes, nothing to do, and no file changes. If no: did you pass –force? If no, the job stops with a message. If yes, it replaces the plan and moves the old files to a folder named replaced- and the old fingerprint.

The code is in forecast_job/store.py. This is its heart:

def fingerprint(inputs, overrides=None):
h = hashlib.sha256()
for df in (inputs.demand, inputs.tickets, inputs.promotions, inputs.price_changes, inputs.products):
h.update(pd.util.hash_pandas_object(df, index=False).to_numpy().tobytes())
if overrides is not None and len(overrides):
h.update(pd.util.hash_pandas_object(overrides, index=False).to_numpy().tobytes())
settings = {k: v for k, v in vars(config).items() if k.isupper()}
h.update(json.dumps(settings, sort_keys=True, default=str).encode())
h.update(__version__.encode())
return h.hexdigest()[:16]

hash_pandas_object turns every row of a table into a number, and h.update feeds them into the hash. vars(config) reads every name in capital letters in config.py.

The snapshot end-of-m08 has the job of this whole module. In this lesson, you copy the code and one test file. You copy the other tests and the planner’s files in the next two lessons.

  1. In forecast-job, with the virtual environment active, copy the new files from the snapshot end-of-m08. Four files are new; three change.

    Terminal window
    S=../nextia-learning-labs/C35/forecast-job/end-of-m08
    for f in forecast_job/store.py forecast_job/monitor.py forecast_job/overrides.py \
    forecast_job/note.py forecast_job/__main__.py tests/test_job.py README.md; do
    cp "$S/$f" "$f"
    done

    store.py is this lesson’s code. __main__.py also uses monitor.py, overrides.py and note.py; the next two lessons explain them.

  2. Run the tests. test_job.py now checks four things: a run writes the plan, a second run changes nothing, other inputs need --force, and bad data writes nothing.

    Terminal window
    python -m pytest -q
    17 passed in 3.16s
  3. Delete the old folder plan (the job now uses runs). On Windows (PowerShell), use Remove-Item -Recurse plan.

    Terminal window
    rm -rf plan

    Plan the same week again, twice:

    Terminal window
    python -m forecast_job run --week 2025-03-10
    python -m forecast_job run --week 2025-03-10
    Week 2025-03-10: 12839 units to order (expected cost 32038 dollars), 397 agent hours (expected cost 3665 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-10
    Week 2025-03-10 is already planned with the same inputs (244b2032b74c7443). Nothing to do.

    The second run saw the same fingerprint and did nothing.

  4. Look at the new folder:

    Terminal window
    ls runs/2025-03-10
    checks.txt forecast.csv orders.csv roster.csv run.json totals.csv

    checks.txt holds the data check’s warnings (no problems). run.json holds the fingerprint and the summary:

    {
    "week": "2025-03-10",
    "fingerprint": "244b2032b74c7443",
    "job_version": "1.0.0",
    "summary": {
    "order_units": 12839,
    "order_value_usd": 596273,
    "order_expected_cost_usd": 32038,
    "agent_hours": 397,
    "roster_expected_cost_usd": 3665,
    "peak_hour": "2025-03-10 10:00:00",
    "peak_agents": 6,
    "overrides": 0
    }
    }
  5. Now make Tomás’s change again: set LEFT_OVER_SHARE = 0.30 in forecast_job/config.py. Run the week:

    Terminal window
    python -m forecast_job run --week 2025-03-10
    Week 2025-03-10 was planned with other inputs (244b2032b74c7443, now eb041f889a02059e). Run again with --force to replace the plan (the old one is kept).

    The setting is part of the fingerprint, so the job refuses to replace Kenji’s plan. It ends with exit code 2, which a scheduler sees as a failure.

  6. Set LEFT_OVER_SHARE back to 0.25, and run the week once more. The fingerprint is 244b2032b74c7443 again, so the job prints Nothing to do.

To try a setting without touching the real plans, give the job another folder: --store try-runs. Use --force only when the new plan is the one that people will use, and tell them.

You tested forecasts with a backtest. The job needs the same test, because it is more than a forecast: it has data checks, a store and, in the next lessons, alarms. A replay runs the whole job week after week on past Mondays, each week seeing only what was known on its Monday.

Terminal window
python -m forecast_job replay --start 2025-03-03 --weeks 3
Week 2025-03-03: 12847 units to order (expected cost 32485 dollars), 418 agent hours (expected cost 3734 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-03
Week 2025-03-10 is already planned with the same inputs (244b2032b74c7443). Nothing to do.
Check 2025-03-03: all categories +5.7%, largest plants-and-seeds -22.8%, 0 alarms
Week 2025-03-17: 13504 units to order (expected cost 32004 dollars), 382 agent hours (expected cost 3708 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-17
Check 2025-03-10: all categories -4.0%, largest plants-and-seeds -56.1%, 1 alarms
BIAS ALARM plants-and-seeds: the forecast was too low 2 weeks in a row (-23% and -56%, weeks of 2025-03-03, 2025-03-10). Look for the cause before the next order (a season that starts early, a stock-out, a price change), then refit or change the method.
Replayed 3 weeks from 2025-03-03: 1 bias alarms, 0 weeks stopped.

It takes about 10 seconds. The replay planned 3 and 17 March and kept the plan of 10 March, whose fingerprint did not change. After each week, it compared the plan of the week before with what was sold. Those Check lines and the alarm are the topic of the next lesson.

Mei’s sales feed dropped one day: Sunday 19 October 2025 has no rows. Plan the Monday after it:

Terminal window
python -m forecast_job run --week 2025-10-20
Week 2025-10-20: STOPPED, the data is not safe to use: the sales feed is late: the last day is 2025-10-18, expected 2025-10-19. Nothing was written.

The job ends with exit code 2 and writes no folder. Kenji keeps last week’s plan, and Mei fixes the feed. Without the check, the job would forecast from a history that ends one day early, and nobody would see it.

A job is useful only if it runs on time. On Linux and macOS, the simplest scheduler is cron. It reads a table of lines, and each line says when to run a command. The job’s README.md has this line:

0 6 * * 1 cd /path/to/forecast-job && .venv/bin/python -m forecast_job run --week "$(date +\%F)"

The first five fields say when: minute 0, hour 6, any day of the month, any month, weekday 1 (Monday). $(date +\%F) writes today’s date, for example 2025-03-10. Add the line with crontab -e, with your folder in place of /path/to/forecast-job. On Windows, Task Scheduler does the same work. Your computer must be on at 06:00 for cron to run.

A real team also needs a message when the job fails, a second try after a crash, and one place to see every run. The MLOps course covers that. Our store makes the second try safe.

Make a forecast a job when people use it on a schedule, as Kenji and Grace do. Keep a notebook for questions that you ask once, such as Omar’s 30% cost. A job has costs:

  • Disk. Each week’s folder takes about 0.5 MB: a year of plans is about 30 MB. Keep them; the next two lessons read old plans.
  • Discipline. A setting change makes a new fingerprint. You must decide, with --force, which plan people use.
  • Time. The job refits every model every Monday: about 5 seconds here. For a much bigger shop, that can take hours; then you refit less often and only update the forecast.

Common mistake

The job runs on a Sunday

Symptom: Tomás schedules the job for Sunday at 23:00, to have the plan ready early. On Monday, there is no new plan. The scheduler’s log shows:

2025-03-09 is not a Monday. The job runs for a week that starts on a Monday.

Cause: the cron line passes today’s date, and today is Sunday. The job plans a week that starts on a Monday, with the sales up to the Sunday before. At 23:00, Sunday’s sales are not complete yet.

Fix: run the job on Monday morning, after Mei’s feed for Sunday has arrived, with some time for delays. If the feed is late, the data check stops the job; run it again when the feed is there.

Consequence in AI work: a scheduled job that starts before its data is ready forecasts from an incomplete day, and the plan looks normal. Check the data’s last day, and schedule the job after the data.

You have plans for 3, 10 and 17 March in runs/. Before you run the next command, predict which weeks the replay plans, which weeks it keeps, and how many Check lines it prints.

Terminal window
python -m forecast_job replay --start 2025-03-03 --weeks 4
Show the answer

The first three weeks already have plans with the same fingerprints (already planned … Nothing to do. three times). It plans only 24 March:

Week 2025-03-24: 15334 units to order (expected cost 33596 dollars), 382 agent hours (expected cost 3667 dollars), 0 warnings, 0 overrides. Plan: runs/2025-03-24

It prints three Check lines, for 3, 10 and 17 March, with 3 alarms. A check compares a plan’s first 7 days with their sales. So the week of 24 March is checked only when the job runs on 31 March. A replay is safe to run again and again: it plans only the missing weeks.

Check your understanding

Explain with Larkfield's example why a forecasting job must keep every plan and never replace one silently.

Show the answer

Kenji orders on Monday from the plan of that week: 12,839 units for the week of 10 March. Suppose that a rerun replaces the plan, for example after Tomás changed the left-over cost to 30% (12,519 units). Then the files that explain Kenji’s order are gone. Nobody can compare the forecast that was used with what was sold, and nobody can explain the order later. The job keeps one folder per Monday, refuses to replace a plan made with other inputs, and with --force keeps the old plan.

Check your understanding

What is in the job's fingerprint, and what does a second run with the same fingerprint do?

Show the answer

The fingerprint is a short hash of everything that can change the plan. That is the sales and tickets known on the Monday, the promotions, the price changes and the product list. It is also every setting in config.py, the planner’s overrides and the job’s version. A second run with the same fingerprint does nothing. It prints already planned with the same inputs (244b2032b74c7443). Nothing to do. So a retry after a crash is harmless: the run is idempotent.

Check your understanding

Predict: Mei delivers a corrected sales file for the week before 10 March, with 3 rows changed. You run the week of 10 March again. What happens, and what should you do?

Show the answer

The sales are part of the fingerprint, so it changes. The job stops with Week 2025-03-10 was planned with other inputs (…). Run again with --force to replace the plan (the old one is kept). and exit code 2. Decide with Kenji which plan counts. If his order has not gone out, run with --force: the old plan is kept in runs/2025-03-10/replaced-244b2032b74c7443/. If the order is already placed, keep the old plan; it is what happened.

Recap

  • A job runs the same steps every week with no person at the keyboard: get the data, check it, forecast, reconcile, plan, keep. A late feed stops the week, and nothing is written.
  • A fingerprint of all the inputs makes runs idempotent: the same inputs give Nothing to do., and other inputs need --force, which keeps the old plan.
  • Every plan stays in runs/<Monday>/. A replay runs the job on past Mondays, as a backtest of the whole job.
  • cron can start the job every Monday at 06:00.

The job now keeps a plan for every week, but nobody looks at it again. Next, you compare each plan with what was really sold, and let the job raise an alarm when the forecast is off.

Next: Watch the forecast