Bundle model dependencies

What you will be able to do

Put everything that the model needs to run, and to be checked, into one folder with a version number that never changes.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• The inference contract and contract.py, from Prediction inputs and outputs.
• What the model artifact holds, and why the whole pipeline is saved, from Package artifacts in Machine Learning: From Problem to Reliable Model.

What you need on your computer

• ~/projects/escalation-service with .venv, requirements.txt and escalation/contract.py, from Prediction inputs and outputs: Write the contract.
• The machine learning course’s ~/projects/ticket-model, next to it, with model/ (from python train.py) and data/ (valid.csv and extra_features.csv). If you downloaded C17/start instead, it has both folders, but no .venv: for the first example, use the .venv of escalation-service.
• If your project is missing or broken, you can copy the finished module, end-of-m01, and read along. Use a snapshot has the steps.

Tested with Python 3.14.6 on macOS, Apple silicon. The Windows and Linux steps are not tested.

A box of flat-pack furniture

A flat-pack bookcase comes in one box. It holds the boards, a bag of screws, the instructions and a list of parts to check against. The box has a model number on the outside. If the screws were in another box, the boards would be useless. And when the factory changes the design, it prints a new model number. It never puts different boards in a box with the old number.

The machine learning course compared a model artifact to a recipe that must include the oven temperature (Package artifacts). A service needs more than that course’s model/ folder. It needs the contract, the versions that were tested, a set of tickets to check with, and a version that names all of it.

The problem: the model alone

Tomás looks at the pipeline from the machine learning course: “The real model is the last step, the logistic regression. Save only that: the file is five times smaller.” He saves pipeline[-1] and scores the example ticket:

Python, in ~/projects/ticket-model

import joblib
import pandas as pd

pipeline = joblib.load("model/escalation_model.joblib")
model = pipeline[-1]  # only the logistic regression
ticket = pd.DataFrame([dict(channel="email", team="payment", segment="home", region="west",
    priority=1, order_value=89.5, word_count=112, customer_tenure_days=400,
    prior_tickets_90d=2, created_hour=10, prior_escalations_90d=1)])
model.predict_proba(ticket)

The output (shortened):

UserWarning: X has feature names, but LogisticRegression was fitted without feature names
...
ValueError: could not convert string to float: 'email'

The file is smaller (1,087 bytes instead of 5,354), but the model is useless alone. The logistic regression never saw the text email. It learned from 28 numbers that the preprocessing made: one column for each known category, and the scaled numbers with their missing-value indicators. The medians, the means, the scales and the list of categories are all in the preprocessing steps. Without them, nobody can make those 28 numbers exactly as in training.

So the first rule of this lesson is the rule of the machine learning course: save the whole pipeline, preprocessing and model, as one object. The failure case below shows what happens when Tomás tries to rebuild the preprocessing instead.

The idea: a bundle

A file that training makes and that another program uses later is an artifact, as in the machine learning course. For serving, one artifact is not enough. The service needs a set of files that travel together, under one version. In this course that set is a bundle: one folder for each model version. It holds everything that the model needs to run, and everything that you need to check it.

Training's model folder becomes a versioned bundle of five files. The service reads the bundle, and checks it against a digest that comes from outside the bundle.

Text description of the diagram

On the left is the training side: the machine learning course’s folder ticket-model/model/ with escalation_model.joblib, model_info.json and check_sample.csv, and the file data/valid.csv for the parity cases. An arrow goes to python make_bundle.py --version 1.0.0.

In the middle is the versioned bundle, bundles/escalation-1.0.0/. It has five files: model.joblib (the pipeline: preprocessing and model), contract.json (features, order, rules, threshold), metadata.json (version, data hashes, tested versions), parity_cases.csv (14 tickets and their expected scores) and SHA256SUMS (the SHA-256 hash of each file above). Below it, a dashed box says that a new version, such as bundles/escalation-1.1.0/, is a new folder, and that 1.0.0 never changes.

On the right is the serving side: escalation-service reads the bundle with load_bundle at start-up. Below the service, a dashed box holds MODEL_SHA256, 8f233029aa4c...: the digest, which is the hash of SHA256SUMS, kept in a trusted place outside the bundle. A dashed arrow marked “pinned” goes from it to the service.

Each file has one job:

• model.joblib: the fitted pipeline, the same bytes as the machine learning course’s escalation_model.joblib. It scores tickets exactly as in training.
• contract.json: the contract of the last lesson, written down for people and programs. It has the features in order, the JSON schema of TicketFeatures and the known categories. It also has the missing-value rules and the meaning of the output, with the threshold.
• metadata.json: where the model came from. It records the version, the training date and the number of training rows. It also has the hashes of the training data, the validation results and the package versions that made the model.
• parity_cases.csv: 14 fixed tickets with the score that the model gave each one at bundle time. Any copy of the model, anywhere, must give the same scores. It does the job of the machine learning course’s check_sample.csv, the photo of the finished cake.
• SHA256SUMS and the digest come in the next lesson: they prove that nobody changed the files.

Omar asks: “The threshold is in contract.json. Why not in the service’s settings?” Because the threshold belongs to this model. A retrained model gives other scores, and 0.19 can then flag far more than 20% of tickets. Module 5 shows exactly that with a real candidate. So the threshold travels with the model, in the same folder.

The parity cases

parity_cases.csv holds four kinds of tickets, because each kind can break in a different way:

Kind	How many	What it checks
normal	6	Real validation tickets with no missing value
missing	3	order_value null, a guest with customer_tenure_days null, and both: the imputer’s medians
extreme	3	Values at the edges of the contract: T-900004 has every risk at its maximum and scores exactly 1.0
unknown	2	A channel the model never saw (social), and an unknown region and team

Module 3 uses the same 14 cases to prove that the running service gives the training scores.

A version is never changed

Once bundles/escalation-1.0.0/ exists, nobody changes it. A retrained model, a new threshold or even a fixed typo in contract.json is a new version in a new folder.

Why so strict? Other things point at “1.0.0”: the service’s settings, every logged answer ("model_version": "1.0.0") and the release record. If 1.0.0 could change, the same name would mean two models. A rollback to 1.0.0 would then not return to the model that worked. make_bundle.py enforces the rule: it stops if the folder exists. The cost is small: one folder of about 12 kB for each version, kept as long as any service or record may need it.

The versions that were tested

metadata.json records the scikit-learn, numpy, pandas and joblib versions. In the next lesson, the service refuses to load the bundle with any other versions. scikit-learn promises a saved object only for the version that saved it (Package artifacts: When versions differ in the machine learning course). The Python version is recorded too (3.14.6), but not checked. The container in Module 4 runs Python 3.14.8, and a patch release of Python does not change how these packages compute.

Write the bundle maker

Start a new file, escalation/bundle.py. It names the files of a bundle and the packages whose versions must match. The next lesson adds the code that loads a bundle.

escalation/bundle.py

"""The model bundle: everything the model needs to run, in one folder.

    model.joblib       the fitted pipeline (preprocessing + model)
    contract.json      the features, their order and rules, and the threshold
    metadata.json      the version, the training data and the tested versions
    parity_cases.csv   fixed tickets and the scores they got in training
"""

FILES = ["model.joblib", "contract.json", "metadata.json", "parity_cases.csv"]
PACKAGES = ["scikit-learn", "numpy", "pandas", "joblib"]

make_bundle.py goes in the project folder, next to requirements.txt. The first part chooses the parity cases:

make_bundle.py

"""Make a model bundle from the C05 project's model/ folder.

    python make_bundle.py ../ticket-model --version 1.0.0

Reads ../ticket-model/model/ (escalation_model.joblib, model_info.json) and
../ticket-model/data/ (for the parity cases), and writes
bundles/escalation-<version>/ with the model, its contract, its metadata and the
parity cases.

Run it only on a model that you trained yourself, or that comes from a source
you trust: it loads the joblib file, and loading can run code.
"""

import argparse
import json
import shutil
import sys
from importlib.metadata import version
from pathlib import Path

import joblib
import pandas as pd

from escalation.bundle import PACKAGES
from escalation.contract import FEATURES, TicketFeatures

# Fixed tickets that every copy of the model must score the same way.
# Normal: real validation tickets. Missing: the two numbers that may be null.
# Extreme: values at the edges of the contract. Unknown: a category the
# model never saw.
EDGE_CASES = [
    ("missing", "T-900001", dict(team="account", order_value=None)),
    ("missing", "T-900002", dict(segment="guest", customer_tenure_days=None, prior_tickets_90d=0, prior_escalations_90d=0)),
    ("missing", "T-900003", dict(order_value=None, customer_tenure_days=None)),
    ("extreme", "T-900004", dict(priority=1, order_value=5000, word_count=10000, prior_tickets_90d=500, prior_escalations_90d=500, created_hour=23)),
    ("extreme", "T-900005", dict(priority=3, order_value=0, word_count=0, customer_tenure_days=0, prior_tickets_90d=0, prior_escalations_90d=0, created_hour=0)),
    ("extreme", "T-900006", dict(customer_tenure_days=20000, created_hour=0)),
    ("unknown", "T-900007", dict(channel="social")),
    ("unknown", "T-900008", dict(region="overseas", team="installation")),
]
BASE = dict(channel="email", team="payment", segment="home", region="west", priority=2, order_value=120.0,
            word_count=80, customer_tenure_days=500, prior_tickets_90d=1, created_hour=11, prior_escalations_90d=0)


def read(path):
    """Read a ticket CSV the way C05 does: only an empty cell is missing."""
    return pd.read_csv(path, keep_default_na=False, na_values=[""])


def parity_cases(data: Path) -> pd.DataFrame:
    valid = read(data / "valid.csv")
    extra = read(data / "extra_features.csv")[["ticket_id", "prior_escalations_90d"]]
    valid = valid.merge(extra, on="ticket_id", how="left", validate="one_to_one")
    normal = valid.dropna(subset=["order_value", "customer_tenure_days"]).head(6)
    rows = [{"case": "normal", "ticket_id": r.ticket_id, **{f: getattr(r, f) for f in FEATURES}}
            for r in normal.itertuples()]
    rows += [{"case": case, "ticket_id": tid, **{**BASE, **change}} for case, tid, change in EDGE_CASES]
    cases = pd.DataFrame(rows, columns=["case", "ticket_id"] + FEATURES)
    for name in ["priority", "word_count", "customer_tenure_days", "prior_tickets_90d", "created_hour",
                 "prior_escalations_90d"]:
        cases[name] = cases[name].astype("Int64")  # whole numbers, with an empty cell when missing
    for row in cases[FEATURES].to_dict("records"):  # every case must pass the contract
        TicketFeatures(**{k: (None if pd.isna(v) else v) for k, v in row.items()})
    return cases

The edge cases change only a few values of one BASE ticket, so each case tests one thing. The last loop checks every case with the contract: a parity case that the service would reject is useless.

The second part, main(), goes at the end of the same file. It writes the four files:

make_bundle.py (continued)

def main():
    parser = argparse.ArgumentParser(description="Make a model bundle from a C05 project.")
    parser.add_argument("project", type=Path, help="the C05 ticket-model folder")
    parser.add_argument("--version", required=True, help="the model version, for example 1.0.0")
    parser.add_argument("--model-dir", default="model", help="the folder in the project with the model")
    args = parser.parse_args()

    source = args.project / args.model_dir
    info = json.loads((source / "model_info.json").read_text(encoding="utf-8"))
    pipeline = joblib.load(source / "escalation_model.joblib")
    if list(pipeline.feature_names_in_) != FEATURES:
        sys.exit("Stopped: the model was trained on other features than this service expects.")

    out = Path("bundles") / f"escalation-{args.version}"
    if out.exists():
        sys.exit(f"Stopped: {out} exists. A version is never changed: choose a new version.")
    out.mkdir(parents=True)

    shutil.copyfile(source / "escalation_model.joblib", out / "model.joblib")

    cases = parity_cases(args.project / "data")
    cases["expected_score"] = pipeline.predict_proba(cases[FEATURES])[:, 1]
    cases.to_csv(out / "parity_cases.csv", index=False)

    contract = {
        "model": "escalation",
        "model_version": args.version,
        "unit": "one support ticket, scored when it is created",
        "features": FEATURES,
        "schema": TicketFeatures.model_json_schema(),
        "known_categories": info["features"]["categories"],
        "missing_values": {
            "order_value": "null when the ticket has no order; the pipeline fills the training median",
            "customer_tenure_days": "null for a guest; the pipeline fills the training median",
            "other features": "never null: the request is rejected",
        },
        "unknown_categories": "allowed; scored as none of the known values; reported as a warning",
        "output": {
            "score": "0 to 1; a higher score means a higher risk of escalation; it ranks tickets",
            "flag": "true if score >= threshold: send the ticket to the senior team",
            "threshold": info["threshold"],
        },
    }
    (out / "contract.json").write_text(json.dumps(contract, indent=2) + "\n", encoding="utf-8")

    metadata = {
        "model_version": args.version,
        "description": info["model"],
        "trained_at": info["trained_at"],
        "train_rows": info["train_rows"],
        "data_sha256": info["data_sha256"],
        "evaluation": {"on": info.get("evaluated_on", "valid (May 2026)"), **info["valid"]},
        "review_capacity": info["review_capacity"],
        "python": info["versions"]["python"],
        "versions": {p: version(p) for p in PACKAGES},
    }
    trained = {"scikit-learn": info["versions"]["scikit-learn"], "numpy": info["versions"]["numpy"],
               "pandas": info["versions"]["pandas"], "joblib": info["versions"]["joblib"]}
    if trained != metadata["versions"]:
        sys.exit(f"Stopped: the model was trained with {trained}, but this environment has {metadata['versions']}.")
    (out / "metadata.json").write_text(json.dumps(metadata, indent=2) + "\n", encoding="utf-8")

    print(f"Made {out}: {len(cases)} parity cases, threshold {info['threshold']}.")


if __name__ == "__main__":
    main()

Three checks stop the script before it writes a wrong bundle. The pipeline’s features must equal FEATURES. The folder must not exist. And the installed package versions must equal the ones that trained the model. The expected scores are computed here, at bundle time, with the same pipeline that goes into the bundle.

Make the bundle

1. In ~/projects/escalation-service, with .venv active, make version 1.0.0:

   Terminal

   python make_bundle.py ../ticket-model --version 1.0.0

   The output:

   Made bundles/escalation-1.0.0: 14 parity cases, threshold 0.19.

2. List the files.

   macOS / Linux

   ls -l bundles/escalation-1.0.0

   Windows (PowerShell)

   Get-ChildItem bundles\escalation-1.0.0

   The output (macOS, shortened):

   -rw-r--r--  1 ...  4270 Oct  8 10:35 contract.json
   -rw-r--r--  1 ...   879 Oct  8 10:35 metadata.json
   -rw-r--r--  1 ...  5354 Oct  8 10:35 model.joblib
   -rw-r--r--  1 ...  1317 Oct  8 10:35 parity_cases.csv

   model.joblib has 5,354 bytes, like the file from the machine learning course. The whole bundle is about 12 kB.

3. Run the same command again, to see the version rule:

   Terminal

   python make_bundle.py ../ticket-model --version 1.0.0

   The output:

   Stopped: bundles/escalation-1.0.0 exists. A version is never changed: choose a new version.

   The exit code is 1, and nothing in the folder changed.

Open bundles/escalation-1.0.0/metadata.json. Yours has your own trained_at; the rest matches:

bundles/escalation-1.0.0/metadata.json (shortened)

{
  "model_version": "1.0.0",
  "description": "Larkfield ticket escalation: logistic regression in a scikit-learn pipeline",
  "trained_at": "2026-10-08 13:53:54 UTC",
  "train_rows": 16939,
  "data_sha256": {
    "train.csv": "63b8732a28e891189cc28a9bcd505a093a9c4c51c01f3d0e1fba6d4959242d53",
    ...
  },
  "evaluation": {
    "on": "valid (May 2026)",
    ...
    "flag_rate": 0.193,
    "precision": 0.379,
    "recall": 0.47
  },
  "review_capacity": 0.2,
  "python": "3.14.6",
  "versions": {
    "scikit-learn": "1.9.1",
    "numpy": "2.5.3",
    "pandas": "3.0.6",
    "joblib": "1.6.0"
  }
}

A person who finds this folder in a year can answer four questions. Which model is this? What data trained it? How good was it? Which packages run it? Without the folder, nobody could.

Your turn: answer from the bundle only

Answer three questions with the files in bundles/escalation-1.0.0/ only, not with the training project:

1. What threshold does model 1.0.0 use?
2. Which parity case has the highest expected score, and what is it?
3. Which values of channel does the model know?

Show the answer

One way, in python:

Python, in ~/projects/escalation-service

import json
import pandas as pd

contract = json.load(open("bundles/escalation-1.0.0/contract.json"))
cases = pd.read_csv("bundles/escalation-1.0.0/parity_cases.csv")
print(contract["output"]["threshold"])
print(cases.loc[cases["expected_score"].idxmax(), ["case", "ticket_id", "expected_score"]].tolist())
print(contract["known_categories"]["channel"])

The output:

0.19
['extreme', 'T-900004', np.float64(1.0)]
['chat', 'email', 'phone', 'unknown', 'web_form']

np.float64(1.0) is numpy’s way to show the number 1.0. The value unknown is a real category in Larkfield’s data (the channel was not recorded), not a missing value. social is not in the list, so it gets a warning.

Common mistake

Rebuilding the preprocessing at scoring time

Symptom: Tomás keeps only the logistic regression, and rebuilds the preprocessing with the same settings, fitted on the tickets that he must score. On the whole of July (1,400 tickets) the scoring fails:

ValueError: X has 29 features, but LogisticRegression is expecting 28 features as input.

He removes the 91 tickets from the new social channel and tries again. Now it runs without any error, but compared with the real pipeline:

prep refitted on July without social (1309 rows): largest difference 0.068, median 0.007; flagged 0.177 vs 0.199; flag changed for 29 tickets

Cause: the rebuilt preprocessing learned July’s medians, means and categories, not training’s. A new category adds a column, so the shapes do not match. Without a new category, the shapes match, but every number is moved a little: 29 tickets changed their flag, and nothing said so.

Fix: never rebuild what fit learned. Bundle the whole fitted pipeline, and check every copy against the parity cases. A difference of 0.068 fails a tolerance of 1e-9 at once.

For a real AI service: the loud failure is the lucky one. The quiet one changes decisions for real customers. This is why a bundle carries the pipeline and the parity cases together.

Check your understanding

Tomás says: the logistic regression is the model, so model.joblib can hold only that, and the service can prepare the data. Give two reasons why the bundle holds the whole pipeline.

Show the answer

First, the model learned from 28 numbers that the fitted preprocessing made (one-hot columns, imputed and scaled numbers, missing-value indicators). Alone, it cannot read a ticket: could not convert string to float: 'email'. Second, preprocessing that is rebuilt at scoring time learns other medians, scales and categories. It either fails (29 features instead of 28) or changes scores quietly: 29 July flags changed with no error.

Check your understanding

Priya finds a typo in the description in contract.json of bundle 1.0.0, which is live. Should she fix the file? What should she do?

Show the answer

No. A bundle version never changes. The settings, the logged answers and the release record all point at “1.0.0”, and a rollback must return to exactly that bundle. She makes a new version, for example 1.0.1, in a new folder, and releases it like any other version.

Check your understanding

Why does the threshold 0.19 travel in the bundle and not in the service's settings?

Show the answer

The threshold was chosen for this model’s scores, so that it flags about 20% of tickets. A retrained model can give higher or lower scores, and the old threshold can then flag far more or far fewer tickets. So the threshold belongs to the model version, and a new version brings its own.

Recap

• A model without its fitted preprocessing cannot score a ticket. Rebuilding the preprocessing fails, or changes scores quietly.
• A bundle is one folder per model version. It holds the pipeline, the contract with the threshold, the metadata with the tested versions, and 14 parity cases.
• A version is never changed. A fix is a new version in a new folder, and make_bundle.py refuses to overwrite.
• bundles/escalation-1.0.0/ now exists, with four files.

Next, you prove that nobody changed the bundle. You add a hash for each file and one digest for the whole bundle, and a loader checks both before it runs anything.

Next: Artifact trust and integrity