Score credit default in batch and online

What you will be able to do

Serve one model in the two ways that a bank needs. A monthly batch job scores every client for the call list. An online endpoint scores one client while a call centre agent waits. Both use one versioned bundle and one contract. You prove that they give the same scores, and you measure each one. Then you watch the contract stop a broken monthly file before it reaches the call centre.

Case studyPortfolio piece

Dataset

Default of Credit Card Clients, Taiwan 2005 (UCI)

Licence

CC BY 4.0 (data); MIT (code)

Time

About 3 hours

Runs in

VS Code, Python 3.12+

Skills used

• Inference contract from real ranges
• Versioned bundle with a pinned digest
• Batch job with partial failure
• Online endpoint
• Parity between two paths
• Containers with limits
• Throughput and p50/p95

Download the starter project (.zip)Finished project (.zip)

Unzip the starter, open the folder in VS Code, and follow the steps below. They start with a new virtual environment.

The starter has the data and the training script. It also has the parts that the course already built: the bundle loader, settings, scoring code, error shape and API key. You write what is new for this model: the contract, the batch job, the endpoint, the parity check and the containers.

Before you startWhat to know and to have readyAbout 180 minutes

What you need to know

The lessons that this case study uses most:

• The contract and the bundle, from Prediction inputs and outputs, Bundle model dependencies and Artifact trust and integrity.
• The two serving patterns, from Batch inference and Online inference.
• Parity, from Check parity.
• Limits and measurement, from Container resource settings and Benchmark the service.

You do not need the machine learning course. If you did its case study Predict credit-card default, this is the same model idea: now you serve it.

What you need on your computer

• Python 3.12 or newer, VS Code with the Python extension, and curl.
• Docker Desktop, as in Build efficiently.
• About 1.2 GB of free disk space: 360 MB for the virtual environment and 790 MB for the image.

Tested with Python 3.14.6 and Docker Desktop (Docker Engine 29.6.1) on macOS, Apple silicon. The Windows and Linux steps are not tested.

The situation

A bank in Taiwan issues credit cards. Its card-risk team, led by Yu-Ting Lin, makes an early-help call to clients who are likely to miss their next payment. An agent offers a payment plan before the client falls behind. Nothing is refused or closed. The team already has a model: it scores each client from the last six monthly statements.

Two groups need the score, at two different times:

• Yu-Ting needs the call list once a month. On statement day, Hao Tsai, the data engineer, delivers one file with every client. Each client with a score of at least 0.17 goes on the list.
• Jia-Hui Wu leads the call centre. Clients phone in all month. When an agent opens a client’s record, the screen must show the risk now. Then the agent can offer the plan during the call. Jia-Hui’s latency budget for the score: 200 ms.

You are the team’s machine learning engineer. You serve the model to both of them.

Batch or online: the same model, two ways to wait

An electricity company prints every bill in one monthly run: nobody waits for one bill. When you phone about your bill, the agent looks at your account now, while you wait. The bank has the same two needs: a batch job and an online endpoint.

Predict before you look: the API scores one client in a few milliseconds. How much longer than a batch job does it need for all 30,000 clients, with one request each?

The same model and the same container limits (1 CPU, 512 MiB). The batch job, in one run, checks and scores the clients in chunks of 5,000. The API answers one client per request, one request at a time. You measure both in Part 7.

Text description of the chart

Two columns. The batch job scored the 30,000 clients of the monthly file in 1.44 seconds (the middle of three runs: 1.44, 1.47, 1.39 s). The online API, with one request per client and one request at a time, took 187.5 seconds (160 requests per second). The batch job was about 130 times faster.

Each request pays a fixed price (HTTP, JSON, a one-row table, the preprocessing), as in Online inference. The batch job pays it once per 5,000 clients. But a batch score is as old as its file. When a client who paid yesterday phones, the agent needs a score from today’s record.

So the idea is one model, two paths: batch for throughput, online for latency. Both use one bundle, one contract and one scoring function. Fixed parity cases prove that they agree.

One bundle, one digest, one image, two paths. Everything above the two paths is shared, so the paths can differ only in how they receive clients and return scores.

Text description of the diagram

From top to bottom: training/train.py writes the model. make_bundle.py turns it into the folder bundles/default-risk-1.0.0 with five files. The digest MODEL_SHA256 is kept in .env, outside the bundle, and the loader checks that the bundle matches it. Inside one image, load_bundle runs its six gates, and the shared code (contract.py and scoring.py) serves two paths. The batch job reads the monthly file of 30,000 clients and writes scores.csv, rejected.csv and summary.json into out/2005-09/. The online API takes one client from the agent’s screen and answers with the score, the flag and the model version.

	Batch path	Online path
Who waits	Nobody: the list is ready the next morning	The agent and the client, on the phone
Measure	Clients per second, and a run that finishes	p50 and p95 latency against the 200 ms budget
A bad client	Set aside in rejected.csv; the others are scored	422 with the field names, at once
A broken file	The job fails; no call list is written	Not possible: one client per request
Freshness	As old as the file	Today’s record

The model and its data

The data is the UCI Default of Credit Card Clients file: 30,000 clients of a bank in Taiwan. For each client, it has six monthly statements (April to September 2005), and whether the client missed the October payment. Its licence is CC BY 4.0. So we give the credit: “Default of Credit Card Clients” by I-Cheng Yeh, UCI Machine Learning Repository, doi:10.24432/C55S3H, CC BY 4.0. The file in the project is converted from XLS to CSV, and its target column is renamed default. Yu-Ting, Jia-Hui, Hao and the call centre are invented. The clients are real, and anonymous.

The model uses 20 inputs, as in the case study of the machine learning course:

Input	Meaning
EDUCATION	1 graduate school, 2 university, 3 high school, 4 others. The codes 0, 5 and 6 are in the data but not documented: the pipeline puts them into 4.
PAY_1 … PAY_6	Repayment status, from September (1) back to April (6): −1 paid on time, 1 to 8 months late. −2 and 0 are not documented.
LIMIT_BAL	The credit limit, in NT dollars.
BILL_AMT1 … BILL_AMT6	The bill of each month. A negative bill means that the client paid more than they owed.
PAY_AMT1 … PAY_AMT6	The amount paid in each month.

Sex, marital status and age are in the data, but they are not inputs, and the monthly file does not contain them.

The model is a scikit-learn pipeline: one-hot codes (rare codes grouped), log-scaled amounts, then a logistic regression. The threshold 0.17 gave the lowest cost on validation when a missed default costs 5 and a false alarm 1. On the test set, it flagged 38.2% of clients and found 70% of the defaults (recall 0.701). The machine learning case study explains each choice.

The data has one statement month, so the “monthly file” holds the same 30,000 clients that the model learned from. You test the paths here, not the model: do not read its flag rate as a test result.

Set up the starter project

1. Download the starter project with the button in the box at the top of this page. Unzip it into your projects folder. You get a folder credit-default-service.

2. In VS Code, open the folder: File › Open Folder…, then choose credit-default-service.

3. Open a terminal in VS Code: Terminal › New Terminal. It starts in the project folder.

4. Make and activate a virtual environment:

   macOS

   python3 -m venv .venv
   source .venv/bin/activate

   Windows

   python -m venv .venv
   .venv\Scripts\Activate.ps1

   Linux

   python3 -m venv .venv
   source .venv/bin/activate

   The prompt starts with (.venv).

5. Install the pinned versions of the packages:

   python -m pip install -r requirements.txt

   This took 20 seconds on our computer. The environment uses about 360 MB.

6. Tell VS Code to use this environment. Open the Command Palette (Ctrl+Shift+P, or Cmd+Shift+P on macOS), choose Python: Select Interpreter, then choose the one in .venv.

7. Run the tests:

   python -m pytest

The output (last lines):

collected 3 items / 5 skipped

tests/test_features.py ..                                                [ 66%]
tests/test_health.py .                                                   [100%]

======================== 3 passed, 5 skipped in 10.07s =========================

The tests for the parts that you write are already in tests/. They are skipped until their file exists, and more pass after each part: 37 at the end.

What is in the starter

Path	What it is
data/credit_default_taiwan.csv.gz, data/README.md	The data (0.98 MB), its licence and its credit
training/train.py	Trains the model into model/ (Part 1)
scripts/export_month.py	Hao’s export: writes the monthly file (Part 1)
default_risk/features.py	Two small functions that the pipeline calls (Part 1)
default_risk/bundle.py, settings.py, scoring.py, errors.py, security.py	The loader with its six gates, the settings, the shared scoring code, the error shape and the API key, as in the course
default_risk/main.py	A first API with /health only (you replace it in Part 5)
make_bundle.py	Makes the bundle from model/ (Part 3)
bench/bench.py, requests/	The benchmark (Part 7) and three request bodies for curl
requirements.txt, requirements-run.txt, .dockerignore, .env.example	The pins for your computer and for the image, as in the course
MODEL_CARD.md, LICENSE	What the model is for and not for; the MIT licence of the code

Part 1: train the model and export the month

Train the model:

python -m training.train

The output:

Trained on 18000 clients. Threshold 0.17 (the lowest cost on validation).
Valid: ROC AUC 0.775, flagged 39.9%, precision 0.395, recall 0.713.
Test:  ROC AUC 0.770, flagged 38.2%, precision 0.406, recall 0.701.

These are the numbers of the machine learning case study. The script wrote model/model.joblib (7,946 bytes, SHA-256 d71c40c4848a…, the same on every run on one computer) and model/model_info.json.

Open default_risk/features.py. The pipeline calls two functions from it: signed_log for the amounts and group_education for the codes 0, 5 and 6. The ticket model did not have this trap: joblib saves the names of these functions, not their code. A service must have this file, unchanged, so the bundle records its SHA-256 (Part 3).

Now make the monthly file, as Hao’s export does:

python scripts/export_month.py data/clients-2005-09.csv

The output:

Wrote data/clients-2005-09.csv: 30000 clients, 21 columns.

Open data/clients-2005-09.csv in VS Code. Its first lines:

client_id,EDUCATION,PAY_1,PAY_2,PAY_3,PAY_4,PAY_5,PAY_6,LIMIT_BAL,BILL_AMT1,BILL_AMT2,BILL_AMT3,BILL_AMT4,BILL_AMT5,BILL_AMT6,PAY_AMT1,PAY_AMT2,PAY_AMT3,PAY_AMT4,PAY_AMT5,PAY_AMT6
C-000001,2,2,2,-1,-1,-2,-2,20000,3913,3102,689,0,0,0,0,689,0,0,0,0
C-000002,2,-1,2,0,0,0,2,120000,2682,1725,2682,3272,3455,3261,0,1000,1000,1000,0,2000

The file has a client ID and the 20 inputs, nothing else (the source’s PAY_0 is renamed PAY_1). Client C-000002 is the example client of this case study.

Part 2: the contract, with ranges from the data

The problem: the pipeline scores anything

Start Python with python, paste this, then leave with exit(). It scores client C-000002 with the raw pipeline, then three changed copies:

import joblib
import pandas as pd

pipeline = joblib.load("model/model.joblib")
client = pd.read_csv("data/clients-2005-09.csv").iloc[[1]].drop(columns="client_id")
print("as exported:     ", pipeline.predict_proba(client)[:, 1])
for change in [{"PAY_1": 12}, {"EDUCATION": 9}, {"LIMIT_BAL": 120}]:
    print(f"{str(change):16}", pipeline.predict_proba(client.assign(**change))[:, 1])

The output:

as exported:      [0.35052096]
{'PAY_1': 12}    [0.53260864]
{'EDUCATION': 9} [0.30945468]
{'LIMIT_BAL': 120} [0.66270463]

All three inputs are impossible, and all three get a score. A limit written in thousands (120 for 120,000) moves the score from 0.35 to 0.66. The model cannot say that its input is wrong; the contract must, as in Prediction inputs and outputs.

The idea: the data sets the limits

The contract allows only the values that the model saw. Print the smallest and largest value of each column:

python -c "import pandas as pd; d = pd.read_csv('data/clients-2005-09.csv').drop(columns='client_id'); print(d.agg(['min', 'max']).T.to_string())"

The output (shortened):

              min      max
EDUCATION       0        6
PAY_1          -2        8
...
PAY_6          -2        8
LIMIT_BAL   10000  1000000
BILL_AMT1 -165580   964511
...
BILL_AMT3 -157264  1664089
...
BILL_AMT6 -339603   961664
PAY_AMT1        0   873552
PAY_AMT2        0  1684259
...

The status code 9 never occurs. For the bills and the payments, the contract uses one range for all six months, from the smallest to the largest value of any month. So a September bill of 1,000,000 is allowed, even if the data has none.

Step 1: write the contract

Make the file default_risk/contract.py:

default_risk/contract.py

"""The inference contract: what a client's record must look like before the model scores it.

The model learned from 20 columns. This file gives, for each one, its name,
its type and the values it may take. The ranges are the smallest and largest
values in the 30,000 clients of the data: the model has never seen anything
outside them. The batch job and the API both check clients with these
classes, so the two paths always apply the same rules.
"""

from typing import Annotated

import pandas as pd
from pydantic import BaseModel, ConfigDict, Field

STATUS = [f"PAY_{m}" for m in range(1, 7)]
BILLS = [f"BILL_AMT{m}" for m in range(1, 7)]
PAYMENTS = [f"PAY_AMT{m}" for m in range(1, 7)]
# The order of the columns that the pipeline was fitted on (training/train.py).
FEATURES = ["EDUCATION"] + STATUS + ["LIMIT_BAL"] + BILLS + PAYMENTS

# Repayment status: -1 paid on time, 1 to 8 months late; -2 and 0 are not
# documented (often read as "no use" and "minimum paid"). 9 never occurs.
Status = Annotated[int, Field(ge=-2, le=8)]
# NT dollars. A negative bill means the client paid more than they owed.
Bill = Annotated[int, Field(ge=-339_603, le=1_664_089)]
Payment = Annotated[int, Field(ge=0, le=1_684_259)]


class ClientFeatures(BaseModel):
    """The 20 features of one client, from the last six monthly statements."""

    model_config = ConfigDict(extra="forbid")

    EDUCATION: int = Field(ge=0, le=6, description="1 graduate school, 2 university, 3 high school, 4 others (0, 5, 6 undocumented).")
    PAY_1: Status = Field(description="Repayment status in the last month (September).")
    PAY_2: Status
    PAY_3: Status
    PAY_4: Status
    PAY_5: Status
    PAY_6: Status = Field(description="Repayment status six months ago (April).")
    LIMIT_BAL: int = Field(ge=10_000, le=1_000_000, description="Credit limit in NT dollars.")
    BILL_AMT1: Bill = Field(description="Bill amount in the last month, NT dollars.")
    BILL_AMT2: Bill
    BILL_AMT3: Bill
    BILL_AMT4: Bill
    BILL_AMT5: Bill
    BILL_AMT6: Bill
    PAY_AMT1: Payment = Field(description="Amount paid in the last month, NT dollars.")
    PAY_AMT2: Payment
    PAY_AMT3: Payment
    PAY_AMT4: Payment
    PAY_AMT5: Payment
    PAY_AMT6: Payment

The file continues with the example client, the request model with its ID, the warning for unseen codes, and the table for the pipeline. Add this below:

default_risk/contract.py (continued)

EXAMPLE = {
    "client_id": "C-000002",
    "EDUCATION": 2,
    "PAY_1": -1, "PAY_2": 2, "PAY_3": 0, "PAY_4": 0, "PAY_5": 0, "PAY_6": 2,
    "LIMIT_BAL": 120000,
    "BILL_AMT1": 2682, "BILL_AMT2": 1725, "BILL_AMT3": 2682, "BILL_AMT4": 3272, "BILL_AMT5": 3455, "BILL_AMT6": 3261,
    "PAY_AMT1": 0, "PAY_AMT2": 1000, "PAY_AMT3": 1000, "PAY_AMT4": 1000, "PAY_AMT5": 0, "PAY_AMT6": 2000,
}


class ClientIn(ClientFeatures):
    """A client to score: the client's ID and features."""

    model_config = ConfigDict(extra="forbid", json_schema_extra={"examples": [EXAMPLE]})

    client_id: str = Field(pattern=r"^C-\d{6}$", description="The client's ID, for example C-000002.")


def unseen_values(client: ClientFeatures, known: dict[str, list[int]]) -> list[str]:
    """Warnings for status codes inside the contract that the model never saw for that month.

    They are allowed: the encoder scores them like its group of rare codes.
    But the score is less certain, so the answer says so.
    """
    return [
        f"{name}: {getattr(client, name)} is a value the model never saw"
        for name in STATUS
        if getattr(client, name) not in known[name]
    ]


def to_frame(clients: list[ClientFeatures]) -> pd.DataFrame:
    """The clients as a table with the training columns, in the training order."""
    rows = [c.model_dump(include=set(FEATURES)) for c in clients]
    return pd.DataFrame(rows, columns=FEATURES).astype("int64")

Three rules in this file are decisions, not code:

• No value may be missing. The data has no missing values, so the model never learned what one means. A missing key or a null is rejected.
• No extra field. extra="forbid" rejects an AGE field. If the call centre tool sends personal data that the model must not use, the request fails, and somebody fixes the tool.
• Inside the range but never seen: a warning. In the data, PAY_5 and PAY_6 never have the code 1. The contract allows it (1 is a real status), and the answer warns about it.

Step 2: try it

Start Python again and paste:

from pydantic import ValidationError
from default_risk.contract import EXAMPLE, ClientIn

print(ClientIn(**EXAMPLE).LIMIT_BAL)
for change in [{"PAY_1": 12}, {"EDUCATION": 9}, {"LIMIT_BAL": 120}, {"AGE": 35}]:
    try:
        ClientIn(**{**EXAMPLE, **change})
    except ValidationError as error:
        first = error.errors()[0]
        print(change, "->", first["loc"][0], first["msg"])

The output:

120000
{'PAY_1': 12} -> PAY_1 Input should be less than or equal to 8
{'EDUCATION': 9} -> EDUCATION Input should be less than or equal to 6
{'LIMIT_BAL': 120} -> LIMIT_BAL Input should be greater than or equal to 10000
{'AGE': 35} -> AGE Extra inputs are not permitted

The three inputs that the pipeline scored are now refused, each with its field and reason. Run python -m pytest: 13 passed, 4 skipped.

Part 3: one bundle, pinned by its digest

Both paths load one bundle. make_bundle.py is in the starter, as in Bundle model dependencies. Only its parity cases are new: six real clients from the monthly file, and six at the edges of the contract:

make_bundle.py (part)

BASE = {k: v for k, v in EXAMPLE.items() if k != "client_id"}
EDGE_CASES = [
    ("odd code", "C-900001", dict(EDUCATION=5)),
    ("odd code", "C-900002", dict(EDUCATION=0, **{c: 0 for c in FEATURES if c.startswith(("BILL", "PAY_AMT"))},
                                  **{s: -2 for s in STATUS})),
    ("extreme", "C-900003", {c: LOW[c] if c != "EDUCATION" else 1 for c in FEATURES}),
    ("extreme", "C-900004", {c: HIGH[c] if c != "EDUCATION" else 1 for c in FEATURES}),
    ("extreme", "C-900005", dict(BILL_AMT1=-2000)),
    ("unseen", "C-900006", dict(PAY_5=1, PAY_6=1)),
]

Each case tests one rule that a serving path could break:

• an undocumented code that the pipeline must group, and a client with no bills;
• every input at its lowest and highest value (LOW and HIGH come from the contract);
• a negative bill, for the signed log, and a code never seen for that month.

Make the bundle:

python make_bundle.py --version 1.0.0

The output:

Made bundles/default-risk-1.0.0: 12 parity cases, threshold 0.17.
Bundle digest (MODEL_SHA256): da75a5cfd4982660823e22af049358e3359fa1d03b19769d55eadb7d623c9150

Your digest is different, because metadata.json holds the time of your training. (The finished project’s bundle has 078d4b443c1e….) Keep yours: both paths need it. The five files, 16 kB in all:

File	Size	What it holds
model.joblib	7,946 B	The fitted pipeline
contract.json	5,460 B	The features, the JSON schema with the ranges, the known status codes, the threshold and the decision
metadata.json	890 B	Version, training data hash, results, tested package versions, and the SHA-256 of features.py
parity_cases.csv	1,595 B	The 12 cases and their training scores
SHA256SUMS	322 B	The hash of each file above; its own hash is the digest

Run the command again, and it stops: A version is never changed: choose a new version.

default_risk/bundle.py has the six gates of module 1. Gate 3 has one new check: the SHA-256 of features.py must equal the one in metadata.json. Try a wrong digest:

python -c "from default_risk.bundle import load_bundle; load_bundle('bundles/default-risk-1.0.0', '0' * 64)"

The last line of the output:

default_risk.bundle.BundleError: bundles/default-risk-1.0.0: the bundle digest is da75a5cfd498..., but MODEL_SHA256 expects 000000000000...

Gate 1 stopped it before joblib.load ran. Run python -m pytest: 17 passed, 3 skipped.

Part 4: path 1, the monthly batch job

The job follows the rules of Batch inference. It sets a few bad rows aside and scores the others. A broken file fails the job, because a list from it is worse than no list. A crash never leaves half a file.

Bad rows are set aside one by one; a bad file stops the whole job. Every file appears whole or not at all.

Text description of the diagram

The job starts with the monthly file and MODEL_SHA256. If the bundle fails one of the six gates, the job stops with exit code 2 and writes nothing. Otherwise it reads 5,000 rows at a time. Each row that does not match the contract is set aside with its row number, client ID and problem. The good rows of the chunk are scored in one call. When all rows are read, the job checks the share of rows set aside. More than 1%: the job fails with exit code 1 and writes no scores.csv. Otherwise it writes scores.csv.partial and renames it to scores.csv. In both cases, it writes summary.json with the status, the counts, the hashes and the times.

Why 1%? The September file has no bad rows. More than 300 bad rows in 30,000 means that the file changed, not 300 clients.

Step 1: write the batch job

Make the file default_risk/batch.py. The first half reads and checks the rows:

default_risk/batch.py

"""The monthly batch job: score every client in a file with the same bundle and contract as the API.

    python -m default_risk.batch data/clients-2005-09.csv out/2005-09

Writes three files into the output folder:

    scores.csv      client_id, score, flag, model_version (one row per good client)
    rejected.csv    row, client_id, problem (one row per client that fails the contract)
    summary.json    what was read and written (with SHA-256 hashes), with which model, and how long it took

A bad row does not stop the job: it is set aside in rejected.csv, and the
other rows are scored. But if more than --max-rejected of the rows are bad,
the file itself is probably broken, so the job fails (exit code 1) and writes
no scores.csv. Every file is written under a temporary name and renamed at
the end, so a reader never sees half a file.
"""

import argparse
import hashlib
import json
import sys
import time
from datetime import datetime, timezone
from pathlib import Path

import pandas as pd
from pydantic import ValidationError

from default_risk.bundle import BundleError, load_bundle
from default_risk.contract import FEATURES, ClientIn
from default_risk.scoring import score_clients
from default_risk.settings import SettingsError, load_settings

try:
    import resource  # not on Windows
except ImportError:
    resource = None

COLUMNS = ["client_id"] + FEATURES


def problem(error: ValidationError) -> str:
    first = error.errors()[0]
    where = ".".join(str(p) for p in first["loc"]) or "row"
    return f"{where}: {first['msg']}"


def check_rows(chunk: pd.DataFrame, first_row: int):
    good, bad = [], []
    for offset, row in enumerate(chunk.to_dict("records")):
        values = {k: v for k, v in row.items() if not pd.isna(v)}  # an empty cell is a missing value
        try:
            good.append(ClientIn(**values))
        except ValidationError as error:
            bad.append({"row": first_row + offset, "client_id": values.get("client_id"), "problem": problem(error)})
    return good, bad


def sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def write_atomic(path: Path, write) -> None:
    """Write to NAME.partial, then rename: the rename replaces the old file in one step."""
    temporary = path.with_name(path.name + ".partial")
    write(temporary)
    temporary.replace(path)


def peak_memory_mb() -> float | None:
    if resource is None:
        return None
    peak = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss  # KiB on Linux, bytes on macOS
    return round(peak / (1024 * 1024 if sys.platform == "darwin" else 1024), 1)

The job reads every cell as text (dtype=str), and the contract converts it, so "2" in the file and 2 in a JSON request become the same number. The second half runs the job:

default_risk/batch.py (continued)

def main() -> int:
    parser = argparse.ArgumentParser(description="Score a CSV file of clients.")
    parser.add_argument("input", type=Path)
    parser.add_argument("output", type=Path, help="a folder for scores.csv, rejected.csv and summary.json")
    parser.add_argument("--chunk", type=int, default=5000, help="clients per chunk")
    parser.add_argument("--max-rejected", type=float, default=0.01, help="the largest share of bad rows")
    args = parser.parse_args()

    started_at = datetime.now(timezone.utc)
    started = time.perf_counter()
    try:
        settings = load_settings()
        bundle = load_bundle(settings.model_bundle, settings.model_sha256)
        reader = pd.read_csv(args.input, usecols=COLUMNS, dtype=str, keep_default_na=False,
                             na_values=[""], chunksize=args.chunk)
    except (SettingsError, BundleError, ValueError, OSError) as error:
        print(f"Stopped: {error}", file=sys.stderr)
        return 2
    seconds = {"load": time.perf_counter() - started, "check": 0.0, "score": 0.0}

    scored, rejected, rows = [], [], 0
    for chunk in reader:
        step = time.perf_counter()
        good, bad = check_rows(chunk, first_row=rows + 2)  # row 1 is the header
        seconds["check"] += time.perf_counter() - step
        rows += len(chunk)
        rejected += bad
        if good:
            step = time.perf_counter()
            results, _ = score_clients(bundle, good)
            scored += [{"client_id": c.client_id, "score": r.score, "flag": int(r.flag),
                        "model_version": bundle.version} for c, r in zip(good, results)]
            seconds["score"] += time.perf_counter() - step

    step = time.perf_counter()
    args.output.mkdir(parents=True, exist_ok=True)
    scores = pd.DataFrame(scored, columns=["client_id", "score", "flag", "model_version"])
    rejects = pd.DataFrame(rejected, columns=["row", "client_id", "problem"])
    write_atomic(args.output / "rejected.csv", lambda p: rejects.to_csv(p, index=False))
    failed = rows == 0 or len(rejected) / rows > args.max_rejected
    if not failed:
        write_atomic(args.output / "scores.csv", lambda p: scores.to_csv(p, index=False))
    seconds["write"] = time.perf_counter() - step
    total = time.perf_counter() - started

    summary = {
        "status": "failed: too many rejected rows" if failed else "done",
        "input": str(args.input),
        "input_sha256": sha256(args.input),
        "rows": rows,
        "scored": 0 if failed else len(scores),
        "rejected": len(rejected),
        "flagged": None if failed else int(scores["flag"].sum()),
        "flag_rate": None if failed else round(float(scores["flag"].mean()), 3),
        "model_version": bundle.version,
        "bundle_sha256": bundle.digest,
        "scores_sha256": None if failed else sha256(args.output / "scores.csv"),
        "started_at": started_at.isoformat(timespec="seconds"),
        "seconds": round(total, 2),
        "steps_seconds": {k: round(v, 2) for k, v in seconds.items()},
        "rows_per_second": round(rows / total),
        "peak_memory_mb": peak_memory_mb(),
    }
    write_atomic(args.output / "summary.json",
                 lambda p: p.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8"))

    if failed:
        print(f"Failed: {len(rejected)} of {rows} rows were rejected (the limit is {args.max_rejected:.0%}). "
              f"No scores.csv was written. See {args.output / 'rejected.csv'}.", file=sys.stderr)
        return 1
    print(f"Read {rows} rows: scored {len(scores)}, rejected {len(rejected)}, flagged {summary['flagged']} "
          f"({summary['flag_rate']}). Model {bundle.version}. {summary['seconds']} s, "
          f"{summary['rows_per_second']} rows/s.")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Step 2: run it on every client

The job needs your digest. Put it in the environment variable MODEL_SHA256 of this terminal:

macOS

export MODEL_SHA256=$(shasum -a 256 bundles/default-risk-1.0.0/SHA256SUMS | cut -c1-64)

Windows

$env:MODEL_SHA256 = (Get-FileHash bundles\default-risk-1.0.0\SHA256SUMS).Hash.ToLower()

Linux

export MODEL_SHA256=$(sha256sum bundles/default-risk-1.0.0/SHA256SUMS | cut -c1-64)

Then run the job:

python -m default_risk.batch data/clients-2005-09.csv out/2005-09

The output:

Read 30000 rows: scored 30000, rejected 0, flagged 11710 (0.39). Model 1.0.0. 1.28 s, 23517 rows/s.

Open out/2005-09/summary.json:

out/2005-09/summary.json

{
  "status": "done",
  "input": "data/clients-2005-09.csv",
  "input_sha256": "971006c66c955c0c5b72c0e90b116f9830df8e699d428dfccc576a6cf8d805cd",
  "rows": 30000,
  "scored": 30000,
  "rejected": 0,
  "flagged": 11710,
  "flag_rate": 0.39,
  "model_version": "1.0.0",
  "bundle_sha256": "da75a5cfd4982660823e22af049358e3359fa1d03b19769d55eadb7d623c9150",
  "scores_sha256": "8cae5c7a9b4130fff5027ef91980cdf12c9288305a64b890d9baf68527766bbe",
  "started_at": "2026-10-08T14:57:13+00:00",
  "seconds": 1.28,
  "steps_seconds": {
    "load": 0.7,
    "check": 0.35,
    "score": 0.15,
    "write": 0.03
  },
  "rows_per_second": 23517,
  "peak_memory_mb": 237.1
}

• The call list has 11,710 clients (39.0%), close to the 38.2% of the test set.
• Checking the rows took longer than scoring them (0.35 s against 0.15 s). The model is cheap; the fixed costs are not.
• Every number can be traced: the input and the output by their hashes, the model by its digest.
• The peak memory, 237 MB, sets the container limit in Part 7.

Run it again into a new folder, and compare the hashes:

python -m default_risk.batch data/clients-2005-09.csv out/2005-09-again

The two summaries (only the lines with scores_sha256):

out/2005-09/summary.json:  "scores_sha256": "8cae5c7a9b4130fff5027ef91980cdf12c9288305a64b890d9baf68527766bbe",
out/2005-09-again/summary.json:  "scores_sha256": "8cae5c7a9b4130fff5027ef91980cdf12c9288305a64b890d9baf68527766bbe",

The same input and bundle gave the same file, byte for byte, so a re-run after a crash is safe.

The partial failure is in the tests. test_a_bad_row_is_set_aside writes 200 clients with PAY_1 set to "late" in one of them. It checks that 199 are scored, and that rejected.csv holds row 7, client C-000006, and the problem PAY_1: Input should be a valid integer, unable to parse string as an integer. test_too_many_bad_rows_fail_the_job checks exit code 1, no scores.csv, and no .partial file left behind. Run python -m pytest: 20 passed, 2 skipped.

Part 5: path 2, the online endpoint

The agents wait, so the slow work (checking and loading the bundle) happens once at start-up, as in Load and initialize. Each request only checks and scores one client.

Step 1: the routes

Make the file default_risk/routes.py:

default_risk/routes.py

import json
import logging

from fastapi import APIRouter, Depends, Request, Response
from pydantic import BaseModel, Field

from default_risk.contract import ClientIn
from default_risk.errors import ErrorResponse
from default_risk.scoring import score_clients
from default_risk.security import require_api_key

predictions = logging.getLogger("default_risk.predictions")

router = APIRouter(prefix="/v1", dependencies=[Depends(require_api_key)])

ERRORS = {
    401: {"model": ErrorResponse, "description": "The API key is missing or wrong."},
    422: {"model": ErrorResponse, "description": "The client does not match the contract."},
}


class ScoreOut(BaseModel):
    client_id: str
    score: float = Field(ge=0, le=1, description="0 to 1. Higher means a higher risk of missing the next payment.")
    flag: bool = Field(description="True: offer the client an early-help plan.")
    threshold: float = Field(description="The score at and above which a client is flagged.")
    model_version: str
    warnings: list[str] = Field(description="Values the model never saw. The score is less certain.")


class ModelInfo(BaseModel):
    model_version: str
    bundle_sha256: str
    threshold: float
    trained_at: str
    features: list[str]


@router.post("/score", tags=["scoring"], summary="Score one client now", responses=ERRORS)
def score(client: ClientIn, request: Request, response: Response) -> ScoreOut:
    bundle = request.app.state.bundle
    [result], times = score_clients(bundle, [client])
    response.headers["Server-Timing"] = ", ".join(f"{k};dur={v:.2f}" for k, v in times.items())
    predictions.info(json.dumps({"request_id": request.state.request_id, "client_id": client.client_id,
                                 "model_version": bundle.version, "score": result.score, "flag": result.flag}))
    return ScoreOut(client_id=client.client_id, score=result.score, flag=result.flag,
                    threshold=bundle.threshold, model_version=bundle.version, warnings=result.warnings)


@router.get("/model", tags=["model"], summary="Which model answers, and how to check it")
def model(request: Request) -> ModelInfo:
    bundle = request.app.state.bundle
    return ModelInfo(model_version=bundle.version, bundle_sha256=bundle.digest, threshold=bundle.threshold,
                     trained_at=bundle.metadata["trained_at"], features=bundle.contract["features"])

score_clients is the function that the batch job calls, here with a list of one client. That one line is the reason why the two paths can agree.

Step 2: the start-up

Replace default_risk/main.py with this file. It is the course’s main.py without the jobs, the shadow model and the monitor:

default_risk/main.py

"""The online path: the credit-default model behind an HTTP API for the call centre.

Start-up does the slow work once: read the settings, check and load the
bundle, score the parity cases. Each request does only the fast work: check
the client, score it, answer with the model version.
"""

import logging
import time
import uuid
from contextlib import asynccontextmanager

from fastapi import FastAPI, Request

from default_risk.bundle import BundleError, load_bundle
from default_risk.errors import add_error_handlers, error_response
from default_risk.routes import router
from default_risk.settings import Settings, SettingsError, load_settings

MAX_BODY_BYTES = 16_384  # one client is about 320 bytes of JSON

logger = logging.getLogger("default_risk")


def create_app(settings: Settings | None = None) -> FastAPI:
    settings = settings or load_settings()
    logging.basicConfig(level=settings.log_level, format="%(levelname)s %(name)s: %(message)s")

    @asynccontextmanager
    async def lifespan(app: FastAPI):
        started = time.perf_counter()
        try:
            app.state.bundle = load_bundle(settings.model_bundle, settings.model_sha256)
        except BundleError as error:
            logger.error("Start-up error: %s", error)
            raise
        logger.info("model %s loaded (bundle %s...) in %.0f ms", app.state.bundle.version,
                    app.state.bundle.digest[:12], (time.perf_counter() - started) * 1000)
        yield

    app = FastAPI(
        title="Credit Default Service",
        version="1.0.0",
        description="Scores a credit-card client for the risk of missing the next payment. "
                    "For early-help offers only: not for refusing credit, limits or prices.",
        lifespan=lifespan,
        docs_url="/docs" if settings.show_docs else None,
        redoc_url=None,
        openapi_url="/openapi.json" if settings.show_docs else None,
    )
    app.state.settings = settings

    @app.middleware("http")
    async def request_id_and_limits(request: Request, call_next):
        request.state.request_id = uuid.uuid4().hex[:12]
        started = time.perf_counter()
        length = request.headers.get("content-length")
        if length and int(length) > MAX_BODY_BYTES:
            response = error_response(request, 413, "body_too_large", f"The body is larger than {MAX_BODY_BYTES} bytes.")
        else:
            response = await call_next(request)
        response.headers["X-Request-ID"] = request.state.request_id
        logger.info("%s %s %d %.1fms id=%s", request.method, request.url.path, response.status_code,
                    (time.perf_counter() - started) * 1000, request.state.request_id)
        return response

    add_error_handlers(app)
    app.include_router(router)

    @app.get("/health", tags=["health"])
    def health() -> dict:
        return {"status": "ok"}

    @app.get("/ready", tags=["health"])
    def ready(request: Request) -> dict:
        bundle = getattr(request.app.state, "bundle", None)
        if bundle is None:
            return error_response(request, 503, "not_ready", "The model is not loaded.")
        return {"status": "ready", "model_version": bundle.version}

    return app


try:
    app = create_app()
except SettingsError as error:
    raise SystemExit(f"Start-up error: {error}") from None

Step 3: call it

Start the API in the terminal where MODEL_SHA256 is set:

First terminal

fastapi dev

The start-up line in its log:

INFO default_risk: model 1.0.0 loaded (bundle da75a5cfd498...) in 968 ms

Open a second terminal (Terminal › New Terminal) and send the example client. On Windows, type curl.exe instead of curl.

Second terminal

curl -i -X POST http://127.0.0.1:8000/v1/score -H "Content-Type: application/json" -d @requests/client.json

The output:

HTTP/1.1 200 OK
date: Thu, 08 Oct 2026 14:57:39 GMT
server: uvicorn
content-length: 119
content-type: application/json
server-timing: frame;dur=0.76, prepare;dur=3.60, model;dur=0.17
x-request-id: ceba7eac2a28

{"client_id":"C-000002","score":0.35052096497827656,"flag":true,"threshold":0.17,"model_version":"1.0.0","warnings":[]}

The score equals the batch job’s for C-000002. In server-timing, the model took 0.17 ms and the preprocessing 3.6 ms: again, the model is not the slow part.

Now send the two other request bodies: a client with the unseen code PAY_5 = 1, and a client with a limit of 120 and the status "late":

Second terminal

curl -X POST http://127.0.0.1:8000/v1/score -H "Content-Type: application/json" -d @requests/unseen-code.json
curl -X POST http://127.0.0.1:8000/v1/score -H "Content-Type: application/json" -d @requests/outside-contract.json

The output:

{"client_id":"C-000002","score":0.33316541823108353,"flag":true,"threshold":0.17,"model_version":"1.0.0","warnings":["PAY_5: 1 is a value the model never saw"]}
{"error":{"code":"invalid_client","message":"The client does not match the model's contract.","request_id":"9fa0a91dca39","fields":["LIMIT_BAL","PAY_1"]}}

The unseen code is scored with a warning. The bad client gets 422 with both fields; the batch job would put it in rejected.csv. One contract, two ways to report.

The log has one prediction line per score, for example INFO default_risk.predictions: {"request_id": "ceba7eac2a28", "client_id": "C-000002", "model_version": "1.0.0", "score": 0.35052096497827656, "flag": true}, to join with the outcomes later, as in Monitor model health.

Run python -m pytest: 37 passed. All the tests run now, including tests/test_parity.py, which you meet in the next part. Keep fastapi dev running.

Part 6: prove that the two paths agree

Jia-Hui asks: “The call list says 0.35 for this client. Will my agent’s screen say 0.35 too?” Not always: the screen sends today’s record, which may have a new payment. That is not a fault. A fault is skew: the same record, scored differently by the two paths.

So compare the paths on fixed records. Each of the 12 parity cases must get its expected_score from both paths, within 1e-9, as in Check parity.

Step 1: the check script

Make the folder parity and the file parity/check_paths.py:

parity/check_paths.py

"""Compare the two serving paths with training on the bundle's parity cases.

    python parity/check_paths.py http://127.0.0.1:8000 bundles/default-risk-1.0.0 out/parity/scores.csv

The batch scores come from a run of the batch job on the parity cases:

    python -m default_risk.batch bundles/default-risk-1.0.0/parity_cases.csv out/parity

The online scores come from POST /v1/score on the running API. Each must
equal expected_score, the score that training gave the same client, within
the tolerance; so the batch and online scores also equal each other.
"""

import argparse
import json
import os
import sys
import urllib.request
from pathlib import Path

import pandas as pd

TOLERANCE = 1e-9


def online_score(url: str, client: dict, key: str | None) -> float:
    request = urllib.request.Request(
        f"{url}/v1/score",
        data=json.dumps(client).encode(),
        headers={"Content-Type": "application/json", **({"X-API-Key": key} if key else {})},
    )
    with urllib.request.urlopen(request, timeout=10) as response:
        return json.load(response)["score"]


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("url")
    parser.add_argument("bundle", type=Path)
    parser.add_argument("batch_scores", type=Path, help="scores.csv of a batch run on the parity cases")
    parser.add_argument("--tolerance", type=float, default=TOLERANCE)
    args = parser.parse_args()

    cases = pd.read_csv(args.bundle / "parity_cases.csv")
    batch = pd.read_csv(args.batch_scores).set_index("client_id")["score"]
    key = os.environ.get("API_KEY")
    failed = 0
    print(f"{'case':9} {'client':9} {'training':>9} {'batch':>9} {'online':>9} {'largest diff':>12}")
    for row in cases.to_dict("records"):
        expected, case = row.pop("expected_score"), row.pop("case")
        online = online_score(args.url, {k: (v.item() if hasattr(v, "item") else v) for k, v in row.items()}, key)
        batch_score = float(batch[row["client_id"]])
        largest = max(abs(batch_score - expected), abs(online - expected), abs(batch_score - online))
        ok = largest <= args.tolerance
        failed += not ok
        print(f"{case:9} {row['client_id']:9} {expected:9.6f} {batch_score:9.6f} {online:9.6f} {largest:12.1e}"
              f"{'' if ok else '  FAIL'}")
    print(f"{len(cases) - failed} of {len(cases)} cases match on both paths within {args.tolerance:g}.")
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(main())

The batch job reads only the columns of the contract, so it can read parity_cases.csv directly: it ignores case and expected_score.

Step 2: run both paths on the cases

In the second terminal, set MODEL_SHA256 again (the command of Part 4, Step 2), then:

Second terminal

python -m default_risk.batch bundles/default-risk-1.0.0/parity_cases.csv out/parity
python parity/check_paths.py http://127.0.0.1:8000 bundles/default-risk-1.0.0 out/parity/scores.csv

The output:

Read 12 rows: scored 12, rejected 0, flagged 6 (0.5). Model 1.0.0. 0.71 s, 17 rows/s.
case      client     training     batch    online largest diff
normal    C-000001   0.761597  0.761597  0.761597      0.0e+00
normal    C-000002   0.350521  0.350521  0.350521      5.6e-17
normal    C-000003   0.123973  0.123973  0.123973      2.8e-17
normal    C-000004   0.142994  0.142994  0.142994      0.0e+00
normal    C-000005   0.178072  0.178072  0.178072      1.1e-16
normal    C-000006   0.142699  0.142699  0.142699      8.3e-17
odd code  C-900001   0.147783  0.147783  0.147783      0.0e+00
odd code  C-900002   0.094542  0.094542  0.094542      6.9e-17
extreme   C-900003   0.173154  0.173154  0.173154      2.8e-17
extreme   C-900004   0.043804  0.043804  0.043804      4.2e-17
extreme   C-900005   0.433764  0.433764  0.433764      0.0e+00
unseen    C-900006   0.316648  0.316648  0.316648      0.0e+00
12 of 12 cases match on both paths within 1e-09.

• The differences are not zero, even on one computer. The batch job adds the numbers in a different order than 12 one-row requests. This is why parity uses a tolerance.
• 1e-9 still catches real skew. A path that skips group_education moved C-900001 from 0.148 to 0.309: more than 100 million times the tolerance.
• The extreme cases test the arithmetic, not the meaning. C-900004 is 8 months late everywhere but scores 0.044, because its payments are also at their maximum. No real client looks like this.

tests/test_parity.py runs the same comparison inside pytest, with the batch job as a subprocess and the API through FastAPI’s test client. So a change that breaks parity fails the tests before it reaches a container. Stop fastapi dev with Ctrl+C.

Part 7: one image, two commands, with limits

The API is always on; the batch job runs once a month and stops. They still need the same code and bundle, so use one image with two commands and the limits of Container resource settings.

Step 1: the Dockerfile and Compose file

Make the file Dockerfile. It is the course’s Dockerfile with default_risk instead of escalation:

Dockerfile

FROM python:3.14.8-slim-trixie

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PIP_ROOT_USER_ACTION=ignore

RUN useradd --create-home --uid 10001 app

WORKDIR /app

COPY requirements-run.txt .
RUN pip install -r requirements-run.txt

COPY bundles ./bundles
COPY pyproject.toml .
COPY default_risk ./default_risk

USER app

EXPOSE 8000
CMD ["fastapi", "run", "--host", "0.0.0.0", "--port", "8000"]

The .dockerignore of the starter sends only requirements-run.txt, pyproject.toml, default_risk/ and bundles/ to the builder. The data, model/ and .env never go into the image.

Make the file compose.yaml:

compose.yaml

# One image, two ways to use it: the online API (always on) and the monthly
# batch job (runs, then stops). Both get the same bundle and the same digest
# from .env (copy .env.example), and the resource limits of a small cloud
# container.
services:
  api:
    image: ${IMAGE:-credit-default-service:1.0.0}
    build: .
    ports:
      - "127.0.0.1:${PORT:-8000}:8000"
    environment: &model
      MODEL_BUNDLE: bundles/default-risk-${MODEL_VERSION:-1.0.0}
      MODEL_SHA256: ${MODEL_SHA256:?Set MODEL_SHA256 in .env}
    cpus: 1.0
    mem_limit: 512m
    memswap_limit: 512m  # the same value: no swap, as in most cloud containers
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/ready')"]
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 10s

  batch:
    image: ${IMAGE:-credit-default-service:1.0.0}
    profiles: ["batch"]  # not started by "up": run it with "docker compose run --rm batch IN OUT"
    environment: *model
    entrypoint: ["python", "-m", "default_risk.batch"]
    volumes:
      - ./data:/app/data:ro
      - ./out:/app/out
    cpus: 1.0
    mem_limit: 512m
    memswap_limit: 512m

Line	Why
&model and *model	YAML names the block and uses it again: the two services cannot get different bundles by mistake.
profiles: ["batch"]	up does not start the job; a scheduler, or you, starts it with docker compose run.
./data:/app/data:ro	The job may read the monthly file, never change it.
512 MiB, no swap	The job’s peak was 237 MB and the API uses about 150 MiB: room for each.

Step 2: start the API and run the job

Copy the example settings file, then paste your digest after MODEL_SHA256= in .env:

macOS

cp .env.example .env

Windows

Copy-Item .env.example .env

Linux

cp .env.example .env

Build the image, start the API and wait until it is healthy. Then read its start-up line:

docker compose up -d --build --wait
docker compose logs api

The first build took 40 seconds. The command ends when the container is Healthy. The start-up line:

api-1  | INFO default_risk: model 1.0.0 loaded (bundle da75a5cfd498...) in 825 ms

Now run the monthly job in the same image, then the parity check against the API in its container:

docker compose run --rm batch data/clients-2005-09.csv out/2005-09-container
docker compose run --rm batch bundles/default-risk-1.0.0/parity_cases.csv out/parity-container
python parity/check_paths.py http://127.0.0.1:8000 bundles/default-risk-1.0.0 out/parity-container/scores.csv

The output (the job lines, and the last line of the check):

Read 30000 rows: scored 30000, rejected 0, flagged 11710 (0.39). Model 1.0.0. 1.77 s, 16963 rows/s.
Read 12 rows: scored 12, rejected 0, flagged 6 (0.5). Model 1.0.0. 0.7 s, 17 rows/s.
12 of 12 cases match on both paths within 1e-09.

Both paths agree in the containers too. But out/2005-09-container/summary.json has the scores_sha256 4afd711ef9d7…, not 8cae5c7a9b41…. Linux and macOS differ in the last digit of some scores. So compare file hashes only between runs of the same image (two runs in the container gave the same hash). Across computers, compare the scores with a tolerance.

Step 3: measure both paths

The batch job measures itself in summary.json. For the API, use the course’s benchmark, as in Benchmark the service: monthly clients, one per request, after 50 warm-up requests. Then read the container’s memory:

python bench/bench.py http://127.0.0.1:8000 --requests 1000 --concurrency 1
docker stats --no-stream

For the charts on this page, we ran it with --concurrency 1, --concurrency 4, and once for all clients (--requests 30000 --warmup 0), while other containers ran on the computer. The three result lines:

1000 requests, concurrency 1: p50 6.1 ms, p95 13.7 ms, p99 25.7 ms, max 54.2 ms, 136 requests/s, 0 errors.
1000 requests, concurrency 4: p50 19.2 ms, p95 71.8 ms, p99 103.5 ms, max 249.1 ms, 158 requests/s, 0 errors.
30000 requests, concurrency 1: p50 4.7 ms, p95 13.5 ms, p99 27.2 ms, max 339.7 ms, 160 requests/s, 0 errors.

docker stats showed 152.1 MiB of the 512 MiB limit for the API.

Predict before you look: Jia-Hui’s budget is 200 ms. How many of the 1,000 answers took more than 30 ms?

1,000 measured answers of the API in its container (1 CPU, 512 MiB), one request at a time. 650 of them took 4 to 7 ms. Eight answers took 30 ms or more and are not drawn; the slowest took 54.2 ms.

Text description of the chart

A histogram of 1,000 answer times in bins of 1 millisecond, from 0 to 30 ms. No answer is faster than 3.7 ms. 650 answers fall between 4 and 7 ms, and the count falls off to the right, with a thin tail past 10 ms (113 answers). A dashed line marks the median, p50, at 6.1 ms, and a second line marks p95 at 13.7 ms. Eight answers of 30 ms or more are not drawn; the slowest took 54.2 ms. All 1,000 answers are far below the call centre’s budget of 200 ms.

• The budget is not the risk. p95 is 13.7 ms against 200 ms; even the slowest answer fits.
• More requests at once only wait longer. At concurrency 4, p50 rose from 6.1 to 19.2 ms, but throughput only from 136 to 158 requests per second. One worker on 1 CPU is busy, so requests queue, as in module 4.
• Benchmarks vary. An earlier run, at a quieter moment, gave p95 7.9 ms and 210 requests per second. Record the conditions with every result.

We did not measure a cloud platform or the call centre’s network, so we claim no number for them.

Leave the containers running for Part 8.

Part 8: a failure: the new export

Hao’s team moves the export to a new data warehouse. Before the switch, Hao sends a test export of September from the new system. Nobody notices that the new view stores the credit limit in thousands of NT dollars: 20,000 is now 20. The script can make the same file:

python scripts/export_month.py --limit-in-thousands data/clients-2005-09-new-export.csv

Predict before you look: what does the batch job do with this file? And what would the model do without the contract?

First, the model alone. Start Python and paste:

import joblib
import pandas as pd
from default_risk.contract import FEATURES

pipeline = joblib.load("model/model.joblib")
for name in ["data/clients-2005-09.csv", "data/clients-2005-09-new-export.csv"]:
    scores = pipeline.predict_proba(pd.read_csv(name)[FEATURES])[:, 1]
    print(f"{name}: {(scores >= 0.17).sum()} of {len(scores)} flagged ({(scores >= 0.17).mean():.1%})")

The output:

data/clients-2005-09.csv: 11710 of 30000 flagged (39.0%)
data/clients-2005-09-new-export.csv: 29498 of 30000 flagged (98.3%)

Without a contract, the call list would have 29,498 clients instead of 11,710. Now the batch job, in its container:

docker compose run --rm batch data/clients-2005-09-new-export.csv out/2005-09-new-export

The output:

Failed: 30000 of 30000 rows were rejected (the limit is 1%). No scores.csv was written. See out/2005-09-new-export/rejected.csv.

The exit code is 1, so a scheduler marks the run as failed. The folder has only rejected.csv and summary.json. The first lines of rejected.csv:

row,client_id,problem
2,C-000001,LIMIT_BAL: Input should be greater than or equal to 10000
3,C-000002,LIMIT_BAL: Input should be greater than or equal to 10000

Common mistake

The monthly job fails: 30,000 of 30,000 rows rejected, LIMIT_BAL below 10000

Symptom. The batch job exits with code 1 and writes no scores.csv. Every line of rejected.csv has the same problem: LIMIT_BAL: Input should be greater than or equal to 10000. summary.json has "status": "failed: too many rejected rows".

Cause. The export changed its unit: the credit limit is in thousands. The model has no way to know this. It would have flagged 98.3% of clients. The contract’s lower limit of 10,000, the smallest limit in the data, caught it.

Fix. Fix the export, not the contract. Hao’s team multiplies the limit by 1,000 in the new view and sends a new test file. The job runs again, into a new folder. Do not lower the contract’s limit to make the file pass. The contract describes what the model learned, and the model never saw a limit of 20 dollars.

What it means for a real AI service. A batch job must fail completely, and visibly, when its input changes, and it must keep the last good output. Here, the September call list in out/2005-09/ is untouched. But a contract catches only values outside its ranges. If the new view had stored the bills in thousands, every bill would still be inside its range, and the job would have run. The second defence is the flag rate in summary.json: compare it with last month’s (39.0%) before the calls start.

Stop the API and remove the image when you finish:

docker compose down
docker image rm credit-default-service:1.0.0

Our run-through

We ran every step of this page from starter.zip in a new folder with a new virtual environment. We wrote each file as the page shows it. The commands and full outputs are in the course’s repository (reference/c17/case-studies/credit-default/RUNS.md).

Check	Result
Tests after each part	3 (start), 13, 17, 20, then 37 passed; the finished project: 37 passed
Training	Byte for byte the same model.joblib as the course’s bundle (d71c40c4848a…)
Batch job, 30,000 clients	11,710 flagged (39.0%); 1.15 to 1.77 s on the computer and in the container; peak memory 215 to 237 MB
Online API	Healthy after up --wait in about 6 s; model loaded in 0.5 to 1.0 s; about 150 MiB of memory
Parity	12 of 12 cases, on the computer and in the containers; largest difference 1.1e-16
Failure	The changed export: exit code 1, 30,000 rows rejected, no scores.csv
Not tested	Windows and Linux; a scheduler for the monthly run; a cloud deployment

Responsible use

A credit score decides how a bank treats a person, so the limits of this model are part of the product.

Only for a supportive call. The threshold, the costs and the fairness checks of the machine learning case study were made for an offer of help. The model must not refuse credit, lower a limit, set a price, or decide anything on its own. The API’s description and MODEL_CARD.md say so.

A score is not a judgement. A score of 0.35 means “about a third of clients like this one missed the next payment in 2005”. The agent offers help; the score does not label the client.

Less personal data on each path. The monthly file has no sex, marital status or age, and the API rejects them. Data that a service never receives cannot leak from it. But the model can still treat groups differently through other inputs. In the machine learning case study, the model flags young clients more often, mostly through their lower credit limits. The flag rate by age band needs a review before real use, by people qualified to do it.

Logs hold personal data. Each prediction line has a client ID and a score. A real bank keeps such logs under its data rules: who may read them, and for how long.

The data is old and from one place. One bank in Taiwan, 2005, during a period of high card debt. A real bank trains and tests on its own recent data.

What this project does about it:

• The contract refuses what the model never saw.
• Every answer and every summary names the model version and the bundle digest.
• The job fails instead of sending a broken list.
• The batch summary gives a flag rate to check each month.

Make it a portfolio piece

This project shows an employer that you can serve one model in two ways and prove that they agree. Publish it as a repository, under your name.

What to put in the repository:

credit-default-service/
  README.md               what it does, how to run both paths, the data and its licence
  MODEL_CARD.md           intended use, results, what it is not for
  LICENSE                 MIT, for the code (the data keeps CC BY 4.0: see data/README.md)
  data/                   the data file and its attribution
  training/train.py       rebuilds the model byte for byte
  make_bundle.py          makes a versioned bundle with parity cases and SHA256SUMS
  bundles/                your bundle (keep it in Git: it is the released model)
  default_risk/           contract, bundle loader, scoring, batch job, API
  parity/check_paths.py   proves that both paths agree
  bench/bench.py          the benchmark
  tests/                  the 37 tests
  Dockerfile, compose.yaml, .dockerignore, .env.example

1. Start from your project folder, or from the finished project. Check that .gitignore lists .venv/, .env, out/, model/ and data/clients-*.csv.
2. Make the repository: git init, then git add . and git commit -m "Credit default service: batch and online".
3. Check with git ls-files that bundles/default-risk-1.0.0/ and data/credit_default_taiwan.csv.gz are in it, and that .env is not.
4. Push it to GitHub as a public repository. Keep LICENSE and data/README.md: CC BY 4.0 requires the credit and the list of changes.
5. Write the first paragraph of the README in your own words. Add your own measured numbers: the batch run’s time and the API’s p50 and p95, with the conditions.

How to describe it in an interview, in two sentences: “I served a credit-default model as a monthly batch job and an online API from one versioned bundle with a pinned digest, and I proved with fixed parity cases that both paths give the same scores within 1e-9. In containers with 1 CPU and 512 MiB, the batch job scored 30,000 clients in about 1.4 seconds, the API answered at a p95 of 14 ms, and the contract stopped an export with the wrong unit before it became a call list.”

Go further:

1. A month-to-month check. Make the batch job compare its flag rate with the last run’s summary. It warns or fails when the rate moves by more than a set amount. Test it with a file where the bills are in thousands.
2. A scheduler. Run the batch container from a scheduled job on a container platform, and keep each month’s summary.
3. A new version in shadow. Train a 1.1.0 bundle and run the batch job with both bundles on the same file. Compare the call lists before you switch, as in Version and rollout.
4. Reasons on the agent’s screen. Return the client’s three main reasons with the score (the machine learning case study computes them), so that agents can explain the offer. Test that both paths give the same reasons.

Knowledge check

Check your understanding

The call list says 0.35 for client C-000002. During a call two weeks later, the agent's screen shows 0.21. Is this training-serving skew? How do you find out?

Show the answer

Not necessarily. The batch job scored the September statement; the agent’s screen sends today’s record, which may include a new payment or a changed status. A different input gives a different score. To find out, compare the paths on the same input. Run check_paths.py on the parity cases. Or send the client’s September record to the API, and compare the answer with the batch score. If both agree within 1e-9, the paths are fine, and the difference comes from the data. If they do not agree, one path does something that the other does not, and that is skew.

Check your understanding

Hao asks you to lower the contract's minimum for LIMIT_BAL from 10000 to 0, so that the new export passes. What do you answer, and why?

Show the answer

No. The minimum is the smallest limit that the model learned from. A limit of 20 is a value in thousands, not a real limit. The model would score it as a client with almost no credit, and it would flag 98.3% of clients. The contract protects the meaning of the model’s inputs, so the fix belongs in the export, which must deliver NT dollars. A new contract is right only with a new model that was trained on data in that unit, and that is a new bundle version.

Check your understanding

The batch job scores 30,000 clients about 130 times faster than the API does with one request per client, with the same model. Where does the difference come from, and why does the bank still need the API?

Show the answer

Each request pays fixed costs: the HTTP connection, the JSON, the contract check, a one-row table, and the preprocessing of one row. The model itself takes about 0.1 ms. The batch job pays the table and preprocessing costs once per 5,000 clients. So it reaches about 20,000 clients per second, against about 160 for the API. But the batch result is as old as its file. The call centre needs a score from today’s record while the client is on the phone. That needs an answer for one client in milliseconds: the online path.

Recap

• One model can need two paths: a batch job for throughput (every client, on a schedule) and an online endpoint for latency (one client, now). Here the batch job was about 130 times faster for all clients; the API met a 200 ms budget with a p95 of 14 ms.
• Both paths load the same bundle, pinned by its digest outside the bundle, and share the contract and the scoring code. A pipeline that calls your own functions also needs their code: the bundle records the hash of features.py.
• The contract’s ranges come from the data. They stopped a credit limit in thousands, which the model alone would have scored, flagging 98.3% of clients.
• The batch job sets bad rows aside and fails when too many are bad. It writes files whole or not at all, and leaves a summary with hashes, times and the flag rate.
• Parity on fixed cases proves that the paths agree: 12 of 12 within 1e-9. Differences of 1e-16 are normal, even on one computer. So compare scores with a tolerance, and compare the hashes of files only on the same image.
• One image runs both paths with limits (1 CPU, 512 MiB). Record the conditions with each measurement, and claim nothing that you did not measure.

Next, the second case study: a retrained house-price model, shadowed on real requests, promoted and rolled back.

Next: Release a new house-price model