Batch inference

What you will be able to do

Score every new ticket of the day in one scheduled run, so that Grace’s team has its review list each morning. A few bad rows must not stop the run, and a broken file must not produce a list that looks correct.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• The escalation model, its threshold 0.19 and the 20% review capacity, from Thresholds and calibration in Machine Learning: From Problem to Reliable Model.
• How predict.py scored the July file and stopped on bad input, from Plan production checks in the machine learning course.
• The contract (TicketIn, missing key versus null), from Prediction inputs and outputs.
• The bundle and its digest, MODEL_SHA256, from Bundle model dependencies and Artifact trust and integrity.

What you need on your computer

• The project ~/projects/escalation-service (Windows: $HOME\projects\escalation-service) with its .venv, active in a terminal.
• escalation/contract.py and the tests, from Prediction inputs and outputs.
• bundles/escalation-1.0.0/ with SHA256SUMS, and escalation/bundle.py, from Artifact trust and integrity. python -m pytest gives 18 passed.
• The machine learning course’s ~/projects/ticket-model/data/july_tickets.csv. If you do not have it, download july_tickets.csv from that course’s data folder in the labs.
• If your project is missing or broken, copy the snapshot end-of-m01. Use a snapshot has the steps.

Tested with Python 3.14.6 and Docker Desktop (Docker Engine 29.6.1) on macOS, Apple silicon. The Windows and Linux steps are not tested.

The post comes once a day

The post arrives once a day. Nobody waits at the door for each letter: you read them all at breakfast. Much of the work with a model can be done the same way. Collect the cases, score them all at once, and deliver the results before people need them.

Larkfield’s help desk works like this today. Each night at 02:00, Mei’s export writes a file with the tickets of the day before. The model scores the file before the senior team starts at 08:00, and Grace’s team reviews the flagged tickets first. The data in this course is the synthetic Larkfield data from the machine learning course.

Batch: the scores are ready before anyone needs them. The cost is the wait: a ticket can be almost a day old when its flag arrives.

Text description of the diagram

Three actors: the help desk system, the batch job of escalation-service, and the senior team. All day, new tickets arrive without a score. At 02:00, Mei’s nightly file with the tickets of one day goes to the batch job. The job checks every row, scores the tickets in chunks of 500, and writes three files: scores.csv, rejected.csv and summary.json. 1,400 tickets take about 1 second. At 08:00, the senior team opens scores.csv and sees yesterday’s flagged tickets. The last note: a ticket that arrived at 09:05 yesterday gets its flag about 23 hours later.

Look at the last note first. That delay is the price of batch work, and the next lesson removes it. This lesson makes the nightly run safe.

The problem: one bad row, no list

In the machine learning course, predict.py checked the whole file first, and any failed check stopped the run with exit code 1. Nothing was scored. That is safe, but it is too strict for a file that arrives every night.

Count what one typo costs. The July file has 1,400 tickets. Suppose one agent typed urgent in the priority field. That is 1 bad row in 1,400, or 0.07%. With that course’s rule, the other 1,399 tickets get no score, and Grace’s team starts the day with no list at all.

The opposite mistake is worse. Suppose the job does not check, or it crashes after it has written half of scores.csv. Grace’s team opens a file that looks normal, with 700 tickets instead of 1,400. Nobody can see that half of the flags are missing.

Tomás asks: “Why not loop over the file and call the model? It is ten lines.” The ten lines give scores. They do not answer three questions that the morning review depends on:

1. Which rows were not scored, and why?
2. Is the output file complete, or did the run stop halfway?
3. Which model scored this file, and does a second run give the same scores?

The idea: batch scoring

Batch scoring means scoring many cases together in one run, usually on a schedule, and storing the results for later use. Nobody waits for a single answer. So the important number is throughput, the number of cases scored per second, not the time of one case. On the test computer, the job scored the 1,400 July tickets in 0.75 seconds, with the time to load the model. That is about 1,900 tickets per second.

A safe batch job answers the three questions with three rules.

Set bad rows aside, but stop a broken file. Each row goes through the contract from Module 1. A good row is scored. A bad row goes to rejected.csv, with its row number and the problem, and the run continues. But if many rows are bad, the file itself is probably broken, for example after a change in Mei’s export. So the job has a limit: if more than 5% of the rows are bad, it fails with exit code 1 and writes no scores.csv. This is partial failure handling: a small failure stays small, and a large one stops the run.

Never show half a file. The job writes each output under a temporary name, such as scores.csv.partial. At the end, it renames the file to scores.csv. The file system does a rename in one step, so a reader sees either no file or the complete file. This is an atomic write: it happens completely, or not at all.

Record what made the output. Each run writes summary.json. It records the rows scored and rejected, the flag rate, the model version, the bundle digest and the SHA-256 of the input file. Each run also writes into its own folder, such as out/2026-07-31/, so a new run never overwrites an old result.

Partial failure in the batch job. A missing column stops the run at once. Bad rows are set aside. Too many bad rows fail the run, so a broken file never produces a review list.

Text description of the diagram

Mei’s file goes to a first question: is every column there? If not, the run stops with exit code 2, and nothing is written. If every column is there, each row is checked with the contract. Good rows are scored, 500 at a time. Bad rows go to rejected.csv, with the row number, the ticket_id and the problem. Then a second question: are the bad rows at most 5% of the file? If yes, the run is done, with exit code 0, and scores.csv.partial is renamed to scores.csv. If no, the run fails with exit code 1, and no scores.csv exists. summary.json is written in both cases.

Build the batch job

The batch job needs three new files in escalation/. Each one has one job, and the service in Module 3 uses the first two again.

settings.py: the settings from the environment

The job must know which bundle to load and which digest to expect. As in the API and Docker courses, the settings come from environment variables, and a wrong setting stops the program before it does any work. Make escalation/settings.py:

escalation/settings.py

"""Settings from environment variables. A wrong setting stops the service at start."""

import os
from dataclasses import dataclass, field
from pathlib import Path


class SettingsError(Exception):
    """A required setting is missing or not valid. The service must not start."""


@dataclass(frozen=True)
class Settings:
    model_bundle: Path
    model_sha256: str
    api_key: str | None = field(repr=False)
    log_level: str = "INFO"
    show_docs: bool = True
    max_job_tickets: int = 5000
    max_queued_jobs: int = 4
    extra_ms_per_ticket: float = 0.0


def read_secret(name: str) -> str | None:
    """Return the value of NAME, or the contents of the file in NAME_FILE."""
    path = os.environ.get(f"{name}_FILE")
    if path:
        try:
            return Path(path).read_text(encoding="utf-8").strip() or None
        except OSError as error:
            raise SettingsError(f"{name}_FILE: cannot read {path}: {error.strerror}")
    return os.environ.get(name) or None


def load_settings() -> Settings:
    env = os.environ
    sha = env.get("MODEL_SHA256", "").strip().lower()
    if len(sha) != 64:
        raise SettingsError("MODEL_SHA256 must be the 64-character digest of the bundle")
    settings = Settings(
        model_bundle=Path(env.get("MODEL_BUNDLE", "bundles/escalation-1.0.0")),
        model_sha256=sha,
        api_key=read_secret("API_KEY"),
        log_level=env.get("LOG_LEVEL", "INFO").upper(),
        show_docs=env.get("SHOW_DOCS", "true").lower() == "true",
        max_job_tickets=int(env.get("MAX_JOB_TICKETS", "5000")),
        max_queued_jobs=int(env.get("MAX_QUEUED_JOBS", "4")),
        extra_ms_per_ticket=float(env.get("EXTRA_MS_PER_TICKET", "0")),
    )
    if env.get("REQUIRE_API_KEY", "false").lower() == "true" and not settings.api_key:
        raise SettingsError("REQUIRE_API_KEY is true, but API_KEY is not set")
    return settings

The batch job uses only the first two settings. MODEL_BUNDLE has a default. MODEL_SHA256 has none, because the digest must come from you, not from the bundle. The API key and the documentation settings are for the HTTP service in Module 3. The three job settings are for Asynchronous inference.

scoring.py: one function for every pattern

Make escalation/scoring.py. The batch job, the API and the jobs all call this one function, so every pattern scores a ticket in exactly the same way:

escalation/scoring.py

"""Score tickets with a loaded bundle. The API and the batch job both use this.

The work for each request is split into three timed steps, so that a
benchmark can show where the time goes:

    frame     the checked tickets → a table with the training columns
    prepare   the pipeline's preprocessing (one-hot, impute, scale)
    model     the logistic regression's score
"""

import time
from dataclasses import dataclass, field

from escalation.bundle import Bundle
from escalation.contract import TicketFeatures, to_frame, unknown_categories


@dataclass
class Result:
    score: float
    flag: bool
    warnings: list[str]


@dataclass
class Timings:
    """Milliseconds spent in each step of one call."""

    steps: dict[str, float] = field(default_factory=dict)

    def header(self) -> str:
        """The value of a Server-Timing header, for example "frame;dur=0.41"."""
        return ", ".join(f"{name};dur={ms:.2f}" for name, ms in self.steps.items())


def score_tickets(bundle: Bundle, tickets: list[TicketFeatures]) -> tuple[list[Result], Timings]:
    timings = Timings()
    started = time.perf_counter()

    frame = to_frame(tickets)
    now = time.perf_counter()
    timings.steps["frame"] = (now - started) * 1000

    preprocess, model = bundle.pipeline[:-1], bundle.pipeline[-1]
    prepared = preprocess.transform(frame)
    started, now = now, time.perf_counter()
    timings.steps["prepare"] = (now - started) * 1000

    scores = model.predict_proba(prepared)[:, 1]
    started, now = now, time.perf_counter()
    timings.steps["model"] = (now - started) * 1000

    results = [
        Result(
            score=float(score),
            flag=bool(score >= bundle.threshold),
            warnings=unknown_categories(ticket, bundle.known),
        )
        for ticket, score in zip(tickets, scores)
    ]
    return results, timings

score_tickets takes a list of checked tickets and times its three steps. The batch job ignores the times; the next lesson uses them. The pipeline is split only to time the steps. pipeline[:-1] is the preprocessing and pipeline[-1] is the model, so the score is the same as from pipeline.predict_proba.

batch.py: the job

Make escalation/batch.py. The first part reads and checks the rows:

escalation/batch.py (part 1 of 2)

"""Score a file of tickets with the same bundle and contract as the API.

    python -m escalation.batch data/july_tickets.csv out/2026-07

Writes three files into the output folder:

    scores.csv      ticket_id, score, flag, model_version (one row per good ticket)
    rejected.csv    row, ticket_id, problem (one row per ticket that fails the contract)
    summary.json    what was read, what was written, with which model, and how long it took

A bad row does not stop the job: it is set aside in rejected.csv, and the
other rows are scored. But if more than --max-rejected of the rows are bad,
the input itself is probably broken, so the job fails (exit code 1) and
writes no scores.csv. Files are written under a temporary name and renamed at
the end, so a reader never sees half a file.
"""

import argparse
import hashlib
import json
import sys
import time
from datetime import datetime, timezone
from pathlib import Path

import pandas as pd
from pydantic import ValidationError

from escalation.bundle import BundleError, load_bundle
from escalation.contract import FEATURES, TicketIn
from escalation.scoring import score_tickets
from escalation.settings import SettingsError, load_settings

COLUMNS = ["ticket_id"] + FEATURES


def problem(error: ValidationError) -> str:
    first = error.errors()[0]
    where = ".".join(str(p) for p in first["loc"]) or "row"
    return f"{where}: {first['msg']}"


def check_rows(chunk: pd.DataFrame, first_row: int):
    good, bad = [], []
    for offset, row in enumerate(chunk.to_dict("records")):
        values = {k: (None if pd.isna(v) else v) for k, v in row.items()}
        try:
            good.append(TicketIn(**values))
        except ValidationError as error:
            bad.append({"row": first_row + offset, "ticket_id": values.get("ticket_id"), "problem": problem(error)})
    return good, bad


def write_atomic(frame: pd.DataFrame, path: Path) -> None:
    temporary = path.with_suffix(path.suffix + ".partial")
    frame.to_csv(temporary, index=False)
    temporary.replace(path)

Three details matter. check_rows sends each row through TicketIn, the same contract as the API, so a ticket that the API refuses is also refused here. An empty cell becomes None (null), not 0, because the pipeline’s imputer must see a missing value. write_atomic writes the .partial file, then renames it with replace.

The second part runs the job:

escalation/batch.py (part 2 of 2)

def main() -> int:
    parser = argparse.ArgumentParser(description="Score a CSV file of tickets.")
    parser.add_argument("input", type=Path)
    parser.add_argument("output", type=Path, help="a folder for scores.csv, rejected.csv and summary.json")
    parser.add_argument("--chunk", type=int, default=500, help="tickets per chunk")
    parser.add_argument("--max-rejected", type=float, default=0.05, help="the largest share of bad rows")
    args = parser.parse_args()

    started_at = datetime.now(timezone.utc)
    started = time.perf_counter()
    try:
        settings = load_settings()
        bundle = load_bundle(settings.model_bundle, settings.model_sha256)
    except (SettingsError, BundleError) as error:
        print(f"Start-up error: {error}", file=sys.stderr)
        return 2
    try:
        reader = pd.read_csv(args.input, usecols=COLUMNS, dtype=str, keep_default_na=False,
                             na_values=[""], chunksize=args.chunk)
    except ValueError as error:  # a missing column
        print(f"Stopped: {error}", file=sys.stderr)
        return 2

    scored, rejected, rows = [], [], 0
    for chunk in reader:
        good, bad = check_rows(chunk, first_row=rows + 2)  # row 1 is the header
        rows += len(chunk)
        rejected += bad
        if good:
            results, _ = score_tickets(bundle, good)
            scored += [{"ticket_id": t.ticket_id, "score": r.score, "flag": int(r.flag),
                        "model_version": bundle.version} for t, r in zip(good, results)]

    args.output.mkdir(parents=True, exist_ok=True)
    scores = pd.DataFrame(scored, columns=["ticket_id", "score", "flag", "model_version"])
    write_atomic(pd.DataFrame(rejected, columns=["row", "ticket_id", "problem"]), args.output / "rejected.csv")
    failed = rows == 0 or len(rejected) / rows > args.max_rejected
    if not failed:
        write_atomic(scores, args.output / "scores.csv")
    summary = {
        "input": str(args.input),
        "input_sha256": hashlib.sha256(args.input.read_bytes()).hexdigest(),
        "rows": rows,
        "scored": len(scores),
        "rejected": len(rejected),
        "flagged": int(scores["flag"].sum()),
        "flag_rate": round(float(scores["flag"].mean()), 3) if len(scores) else None,
        "model_version": bundle.version,
        "bundle_sha256": bundle.digest,
        "started_at": started_at.isoformat(timespec="seconds"),
        "seconds": round(time.perf_counter() - started, 2),
        "status": "failed: too many rejected rows" if failed else "done",
    }
    (args.output / "summary.json").write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
    print(f"Read {rows} rows: scored {len(scores)}, rejected {len(rejected)}, "
          f"flagged {summary['flagged']} ({summary['flag_rate']}). Model {bundle.version}. {summary['seconds']} s.")
    if failed:
        print(f"Failed: {len(rejected)} of {rows} rows were rejected (the limit is {args.max_rejected:.0%}). "
              f"No scores.csv was written. See {args.output / 'rejected.csv'}.", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

The job loads the bundle once, with all six checks of load_bundle, before it reads a single row. It reads the file in chunks of 500, so a file of a million rows never needs a million rows in memory. The exit code tells the scheduler what happened: 0 done, 1 too many bad rows. 2 means that the job could not start, or the file has the wrong columns.

Worked example: 20 tickets, then July

Amira starts small, so that every number is visible. She copies the header and the first 20 tickets of July into data/twenty.csv. In row 4 (ticket T-119235), she changes the priority from 3 to urgent. One bad row in 20 is exactly 5%, the limit.

Predict: does the run finish? How many tickets are scored?

Terminal, in ~/projects/escalation-service

python -m escalation.batch data/twenty.csv out/twenty

The output:

Read 20 rows: scored 19, rejected 1, flagged 3 (0.158). Model 1.0.0. 1.16 s.

out/twenty/rejected.csv names the row, the ticket and the problem:

row,ticket_id,problem
4,T-119235,"priority: Input should be a valid integer, unable to parse string as an integer"

scores.csv has 19 rows. T-119235 is not in it, and the other tickets keep their order:

ticket_id,score,flag,model_version
T-119233,0.037624397312637836,0,1.0.0
T-119234,0.255169256886106,1,1.0.0
T-119236,0.06880610647988575,0,1.0.0
...

The limit says “more than 5%”, and 1 in 20 is not more. Now Amira also changes created_hour in row 9 to 24. Two bad rows in 20 are 10%. She runs the job into out/twenty-two.

The output:

Read 20 rows: scored 18, rejected 2, flagged 2 (0.111). Model 1.0.0. 0.79 s.
Failed: 2 of 20 rows were rejected (the limit is 5%). No scores.csv was written. See out/twenty-two/rejected.csv.

The exit code is 1. out/twenty-two/ holds only rejected.csv, with rows 4 and 9, and summary.json, with "status": "failed: too many rejected rows". The job scored 18 tickets, but it does not write them. A list with 10% of the tickets missing is not a list that Grace’s team should trust.

Now the real file. Mei’s nightly file has about 45 tickets. To test with more, Amira scores all 1,400 July tickets in one run:

Terminal

python -m escalation.batch data/july_tickets.csv out/2026-07

The output:

Read 1400 rows: scored 1400, rejected 0, flagged 282 (0.201). Model 1.0.0. 0.75 s.

282 flagged tickets, 20.1%: the same as the machine learning course’s predict.py gave for July. The same model with the same preprocessing gives the same answer. summary.json records the run:

out/2026-07/summary.json

{
  "input": "data/july_tickets.csv",
  "input_sha256": "10dffc093993341a361172d0aa6ea0e7c06521f140cc2a80ee90c713361f7c99",
  "rows": 1400,
  "scored": 1400,
  "rejected": 0,
  "flagged": 282,
  "flag_rate": 0.201,
  "model_version": "1.0.0",
  "bundle_sha256": "8f233029aa4c896dce1a5b3dfa970cd085088386bbe7784519f15a5b1a433b74",
  "started_at": "2026-10-08T14:52:07+00:00",
  "seconds": 0.75,
  "status": "done"
}

input_sha256 names the exact input file, and bundle_sha256 names the exact model. Together they answer Grace’s question: “Which model scored this list, from which file?” Your bundle_sha256 is different, because your bundle holds your own training time. Only the course’s bundle in the labs has the digest 8f233029…. Your seconds is different too.

A re-run. Omar asks: “If the job runs twice by mistake, do we get two different lists?” Amira runs it again into out/2026-07-again and compares the two scores.csv files:

Terminal (Windows: Get-FileHash out\2026-07\scores.csv, out\2026-07-again\scores.csv)

shasum -a 256 out/2026-07/scores.csv out/2026-07-again/scores.csv

The output:

2b89ad330e0be98c2fadf3647c83304f6f134fac21511e029234f8d6fec18515  out/2026-07/scores.csv
2b89ad330e0be98c2fadf3647c83304f6f134fac21511e029234f8d6fec18515  out/2026-07-again/scores.csv

The same input file and the same bundle give the same bytes. So a re-run is safe: if the 02:00 run fails, the scheduler can run it again.

When batch is the right choice

Batch scoring fits when people need the results later, at a known time, and many cases arrive together. Examples: a morning review list, a monthly report, a mailing list. It is the simplest pattern to run. A scheduler starts one command, and nothing must stay up all day. If the run fails, you run it again before anyone needs the results.

It does not fit when a person or a program waits for one answer now. At Larkfield, a ticket that arrives at 09:05 is flagged the next morning, after about 23 hours. For a ticket that can escalate within 72 hours, that is a third of the time gone.

The costs: the delay, a scheduler to start the job, storage for the output folders, and a person who reads rejected.csv. A job that fails at night also needs an alert, or nobody notices until 08:00.

Tomás asks: “The next lesson scores each ticket at once. Then why build batch at all?” Because the batch job stays as the safety net. If the online service is down for an hour, the night run still scores every ticket of the day.

Guided practice: run the batch job

1. Make the three files of Build the batch job in escalation/: settings.py, scoring.py and batch.py.

2. Make the folder data/ in the project, and copy the July file into it:

   macOS / Linux

   mkdir -p data
   cp ../ticket-model/data/july_tickets.csv data/

   Windows (PowerShell)

   New-Item -ItemType Directory -Force data
   Copy-Item ..\ticket-model\data\july_tickets.csv data\

3. Set MODEL_SHA256 to the digest of your bundle. make_bundle.py printed it in Artifact trust and integrity. On your own computer, you can also compute it from SHA256SUMS:

   macOS / Linux

   export MODEL_SHA256=$(shasum -a 256 bundles/escalation-1.0.0/SHA256SUMS | cut -c1-64)

   Windows (PowerShell)

   $env:MODEL_SHA256 = (Get-FileHash bundles\escalation-1.0.0\SHA256SUMS -Algorithm SHA256).Hash.ToLower()

4. Score the July file:

   python -m escalation.batch data/july_tickets.csv out/2026-07

   The line starts with Read 1400 rows: scored 1400, rejected 0, flagged 282 (0.201).

5. Open out/2026-07/summary.json. Check that input_sha256 starts with 10dffc09 and that bundle_sha256 equals your MODEL_SHA256.

6. Give the tests their own settings. The job reads MODEL_BUNDLE and MODEL_SHA256 from the environment, so the tests must set them, not your terminal. In tests/conftest.py, add this import after import pytest:

   tests/conftest.py

   from escalation.bundle import bundle_digest

   Then add this fixture at the end of the file:

   tests/conftest.py

   @pytest.fixture(autouse=True)
   def settings_env(monkeypatch):
       monkeypatch.chdir(ROOT)
       monkeypatch.setenv("MODEL_BUNDLE", str(BUNDLE))
       monkeypatch.setenv("MODEL_SHA256", bundle_digest(BUNDLE))
       for name in ["API_KEY", "API_KEY_FILE", "REQUIRE_API_KEY", "SHADOW_BUNDLE", "SHADOW_SHA256"]:
           monkeypatch.delenv(name, raising=False)

   autouse=True runs the fixture before every test. It computes the digest of the project’s bundle and removes settings of later modules, so a value in your terminal cannot change a test. The job that a test starts as a command gets the same settings.

7. Make tests/test_batch.py. The two tests run the job as a command, exactly as the scheduler does:

   tests/test_batch.py

   import json
   import subprocess
   import sys
   
   import pandas as pd
   
   from tests.conftest import TICKET
   
   
   def run_batch(tmp_path, rows):
       source = tmp_path / "tickets.csv"
       pd.DataFrame(rows).to_csv(source, index=False)
       out = tmp_path / "out"
       result = subprocess.run([sys.executable, "-m", "escalation.batch", str(source), str(out)],
                               capture_output=True, text=True)
       return result, out
   
   
   def test_bad_rows_are_set_aside(tmp_path):
       rows = [{**TICKET, "ticket_id": f"T-{120000 + i}"} for i in range(40)]
       rows[5]["priority"] = "high"
       result, out = run_batch(tmp_path, rows)
       assert result.returncode == 0, result.stderr
       assert len(pd.read_csv(out / "scores.csv")) == 39
       rejected = pd.read_csv(out / "rejected.csv")
       assert rejected.to_dict("records")[0]["row"] == 7
       assert json.loads((out / "summary.json").read_text())["status"] == "done"
   
   
   def test_too_many_bad_rows_fail_the_job(tmp_path):
       rows = [{**TICKET, "ticket_id": f"T-{120000 + i}", "created_hour": 25} for i in range(10)]
       result, out = run_batch(tmp_path, rows)
       assert result.returncode == 1
       assert not (out / "scores.csv").exists()
       assert len(pd.read_csv(out / "rejected.csv")) == 10

   The first test puts one bad row in 40, at index 5. The header is row 1, so the bad row is row 7 of the file. The second test makes every row bad.

8. Run all the tests with python -m pytest. The last line says 20 passed.

Show the expected test output of step 8

tests/test_batch.py ..                                                   [100%]
...
============================== 20 passed in 2.47s ==============================

The 18 tests of Module 1 and the 2 new ones pass. They pass in a new terminal too, without MODEL_SHA256: the fixture settings_env sets it for each test.

Your turn: two empty cells

In the worked example, the bad values were typos. Now try empty cells. Make a fresh data/twenty.csv from the header and the first 20 tickets of July:

macOS / Linux

head -21 data/july_tickets.csv > data/twenty.csv

Windows (PowerShell)

Get-Content data\july_tickets.csv -TotalCount 21 | Set-Content data\twenty.csv

Open it in VS Code. In row 4 (T-119235), delete the value of order_value, 186.4, and keep the commas. In row 9 (T-119240), delete the value of team, returns. Before you run python -m escalation.batch data/twenty.csv out/twenty-empty, write down your prediction: which rows are rejected, what is the exit code, and how many tickets are in scores.csv?

Show the answer

The output:

Read 20 rows: scored 19, rejected 1, flagged 2 (0.105). Model 1.0.0. 0.8 s.

rejected.csv has one row:

row,ticket_id,problem
9,T-119240,team: Input should be a valid string

An empty cell becomes null. The contract allows null for order_value (no order), so row 4 is scored: T-119235 gets 0.0978 instead of 0.1061 with its order. The contract does not allow null for team, so row 9 is rejected. 1 in 20 is 5%, not more, so the exit code is 0, and scores.csv has 19 tickets. To see the exit code, run echo $? on macOS and Linux, or $LASTEXITCODE in PowerShell.

Common mistake

The job stops with Stopped: Usecols do not match columns

Symptom: one night, the job writes nothing at all, and it exits with code 2:

Stopped: Usecols do not match columns, columns expected but not found: ['prior_escalations_90d']

Cause: Mei’s export lost a column. Here, a change in the export left out prior_escalations_90d. The job reads only the columns of the contract (usecols=COLUMNS), and pandas refuses a file without one of them. No row is checked, because without the column no row can be scored correctly.

Fix: fix the export, then run the job again into the same dated folder. Do not “repair” the file with a column of zeros. The model learned that prior_escalations_90d matters, and 0 in every row quietly changes every score.

For a real AI service: a missing column is a problem of the file, not of a row. Stop the whole run, loudly, before scoring. A quiet default is the start of training-serving skew, which Module 3 checks for.

Check your understanding

Mei's file has 1,000 tickets, and 30 of them have created_hour 24. What does the job do, and what does Grace's team see in the morning?

Show the answer

30 of 1,000 is 3%, which is not more than the 5% limit. So the job finishes with exit code 0. scores.csv has the 970 good tickets, and rejected.csv lists the 30 bad ones with their row numbers and the problem created_hour: Input should be less than or equal to 23. summary.json says "rejected": 30 and "status": "done". Grace’s team has its list, and someone should look at the 30 rejected tickets by hand.

Check your understanding

Why does the job write scores.csv.partial and then rename it, instead of writing scores.csv directly?

Show the answer

A run can stop in the middle, for example when the computer restarts. If the job wrote scores.csv directly, a reader could find half a file that looks complete. The file system does the rename in one step, so scores.csv is either absent or complete. The atomic write protects the reader from a list with tickets missing.

Check your understanding

Two runs on the same day gave two scores.csv files with different SHA-256 values. Which two fields of summary.json do you compare first, and why?

Show the answer

Compare input_sha256 and bundle_sha256. With the same input and the same bundle, the job gives the same bytes, as the re-run in this lesson showed. So a different output means that the input file changed, or that another model version scored it. For example, Mei’s export ran again with more tickets.

See it end to endScore credit default in batch and onlineCase study

The case study runs a monthly batch job over all customers with the same rules: rejected rows, a failure limit and a summary.

Recap

• Batch scoring scores many cases in one scheduled run and stores the results for later. Throughput, the cases per second, matters more than the time of one case: 1,400 tickets in 0.75 s.
• Bad rows go to rejected.csv with their row number and problem. More than 5% bad rows fail the run, with no scores.csv. A missing column stops it at once.
• Files are written as .partial and renamed, so nobody reads half a file.
• summary.json records the input hash, the bundle digest and the status. The same input and bundle give the same output, so a re-run is safe.
• The cost is the wait: a ticket can be almost a day old when its flag arrives.

Next, you score one ticket at the moment an agent opens it, and you agree how long the help desk can wait.

Next: Online inference