Focused tests

What you will be able to do

Turn your checks into tests that run in one second with one command, every time that you change the code. The first test that you write proves that the encoding bug from Read failures can never come back without you knowing.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every test, task and check of this lesson, with the instructions. Its first cells make the ticket-cleaner folder as at the start of this lesson. They also install httpx and pytest and download the data files. So the notebook works without your local project. Read the explanations here, and run the code there.

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• assert, from Function contracts: Check each function alone.
• The minimal failing example of the encoding bug, from Read failures: Failure 2.
• Mocked responses, from Basic HTTP client use, and try and except, from Exceptions and logging.

What you need on your computer (only for the local route; the notebook needs none of it)

• ticket-cleaner as at the end of Exceptions and logging. It reads CSV with utf-8-sig, rejects a value that is not a dictionary, and logs its errors. If it is different, do that section again. You can also copy the files from the setup cell of this lesson’s notebook. It has every file as at the start of this lesson.
• requirements.txt and requirements-lock.txt, from Isolate dependencies: Record the versions. This lesson adds pytest to them.
• data/tickets.csv and data/tickets.json. You downloaded them in Reading and writing files: Get the data file. If one is missing or changed, download it again with the same curl command. The checksums are in the dataset card.
• The project’s environment .venv, active in your terminal (the prompt starts with (.venv)). You made it in Isolate dependencies: Guided practice. If it is missing or broken, delete it and make it again from the lock file, as in Make the environment again from nothing.
• An internet connection, to install pytest (about 2 MB).

Tested with Python 3.14.6, pytest 9.1.1 and httpx 0.28.1 on macOS. The notebook was tested in Jupyter.

From assert to a test

In Function contracts, you checked functions with assert lines in a script. That works, but the first failed assert stops the script, so you see one problem at a time. And nobody runs a script that they forgot exists.

A test is a function that checks one fact about your code with assert. A test runner finds all the tests, runs each one separately, and reports every failure. The most common test runner for Python is pytest. Its rules are simple:

• Test files are in a folder called tests, and their names start with test_.
• Test functions have names that start with test_.
• A test passes if it ends without an exception. A failed assert makes it fail, and pytest shows the values that were different.

A good test name says the fact that it checks, as a sentence: test_duplicate_id_is_rejected. When it fails, the name tells you what broke before you read the code.

Install pytest

pytest is a dependency for development: the program does not need it to run, but you need it to work on the program. Add it to the project’s requirements as in Isolate dependencies.

1. Add a line to requirements.txt:

   requirements.txt

   httpx==0.28.1
   pytest==9.1.1

2. Install, and update the lock file:

   python -m pip install -r requirements.txt
   python -m pip freeze > requirements-lock.txt

   The lock file now also has pytest, and the packages that pytest needs, such as pluggy and iniconfig.

3. Make the tests folder: mkdir tests.

Worked example: the regression test first

A regression is a bug that comes back after it was fixed. A regression test reproduces the bug, so it fails while the bug exists, and passes when it is fixed. Write it from the minimal failing example:

tests/test_files.py

import json

from ticket_cleaner.files import read_rows, write_json


def test_csv_with_byte_order_mark(tmp_path):
    path = tmp_path / "tickets.csv"
    path.write_text("id,status\nT-1001,open\n", encoding="utf-8-sig")
    assert read_rows(path) == [{"id": "T-1001", "status": "open"}]


def test_write_json_makes_the_folder(tmp_path):
    path = tmp_path / "reports" / "summary.json"
    write_json({"selected": 0}, path)
    assert json.loads(path.read_text(encoding="utf-8")) == {"selected": 0}
    assert not (tmp_path / "reports" / "summary.json.tmp").exists()

tmp_path is a fixture: a value that pytest makes and gives to a test that names it as a parameter. tmp_path is a new, empty folder for each test, which pytest deletes later. Tests that write files use it, so they never change the files in your project, and never depend on each other. path.write_text(...) writes a whole text file in one call.

The second test checks the promises of write_json: it makes a missing folder, the content is the data, and no temporary file is left.

Run the tests. python -m pytest runs pytest with the interpreter of the active environment, from the project folder, so the tests can import ticket_cleaner:

python -m pytest

Output

============================= test session starts ==============================
platform darwin -- Python 3.14.6, pytest-9.1.1, pluggy-1.6.0
rootdir: /Users/amira/projects/ticket-cleaner
plugins: anyio-4.15.1
collected 2 items

tests/test_files.py ..                                                   [100%]

============================== 2 passed in 0.02s ===============================

Each . is a passed test. (The anyio plugin comes with httpx. You can ignore it.)

Prove that the test can fail. A test that passes with the bug and without it proves nothing. In files.py, change utf-8-sig back to utf-8, and run the tests again:

Output (the important part)

tests/test_files.py F.                                                   [100%]

=================================== FAILURES ===================================
________________________ test_csv_with_byte_order_mark _________________________

    def test_csv_with_byte_order_mark(tmp_path):
        path = tmp_path / "tickets.csv"
        path.write_text("id,status\nT-1001,open\n", encoding="utf-8-sig")
>       assert read_rows(path) == [{"id": "T-1001", "status": "open"}]
E       AssertionError: assert [{'\ufeffid':...tus': 'open'}] == [{'id': 'T-10...tus': 'open'}]
E
E         At index 0 diff: {'\ufeffid': 'T-1001', 'status': 'open'} != {'id': 'T-1001', 'status': 'open'}

tests/test_files.py:9: AssertionError
=========================== short test summary info ============================
FAILED tests/test_files.py::test_csv_with_byte_order_mark - AssertionError: a...
========================= 1 failed, 1 passed in 0.03s ==========================

F is a failed test. The > marks the line that failed. The lines with E show the two values: what the function returned, with '\ufeffid', and what the test expected. That is the bug, found by a test in a fraction of a second. Change the encoding back to utf-8-sig, and run again: 2 passed.

Test the contract: normal, boundary and error cases

For each function, test three kinds of input:

Kind	Question	Examples for find_problems
Normal	Does it work for a usual input?	A valid record has no problems.
Boundary	Does it work at the edges?	Priority 1 and 3 are valid. 0 and 4 are not. Empty input.
Error path	Does it report bad input correctly?	A missing field, high, None instead of a record.

Bugs collect at boundaries: < where <= was needed, an empty list, a first or last item. A test for every boundary finds them.

tests/test_parsing.py

import pytest

from ticket_cleaner.parsing import find_problems, split_records, to_ticket
from ticket_cleaner.records import Ticket


@pytest.fixture
def valid_raw():
    return {"id": "T-1001", "status": "open", "category": "billing", "priority": "2"}


def test_valid_record_has_no_problems(valid_raw):
    assert find_problems(valid_raw) == []


def test_text_is_cleaned(valid_raw):
    valid_raw["id"] = " t-1001 "
    valid_raw["status"] = "Open "
    expected = Ticket(id="T-1001", status="open", category="billing", priority=2)
    assert to_ticket(valid_raw) == expected


@pytest.mark.parametrize("priority", ["0", "4", "high", "2.5", "-1"])
def test_priority_outside_1_to_3_is_a_problem(valid_raw, priority):
    valid_raw["priority"] = priority
    assert find_problems(valid_raw) == [f"priority must be 1, 2 or 3, not {priority!r}"]


def test_missing_field_is_a_problem(valid_raw):
    del valid_raw["category"]
    assert find_problems(valid_raw) == ["missing category"]


def test_duplicate_id_is_rejected(valid_raw):
    tickets, rejected = split_records([valid_raw, dict(valid_raw)])
    assert len(tickets) == 1
    assert rejected[0].row == 2
    assert rejected[0].problems == ["duplicate id T-1001"]


def test_a_value_that_is_not_a_dictionary_is_rejected():
    tickets, rejected = split_records([None])
    assert tickets == []
    assert rejected[0].problems == ["not a record"]


def test_empty_input_gives_empty_results():
    assert split_records([]) == ([], [])

Three new tools:

• @pytest.fixture makes your own fixture. valid_raw gives each test that names it a new valid record. A test can change its copy, as test_text_is_cleaned does, without affecting other tests.
• @pytest.mark.parametrize runs one test several times with different values. Here, one test function becomes five tests, one for each bad priority. In the output, each one counts separately.
• del valid_raw["category"] removes a key from the dictionary, to make a record with a missing field.

The last two tests are the error path and the empty input from earlier lessons. They became tests too.

Test the web client without the network

The mock from Basic HTTP client use becomes two tests. pytest.raises checks the error path: the test passes only if the code in the with block raises that exception type.

tests/test_remote.py

import httpx
import pytest

from ticket_cleaner.remote import fetch_rows

URL = "https://tickets.example/api/tickets"


def answer_ok(request: httpx.Request) -> httpx.Response:
    return httpx.Response(200, json=[{"id": "T-2001", "status": "open"}])


def answer_unavailable(request: httpx.Request) -> httpx.Response:
    return httpx.Response(503, text="Service Unavailable")


def test_fetch_rows_returns_the_records():
    client = httpx.Client(transport=httpx.MockTransport(answer_ok))
    assert fetch_rows(URL, client=client) == [{"id": "T-2001", "status": "open"}]


def test_fetch_rows_raises_on_server_error():
    client = httpx.Client(transport=httpx.MockTransport(answer_unavailable))
    with pytest.raises(httpx.HTTPStatusError):
        fetch_rows(URL, client=client)

These tests do not need the internet, and they always give the same result. A test that depends on a real server fails when the server or your connection has a problem, even when your code is correct.

Guided practice: the test suite

1. Make tests/test_files.py, and run python -m pytest. Two tests pass.

2. Change the encoding in read_rows to utf-8, and run the tests. test_csv_with_byte_order_mark fails, with the '\ufeffid' difference. Change it back, and run again. Both pass. You now have evidence that the test fails before the fix and passes after it.

3. Make tests/test_parsing.py and tests/test_remote.py. Run all the tests:

   Output (last lines)

   tests/test_files.py ..                                                   [ 13%]
   tests/test_parsing.py ...........                                        [ 86%]
   tests/test_remote.py ..                                                  [100%]
   
   ============================== 15 passed in 0.06s ==============================

   test_parsing.py shows 11 dots: 6 test functions, and the parametrized one counts 5 times.

4. Run one group of tests by name with -k: python -m pytest -k duplicate. Only the duplicate test runs. This is useful while you work on one function.

5. Run the tests with -v to see each test’s name: python -m pytest -v.

Your turn: boundaries of the report

Make tests/test_report.py, with tests for report.py. Write at least these, and give each a name that says the fact:

1. average_priority returns None for no tickets.
2. average_priority rounds to one decimal: priorities 1, 1 and 2 give 1.3.
3. count_by_category returns the categories in alphabetical order.
4. build_summary for a status that no ticket has: selected is 0, by_category is {}, and average_priority is None.

Make the Ticket objects in each test, or with a fixture. Then break each function on purpose, one at a time, and check that a test fails.

Show one answer

tests/test_report.py

import pytest

from ticket_cleaner.records import Ticket
from ticket_cleaner.report import average_priority, build_summary, count_by_category


@pytest.fixture
def tickets():
    return [
        Ticket(id="T-1", status="open", category="login", priority=1),
        Ticket(id="T-2", status="open", category="billing", priority=1),
        Ticket(id="T-3", status="closed", category="billing", priority=2),
    ]


def test_average_priority_of_no_tickets_is_none():
    assert average_priority([]) is None


def test_average_priority_is_rounded_to_one_decimal(tickets):
    assert average_priority(tickets) == 1.3


def test_categories_are_in_alphabetical_order(tickets):
    assert list(count_by_category(tickets)) == ["billing", "login"]


def test_summary_for_a_status_with_no_tickets(tickets):
    summary = build_summary(tickets, [], status="pending")
    assert summary["selected"] == 0
    assert summary["by_category"] == {}
    assert summary["average_priority"] is None

list(a_dictionary) gives its keys in order, so the third test checks the order, not only the counts.

Common mistake

A test that passes for the wrong reason

Symptom: a test passes, but the function is broken. Or a test passes before and after the fix.

def test_find_problems_for_bad_priority(valid_raw):
    valid_raw["priority"] = "high"
    problems = find_problems(valid_raw)
    assert problems

Cause: assert problems checks only that the list is not empty. If find_problems reported the wrong problem, for example missing priority, the test still passes. Other causes: a test function whose name does not start with test_ (pytest never runs it), or a test without any assert.

Fix:

1. Compare with the exact expected value: assert problems == ["priority must be 1, 2 or 3, not 'high'"].
2. Break the code on purpose, and check that the test fails. Do this for every new test, as you did for the encoding.
3. Check the count of tests in the output. If it did not go up after you added a test, pytest did not find it.

Check your understanding

Why does the encoding test write its file in tmp_path, not in data/?

Show the answer

tmp_path is a new, empty folder for each test. The test cannot change your real data, and it does not depend on a file that someone could change or delete. Each test makes exactly the input that it needs, so it gives the same result on every computer.

Check your understanding

Module check: show that the regression test fails before the fix and passes after it. Which two commands do you run, and what do you change between them?

Show the answer

Change the encoding in read_rows to utf-8 (the bug), and run python -m pytest: test_csv_with_byte_order_mark fails, and the E lines show '\ufeffid' against 'id'. Change it back to utf-8-sig (the fix), and run python -m pytest again: all tests pass. Keep both outputs as evidence.

Recap

• A test is a function named test_... in a file tests/test_....py. Run all of them with python -m pytest.
• Test the normal case, the boundaries and the error paths. Compare with exact values.
• Fixtures give each test fresh data: your own with @pytest.fixture, and a temporary folder with tmp_path.
• @pytest.mark.parametrize runs one test with many values. pytest.raises checks that an error is raised.
• Mock the network in tests. Write a regression test for every bug that you fix, and check that it fails without the fix.

You can now find failures, handle errors, and prove with tests that the code works. In the last module, you make ticket-cleaner a project that another person can install and run.

Next: Notebook discipline