Fix a bug in an unfamiliar codebase

What you will be able to do

Fix a customer’s bug in a small project that you have never seen. Map the code before you ask for a change, ask the assistant targeted questions, and reproduce the bug yourself. Then shrink it to the smallest input, write the failing test, fix it with the assistant, and review the diff. At the end, compare your fix with the fix of the people who wrote the code.

Case studyPortfolio pieceSynthetic data

Dataset

delivery-slots, Larkfield's delivery slot tool (made for this case study)

Licence

MIT

Time

About 2.5 hours

Runs in

VS Code, Python 3.12+, Git

Skills used

• Map an unfamiliar codebase
• Targeted questions
• Read the Git history
• Reproduce and shrink
• Failing test first
• Review a generated diff
• Compare fixes under one measure

Download the starter project (.zip)Finished project (.zip)

Unzip the starter, open the folder in VS Code, and follow the steps below. They start with a new virtual environment.

The course team wrote delivery-slots for this case study, under the MIT licence. It has a Git history, two issues and 35 tests. Larkfield, the delivery team, the customer and the bookings are invented. The postcode districts are real districts of Leeds. The assistant session is real: you can read every prompt and reply in the recorded transcript. The maintainers’ fix in the finished project is constructed. The course team wrote it as the delivery team’s own fix, so that you can compare two fixes.

Before you startWhat to know and to have readyAbout 150 minutes

What you need to know

This case study uses the whole course on code that is not yours. It leans most on Break work into steps (investigate before you change), Reproduce first, Choose meaningful checks and Inspect the diff. From the Git course, you need commits and git log, from Track a first change, and diffs, from Read and review diffs.

What you need on your computer

• Python 3.12 or newer, Git and VS Code, as in the earlier modules. About 40 MB of free disk space.
• No account and no money. A coding assistant is optional: the recorded session is supplied, and a patch file gives you its change.

Tested on macOS (Apple silicon) with Python 3.14.6. The Windows commands are written, not tested.

The situation

On Monday morning, Grace from customer support adds an issue to the delivery team’s project. A customer, Ruth, lives in Armley, postcode LS12 2AB. Larkfield’s delivery page promises evening slots for West Leeds on Tuesdays and Thursdays, from 18:00 to 20:00. Ruth works during the day. At 10:00 she ordered a garden bench, and the checkout offered her only three daytime slots for Tuesday. Her sister in LS4 has an evening delivery booked for the same day.

The delivery team is two people, Hana and Luis, and both are at a trade fair until Friday. Amira, who keeps the help-desk tools, offers to look at it. She has never opened delivery-slots.

The objective. Ruth gets her evening slot for Tuesday. Every customer gets the slots of their own zone. And Hana and Luis can accept the fix when they are back. It has a failing test first, a small diff, and no change that nobody asked for.

What you will do

Part	What you do	Who does it
1	Map the project: files, commands, the path of the issue’s command	you, then the assistant
2	Ask targeted questions about one function	the assistant; you check
3	Reproduce the bug, and read the history	you
4	Shrink it to the smallest input	you
5	Write the failing test, and commit it	you
6	Fix it	the assistant, inside your limits
7	Review the diff	the assistant and you
8	Compare the fixes under one measure, and recommend one	you
9	Run it end to end on new input	you

The codebase

Fact	Value
Project	delivery-slots: lists a postcode’s delivery slots, books a slot, prints a driver’s list
Licence	MIT, “Copyright (c) Nextia Learning”
Size	9 Python files (402 lines), 7 test files (317 lines), 3 CSV files
History	6 commits by Hana, Luis and Grace; issues 3 (closed) and 12 (open)
Dependencies	none to run it; pytest and ruff to work on it

The project card says what is constructed and lists the checksums. The history was written to fit the habits that you practise here: an earlier issue, its fix, and the bug that the fix brought.

Set up the project

1. Download the starter project from the box at the top of this page. Unzip it into ~/projects. You get the folder delivery-slots.

2. In VS Code, choose File > Open Folder and open delivery-slots. Then open a terminal: Terminal > New Terminal.

3. Make and activate a virtual environment, and install the tools from the lock file:

   macOS and Linux

   python3 -m venv .venv
   source .venv/bin/activate
   python -m pip install -q -r requirements-lock.txt

   Windows

   python -m venv .venv
   .venv\Scripts\Activate.ps1
   python -m pip install -q -r requirements-lock.txt

   The lock file installs pytest 9.1.1 and ruff 0.16.10, with four small packages that pytest needs. The program itself needs only Python.

4. Check your starting point:

   git status
   python -m pytest
   git log --oneline

   Output (the end of pytest, then the log)

   On branch main
   nothing to commit, working tree clean
   ======================================== 35 passed in 0.04s ========================================
   ed951ba Add issue 12: no evening slots for LS12
   2ed4c77 Add evening slots for the west zone, and this week's bookings
   bf5b5e6 Accept postcodes typed without a space (fixes issue 3)
   d84536f Print the driver's list for a day and a zone
   8569257 Book a slot for an order
   cb4a3e2 First version: list the slots of a postcode on a day

What this tells us. The tests pass and the tree is clean: this is the baseline from Repository preparation. The hashes are the same on every computer, because the course built the history with fixed authors and dates. All 35 tests pass although the bug is real. So the tests do not cover it, and a passing suite says nothing about issue 12.

Part 1: map first

In a new town, you look at the map before you ask for directions. Then you understand the answer, and you notice when it is wrong.

A map of a codebase answers three questions. Which files are there, and what does each one do? Where does a command start? Which functions does the failing command go through? Make the map before any change. As Break work into steps showed, a change in code that you do not know is a guess.

Make the first map yourself. It takes five minutes:

git ls-files
python -m delivery_slots --help

Output of the second command

usage: delivery_slots [-h] [--data DATA] [--now NOW] {slots,book,manifest} ...

Larkfield's delivery slots.

positional arguments:
  {slots,book,manifest}
    slots               list the slots for a postcode on a day
    book                book a slot for an order
    manifest            print a driver's list for a day and a zone

options:
  -h, --help            show this help message and exit
  --data DATA           folder with zones.csv, slot-plan.csv and bookings.csv (default: data)
  --now NOW             use this time instead of the clock, for example 2026-10-12T10:00

The file list has delivery_slots/ (9 files), tests/ (7 files), data/ (3 CSV files) and issues/ (2 files). Read README.md and issues/12-no-evening-slots-in-ls12.md in full. They are short. They say what the program promises: three zones, a cut-off at 20:00 the day before, and evening slots for the west zone.

The assistant’s map

Amira then asked the assistant for the same map, in ask mode, with no file in the chat. The assistant saw Aider’s repository map: the file names and their functions. Aider also asked to add cli.py, because the prompt names it. Her prompt, from the recorded transcript (the issue’s text is left out here):

Turn 1, prompt (end)

Before anyone changes code, I need a map of the project. Please answer from the
repository map:

1. In two or three lines: what does this project do?
2. Each file in delivery_slots/, with one line on what it does.
3. The functions that this command goes through, in order, from cli.py to the
   lines it prints:

       python -m delivery_slots --now 2026-10-12T10:00 slots "LS12 2AB" --day 2026-10-13

Do not change any file, and do not guess the cause yet.

The reply listed every file with a correct one-line role. It traced the command through cli.main, cli.load, cli.cmd_slots, zones.zone_for and slots.slots_for, and stopped there:

Turn 1, the assistant's reply (end of the first answer)

The summaries do not reveal the internal call order within `zone_for`, `slots_for`, or the CSV
loaders, so this trace stops at those function boundaries. No files have been changed, and no cause
is inferred.

That is a good map. But look at what happened next. The first answer said that the contents of __init__.py and __main__.py “have not been supplied”. Aider then offered to add the six files that the answer named, and the second answer asked for five more. The recording answered yes to every question, so Aider added them and sent two more requests: 12 files in all. The last reply said: “My earlier requests were unnecessary”. Turn 1 cost 3 model calls and US$0.043, about half of the whole session.

In your own session, you decide. The map was complete after the first call. “No, the map is enough” would have saved two calls.

Check the map against the code. Open delivery_slots/cli.py and find cmd_slots. It calls zones.zone_for, then slots.slots_for, then prints. The map is right. You now know where to look: the slots, or the zone.

Part 2: ask targeted questions

A targeted question names one function, one input and one expected fact, and asks for an answer from the code. “What is wrong?” invites a guess. “What does this function return for this input, and at which line?” asks for evidence that you can check.

Amira added three files: zones.py, postcodes.py and data/zones.csv. Her second prompt:

Turn 2, prompt

Targeted questions. For LS12 2AB on Tuesday, the customer was offered 08:00-10:00,
10:00-12:00 and 12:00-14:00. In data/slot-plan.csv, those are the city zone's
slots, not the west zone's.

1. Where in the code is the zone of a postcode decided? Name the function and
   the lines.
2. Walk through that function for "LS12 2AB", with the rows of data/zones.csv
   in their order. Which zone does it return, and at which step?
3. Which other districts in data/zones.csv get a wrong zone in the same way?

Answer from the code only. Do not change any file.

Turn 2, the assistant's reply (part)

In **`delivery_slots/zones.py`, `zone_for()`**, defined on line **27**. The decision is made on
lines **33–36**:
...
3. The first CSV row is **`LS1,city`**.
4. `"LS122AB".startswith("LS1")` is **`True`**.
5. Line **36** immediately returns **`"city"`**.

No later rows are checked—including **`LS12,west`**. This explains why this lookup selects the city
zone.

It also named LS13, LS16 and LS17 as wrong in the same way. And it noted that LS10 and LS11 also match LS1 first, but are city districts anyway. The turn cost one call, US$0.013.

This is a precise claim. It is also only a claim. The next two parts check it with runs.

Part 3: reproduce, and read the history

Run the issue’s command, as Ruth saw it on Monday at 10:00. The --now option replaces the clock:

python -m delivery_slots --now 2026-10-12T10:00 slots "LS12 2AB" --day 2026-10-13
python -m delivery_slots --now 2026-10-12T10:00 slots "LS4 2TE" --day 2026-10-13

Output

LS12 2AB is in zone city.
Slots on Tuesday 13 October 2026 (orders close Monday at 20:00):
  08:00-10:00  7 places left
  10:00-12:00  9 places left
  12:00-14:00  7 places left
LS4 2TE is in zone west.
Slots on Tuesday 13 October 2026 (orders close Monday at 20:00):
  08:00-12:00  11 places left
  18:00-20:00  5 places left

What this tells us. You see what Ruth saw: three daytime slots and no evening. The first line says why: LS12 2AB is in zone city. Her sister’s district gets the west zone’s slots, with the evening. The cut-off is not the cause: on Monday at 10:00 the orders were open. The slot plan is not the cause either: the west zone does have 18:00-20:00 on Tuesdays.

Who changed this line, and why?

git log lists commits. git blame goes one level deeper: for each line of a file, it shows the last commit that changed it. In unfamiliar code, it tells you why a line is as it is. That is often the fastest way to the cause.

git log --oneline -- delivery_slots/zones.py
git blame -s -L 27,37 delivery_slots/zones.py

Output

bf5b5e6 Accept postcodes typed without a space (fixes issue 3)
cb4a3e2 First version: list the slots of a postcode on a day
^cb4a3e2 27) def zone_for(postcode: str, zones: dict[str, str]) -> str:
bf5b5e6e 28)     """The zone of a postcode, for example "west" for "LS5 3QR".
bf5b5e6e 29)
bf5b5e6e 30)     Customers type postcodes with or without the space ("LS5 3QR", "ls53qr"),
bf5b5e6e 31)     so the district is matched at the start of the postcode.
bf5b5e6e 32)     """
bf5b5e6e 33)     code = normalise(postcode).replace(" ", "")
bf5b5e6e 34)     for district, zone in zones.items():
bf5b5e6e 35)         if code.startswith(district):
bf5b5e6e 36)             return zone
bf5b5e6e 37)     raise UnknownPostcode(f"We do not deliver to {normalise(postcode)}.")

Every line of the lookup comes from one commit: Luis’s fix for issue 3. Open issues/3-postcode-without-a-space.md. A customer typed LS53QR without the space. The first version split the postcode at the space, so it found no district. Luis’s fix matched the district at the start of the postcode instead. It fixed issue 3, and its test passes. It also made issue 12. Run git show bf5b5e6 to see the whole change.

The symptom is at the end of the path. The cause is two steps earlier, and the data file's order and the history explain it.

Text description of the diagram

Five steps from left to right. The command of issue 12 goes to cli.cmd_slots, which asks zones.zone_for for the zone. That step is the cause: "LS122AB" starts with "LS1", the first row, so it returns city. Then slots.slots_for returns the city zone’s slots, which is correct for city. The customer sees three daytime slots and no evening slot: the symptom. Below the cause, data/zones.csv has LS1, city in row 1 and LS12, west in row 14, which is never reached. git blame shows that lines 28 to 37 come from commit bf5b5e6, the fix of issue 3.

Part 4: shrink it to the smallest input

The issue’s input is a whole command with data files, a date and a clock. Reproduce first taught you to make it smaller until only the bug is left. Here, call the function directly. In VS Code, make a file shrink.py in the project folder:

shrink.py

from pathlib import Path

from delivery_slots.zones import load_zones, zone_for

zones = load_zones(Path("data") / "zones.csv")
print("LS12 2AB ->", zone_for("LS12 2AB", zones))

# Every district of the file, with a made-up inward code: which ones get another zone?
for district, zone in zones.items():
    got = zone_for(district + " 1AA", zones)
    if got != zone:
        print(f"{district}: file says {zone}, zone_for says {got}")

# The smallest input: two rows and one postcode.
print(zone_for("LS12 2AB", {"LS1": "city", "LS12": "west"}))
print(zone_for("LS12 2AB", {"LS12": "west", "LS1": "city"}))

python shrink.py

Output

LS12 2AB -> city
LS16: file says north, zone_for says city
LS17: file says north, zone_for says city
LS12: file says west, zone_for says city
LS13: file says west, zone_for says city
city
west

What this tells us. Every claim of turn 2 is true. LS12, LS13, LS16 and LS17 get the wrong zone, and the other 11 districts are right. The last two lines are the minimal failing input: two rows and one postcode. The same two rows in the other order give the right answer. So the result depends on the order of the rows, which a lookup should never do.

That is the new idea of this case study, in one picture:

A first match stops at the first row that fits, so the order of the rows decides. A whole match looks up the exact district.

Text description of the diagram

The same postcode, LS12 2AB (LS122AB without the space), two lookups. Before the fix, row 1, LS1, already fits the start of the postcode, so the loop stops and returns city. Rows 2 to 13 are never checked, and row 14, LS12, west, is never reached. After the fix, the code removes the last three characters, gets LS12, and looks up exactly that: west. A district that Larkfield does not serve, LS19, is now refused; before, it was city.

A first match (a prefix match) is right only when no entry is the start of another. UK postcode districts break that rule: LS1 is the start of LS10 to LS19. A whole match needs the exact district. Every UK postcode ends with an inward code of three characters, such as 2AB. So the district is everything before the last three characters, with or without the space.

Delete shrink.py: it was a probe, not part of the fix.

Part 5: write the failing test first

Write the test before anyone changes the code, as in Choose meaningful checks. It must fail now, for the right reason, and pass after the fix. Use the four districts that the shrink found. Add this at the end of tests/test_zones.py:

tests/test_zones.py (added at the end)

@pytest.mark.parametrize(
    "postcode, zone",
    [
        ("LS12 2AB", "west"),
        ("LS13 4EX", "west"),
        ("LS16 7QR", "north"),
        ("LS17 5DT", "north"),
    ],
)
def test_two_digit_districts(zones, postcode, zone):
    """Issue 12: "LS12 2AB" got the city zone's slots."""
    assert zone_for(postcode, zones) == zone

python -m pytest

Output (end)

E       AssertionError: assert 'city' == 'north'
...
FAILED tests/test_zones.py::test_two_digit_districts[LS12 2AB-west] - AssertionError: assert 'cit...
FAILED tests/test_zones.py::test_two_digit_districts[LS13 4EX-west] - AssertionError: assert 'cit...
FAILED tests/test_zones.py::test_two_digit_districts[LS16 7QR-north] - AssertionError: assert 'ci...
FAILED tests/test_zones.py::test_two_digit_districts[LS17 5DT-north] - AssertionError: assert 'ci...
=================================== 4 failed, 35 passed in 0.04s ===================================

The four new cases fail with 'city', the bug itself. The other 35 still pass. Commit the test alone, so that the history shows it failing before any fix:

git add tests/test_zones.py
git commit -m "Add a failing test for issue 12: LS12 2AB is in zone west"

Part 6: fix it with the assistant

Now Amira asked for the change. She switched Aider from ask mode to code mode, where it may edit files, with only zones.py and the test file in the chat. Aider could run the tests after its edit. Her prompt sets the limits:

Turn 3, prompt

I committed a failing test: test_two_digit_districts in tests/test_zones.py.
It fails four times. For example, zone_for("LS12 2AB", zones) returns "city";
expected "west". `python -m pytest -q` prints "4 failed, 35 passed".

Fix zone_for so that these tests pass.
- Change only delivery_slots/zones.py. Do not change the tests or the data.
- Postcodes typed without a space must still work (issue 3,
  test_postcode_without_a_space).
- Keep the change small. In two lines, say why it fixes the cause.

Turn 3, the assistant's reply (first lines) and the test run

Match the complete district by removing the postcode’s final three characters, so LS12 cannot match
LS1.
Removing spaces first preserves support for compact postcodes such as `ls53qr`.
...
Applied edit to delivery_slots/zones.py
.......................................                                                      [100%]
39 passed in 0.03s

The turn cost one call, US$0.016.

Without an assistant, or to use the recorded change: download fix-issue-12.patch into ~/projects, next to the project folder. Apply it, and run the checks:

git apply ../fix-issue-12.patch
python -m pytest
ruff check . && ruff format --check .

Output (end)

======================================== 39 passed in 0.03s ========================================
All checks passed!
19 files already formatted

With your own assistant: do the same three steps. Ask for the map, then the targeted questions, then the fix with the same limits. Your change can differ from the recorded one; Part 8 gives you a way to judge it.

Part 7: review the diff

git diff

Output (the changed lines)

@@ -28,10 +28,10 @@ def zone_for(postcode: str, zones: dict[str, str]) -> str:
     """The zone of a postcode, for example "west" for "LS5 3QR".

     Customers type postcodes with or without the space ("LS5 3QR", "ls53qr"),
-    so the district is matched at the start of the postcode.
+    so remove spaces and the final three characters to get the district.
     """
     code = normalise(postcode).replace(" ", "")
-    for district, zone in zones.items():
-        if code.startswith(district):
-            return zone
+    district = code[:-3]
+    if district in zones:
+        return zones[district]
     raise UnknownPostcode(f"We do not deliver to {normalise(postcode)}.")

Review it line by line, with the checklist of Inspect the diff:

• Needed lines. All four code lines replace the first match with a whole match. The docstring now describes the new rule. No other file changed.
• Changed contracts. The function still returns a zone or raises UnknownPostcode. Its callers, cmd_slots and book, need no change.
• Hidden assumptions. code[:-3] assumes a complete postcode. What happens with a short or wrong one?
• Behaviour that changed on purpose, without a test. LS19 starts with LS1, so the old code gave it the city zone. The new code refuses it. That is right, because Larkfield does not deliver to LS19. But no test says so yet.

For the hidden assumptions, Amira asked the assistant to review its own change (turn 4, ask mode). It listed seven inputs, for example:

Turn 4, the assistant's reply (part)

| `"LS12 ABC"` | `"west"` | Accepts an invalid inward code: it must start with a digit. |
| `"LS12 2A"` | `"city"` | An incomplete inward code makes the extracted district `LS1`, rather than
`LS12`. |
| `"LS1 22AB"` | `"west"` | An extra inward-code character makes the extracted district `LS12`,
rather than the supplied `LS1`. |

Do not believe a review; run it. Make a file review.py with some of these postcodes, and run it on the fix and on the old code. git stash puts your uncommitted change away for a moment, as in Ask for evidence:

review.py

from pathlib import Path

from delivery_slots.zones import UnknownPostcode, load_zones, zone_for

zones = load_zones(Path("data") / "zones.csv")
for postcode in ["LS12 ABC", "LS12 2A", "LS16 7Q", "LS1 22AB", "LS12", "LS19 7XB"]:
    try:
        print(f"{postcode!r:12} -> {zone_for(postcode, zones)}")
    except UnknownPostcode as error:
        print(f"{postcode!r:12} -> refused: {error}")

python review.py
git stash -q
python review.py
git stash pop -q

Output: with the fix, then with the old code

'LS12 ABC'   -> west
'LS12 2A'    -> city
'LS16 7Q'    -> city
'LS1 22AB'   -> west
'LS12'       -> refused: We do not deliver to LS12.
'LS19 7XB'   -> refused: We do not deliver to LS19 7XB.
'LS12 ABC'   -> city
'LS12 2A'    -> city
'LS16 7Q'    -> city
'LS1 22AB'   -> city
'LS12'       -> city
'LS19 7XB'   -> city

What this tells us. Every claim that the assistant made is true. And the fix made none of these inputs worse. The old code gave a zone to every one of them, and the fix now refuses two. A postcode in a wrong format is a separate problem, and the issue did not ask for it.

So Amira decided three things. Accept the fix. Write the format problem down as a new issue for the owners. And add two tests for what the fix changed on purpose. Delete review.py, then add at the end of tests/test_zones.py:

tests/test_zones.py (added at the end)

def test_a_two_digit_district_without_a_space(zones):
    assert zone_for("ls122ab", zones) == "west"


def test_a_district_that_starts_like_a_served_one(zones):
    """LS19 starts with LS1, but Larkfield does not deliver there."""
    with pytest.raises(UnknownPostcode):
        zone_for("LS19 7XB", zones)

python -m pytest
git add -A
git commit -m "Match the whole postcode district (fixes issue 12)"

Output (end of pytest)

======================================== 41 passed in 0.03s ========================================

Common mistake

'Just go back to the first version'

Symptom: after reading git blame, Tomás says that the bug came with Luis’s change. He restores the first version of zone_for, which split the postcode at the space. Issue 12’s four tests pass. Then the suite prints 1 failed, 38 passed: test_postcode_without_a_space fails.

Cause: the first version has the bug of issue 3. A customer who types LS53QR is told “We do not deliver to LS53QR.” Undoing a change also undoes what it fixed. Here, Luis’s test caught it at once. Without that test, issue 3 would be back in production.

Fix: read the issue that a commit fixed before you undo the commit. A fix must keep both promises: districts with two digits, and postcodes without a space. The course measured this candidate (constructed by the course team). It also fails 3 of the 12 rows of the district table in Part 8.

Part 8: compare the fixes under one measure

Hana saw the issue on Monday evening at the fair, and pushed her own fix to a branch, hana/issue-12. You now have two fixes for one bug. Which one should the team keep?

The measure

Tests that you wrote with a fix tend to agree with that fix. A fair comparison needs one measure that comes from the requirement, not from any fix. Here, it is a district table: 12 postcodes, each with the zone that data/zones.csv gives it, or “refused”. It covers issue 12, issue 3, and both together (ls122ab). It also has the other districts that start with LS1, a one-digit district without a space (LS14AP), and a district outside the zones (LS19 7XB).

Download district_table.py into ~/projects, next to the project folder. Run it on your fix:

python -m pytest -q --rootdir=. ../district_table.py

Output

............                                                                                 [100%]
12 passed in 0.01s

The maintainers’ fix

Unzip the finished project into a new folder, ~/projects/finished, so that it does not replace yours. Then look at its history and at Hana’s commit:

macOS and Linux

cd ~/projects/finished/delivery-slots
git log --oneline --all --graph
git show hana/issue-12 -- delivery_slots/zones.py

Windows

cd ~\projects\finished\delivery-slots
git log --oneline --all --graph
git show hana/issue-12 -- delivery_slots/zones.py

Output of the graph (top)

* b7d4195 Match the whole postcode district (fixes issue 12)
| * 49e8534 Match the whole postcode district (fixes issue 12)
| * 6459f37 Add a failing test for issue 12: LS12 2AB is in zone west
|/
* ed951ba Add issue 12: no evening slots for LS12

Hana's zones.py (the changed part)

def district_of(postcode: str) -> str:
    """The district of a postcode: "LS12 2AB" -> "LS12", "ls122ab" -> "LS12".

    The last three characters of a UK postcode (the inward code, such as 2AB)
    are one digit and two letters. The district is everything before them, so
    this works with and without the space.
    """
    return normalise(postcode).replace(" ", "")[:-3]


def zone_for(postcode: str, zones: dict[str, str]) -> str:
    """The zone of a postcode, for example "west" for "LS5 3QR".

    The whole district must be a row of data/zones.csv: LS12 is not LS1.
    """
    try:
        return zones[district_of(postcode)]
    except KeyError:
        raise UnknownPostcode(f"We do not deliver to {normalise(postcode)}.") from None

Hana’s rule is the same as the assistant’s: remove the last three characters, then look up the exact district. She gave the rule a name, district_of, with its own tests, and a docstring that states the rule. Her commit also adds a note to issue 12: three bookings of this week are on the wrong van. Open issues/12-no-evening-slots-in-ls12.md on her branch to read it.

The comparison

The course measured five versions of zones.py, each in a fresh copy of the starter. Every version ran the same 39 tests (the starter’s 35 and your 4) and the district table. Two versions are first ideas that the course team wrote, labelled constructed.

• Real code (the starter, the recorded session)
• Constructed by the course team

The same 12 postcodes for every fix. Split: back to the first version. Longest: longest district first. Session: the assistant's fix. Hana: the maintainers' fix. Only the last two get all 12 right.

Text description of the chart

A column chart with five columns. The unfixed code passes 6 of 12 rows. Going back to the first version passes 9. Trying the longest district first passes 11. The assistant’s fix passes 12, and the maintainers’ fix passes 12.

Version	Code change	The 39 tests	District table	Rows that fail
unfixed (baseline)	–	35 passed, 4 failed	6 of 12	LS12, ls122ab, LS13, LS16, LS17, LS19
back to split (constructed)	+6 −10	38 passed, 1 failed	9 of 12	ls122ab, LS14AP, LS53QR: issue 3 is back
longest first (constructed)	+3 −2	all 39 passed	11 of 12	LS19 7XB gets the city zone
the assistant’s fix (real)	+4 −4	all 39 passed	12 of 12	none
the maintainers’ fix (constructed)	+15 −7	all 39 passed	12 of 12	none

What this tells us.

• The tests could not tell two of these apart. “Longest first” sorts the districts so that LS12 comes before LS1. It passes all 39 tests, issue 12’s tests too. But it still matches LS19 7XB to LS1, and sends a van to a district that Larkfield does not serve. Only the district table, written from the data, shows it.
• The assistant’s fix and the maintainers’ fix behave the same. They give the same result for all 12 rows, and for every malformed postcode of Part 7. The maintainers’ fix is larger because it adds a named function, its tests and a note.
• The size of a diff is not the measure. “Back to split” removes more lines than any other version, and it breaks issue 3.

The recommendation

Merge one fix, and close the other with thanks: both are the same rule. Amira’s fix should go first, because customers in LS12, LS13, LS16 and LS17 are affected now. Its pull request names the four districts, the test that failed first and the district table, as in Independent comprehension. When Hana is back, the team can add her district_of as a small refactor on top, if they like the name. That is a separate change with its own review.

Two things in Hana’s version came from knowing the business, not the code. She stated the rule in the docstring, and she thought about bookings that were already made. The issue asked for neither. The next part shows that the second one matters.

Part 9: run it end to end on new input

Go back to your own project, ~/projects/delivery-slots. Run the whole flow as the checkout and the depot do. Use new inputs, not the ones of your tests: Ruth’s order, a new customer in LS16, and a customer outside the zones. The time is Monday 15:00, before the cut-off:

python -m delivery_slots --now 2026-10-12T15:00 slots "LS12 2AB" --day 2026-10-13
python -m delivery_slots --now 2026-10-12T15:00 slots ls161aa --day 2026-10-14
python -m delivery_slots --now 2026-10-12T15:00 book L-40213 "LS12 2AB" --day 2026-10-13 --start 18:00
python -m delivery_slots --now 2026-10-12T15:00 book L-40214 "LS19 7XB" --day 2026-10-13 --start 08:00
python -m delivery_slots manifest --day 2026-10-13 --zone west
python -m delivery_slots manifest --day 2026-10-13 --zone city

Output

LS12 2AB is in zone west.
Slots on Tuesday 13 October 2026 (orders close Monday at 20:00):
  08:00-12:00  11 places left
  18:00-20:00  5 places left
LS161AA is in zone north.
Slots on Wednesday 14 October 2026 (orders close Tuesday at 20:00):
  08:00-12:00  11 places left
  12:00-16:00  11 places left
Booked order L-40213: LS12 2AB, zone west, Tuesday 13 October 2026 at 18:00.
Error: We do not deliver to LS19 7XB.
Zone west, Tuesday 13 October 2026: 3 deliveries
  08:00  L-40124  LS5 3QR
  18:00  L-40126  LS4 2TE
  18:00  L-40213  LS12 2AB
Zone city, Tuesday 13 October 2026: 5 deliveries
  08:00  L-40115  LS2 7HY
  08:00  L-40118  LS11 5BD
  08:00  L-40121  LS12 3PJ
  10:00  L-40128  LS3 1DH
  12:00  L-40131  LS13 4EX

What this tells us. Ruth’s order is booked on Tuesday evening, on the west van with her sister’s. A postcode typed without a space gets the north zone. LS19 is refused, with exit code 1. But the city driver’s list still has LS12 3PJ and LS13 4EX. Those customers booked before the fix, and each booking keeps the zone that it got then. The fix stops new mistakes. It does not move old bookings.

To find them all, make wrong_zone.py. It compares each booking’s zone with the zone that the fixed code gives today:

wrong_zone.py

from pathlib import Path

from delivery_slots.bookings import load_bookings
from delivery_slots.zones import load_zones, zone_for

zones = load_zones(Path("data") / "zones.csv")
for b in load_bookings(Path("data") / "bookings.csv"):
    right = zone_for(b.postcode, zones)
    if b.zone != right:
        print(f"{b.order}  {b.postcode:9} {b.day}  booked in {b.zone}, should be {right}")

Output of python wrong_zone.py

L-40107  LS16 7QR  2026-10-12  booked in city, should be north
L-40121  LS12 3PJ  2026-10-13  booked in city, should be west
L-40131  LS13 4EX  2026-10-13  booked in city, should be west

These are the same three orders as in Hana’s note. Send the list to Grace, so that customer support can move them. Then delete wrong_zone.py, and undo the booking with git restore data/bookings.csv. In the real shop, the checkout makes the booking, not your copy of the data.

Insights and limits

The assistant did four of the nine steps. Every fact that it gave was checked by a run before anyone used it.

Text description of the diagram

Nine steps. The assistant made the map (3 calls) and answered the targeted questions (1 call). It made the fix (1 call, 39 passed) and reviewed it: all seven claims were true when run. Amira reproduced the bug and shrank it to two rows of data. She wrote the failing test (4 failed, 35 passed). She compared the fixes: 12 of 12 for both full fixes, 11 of 12 for “longest first”. Then she ran it end to end: Ruth was booked, and three old bookings were on the wrong van. The session cost US$0.087 in total.

• The symptom was far from its cause. The slots were right; the zone was wrong, because of the row order. The map and one targeted question found it faster than reading every file.
• The history explained the bug, and an old test stopped the obvious fix from bringing issue 3 back.
• Passing tests are not the measure. “Longest first” passed all 39 tests. The district table, written from the data, caught it.
• The assistant was useful and checkable. Its map, trace and review were correct. One statement in turn 1 was wrong, and it cost two extra calls. The whole session cost US$0.087 at the deployment’s price when it was recorded: 6 calls, 30,327 tokens in and 2,619 out. Prices change. With a free tier or no assistant, you pay nothing.
• Limits. The project and the maintainers’ fix are written for the course. There is one recorded session: another run, model or assistant can give a different fix, and the district table is how you judge it. The fix does not check the postcode format: LS12 2A still gets the city zone. It does not move bookings that were already wrong. Everything was tested on macOS only.

Responsible use

• Customer data in prompts. Amira’s first prompt pasted the whole issue, with Ruth’s name, postcode and order number. Here the data is invented. In a real company, that sends personal data to a model provider. Check your company’s rules first. Send only what the question needs: the district LS12 and the observed slots, not the customer’s email.
• The data is personal too. data/bookings.csv has postcodes and order numbers. Do not paste it into a prompt, and do not let an assistant run scripts against the real booking system. wrong_zone.py only reads a copy, and a person decides what to move.
• Code that belongs to another team. Amira fixed code that Hana and Luis own, while they were away. She worked on a branch, wrote the failing test first, kept the diff small, and left the merge to the team’s normal review. Fix the bug, not the codebase: the refactor and the format check are separate issues for the owners.
• Say how you worked. The pull request says that an assistant wrote the four changed lines. It also gives the limits of the prompt, and how each claim was checked.

Make it a portfolio piece

Publish your finished project in a public repository. It has the MIT licence, so keep the LICENSE file. Add a CASE-STUDY.md:

CASE-STUDY.md (template)

# Fixing a bug in an unfamiliar codebase, with a coding assistant

## Problem
Issue 12: a customer in LS12 got the city zone's slots, with no evening slot.

## Approach
Map first, targeted questions, reproduce, shrink to two rows, failing test first,
assistant fix inside written limits, review by running, one measure for every fix.

## Results
4 failing tests → 41 passing; district table 12 of 12 (unfixed: 6; "longest first": 11).
Three bookings made before the fix were on the wrong van: listed for support.

## What the assistant did, and how I checked it
Map, trace, fix (+4 −4 lines) and a self-review: every claim checked by a run.

## Limits
No postcode format check yet (LS12 2A still gets city). One recorded session.

In two sentences, for an interview: “I fixed a customer’s bug in a codebase that I had never seen. I used an assistant for the map, the fix and a review, checked every claim with a run, and wrote the failing test first. A table of postcodes from the data showed that a fix which passed every test still sent vans to the wrong place.”

Go further:

• Write issue 13, “Check the postcode format”, with its own failing test for LS12 2A, and fix it.
• Turn wrong_zone.py into a check-bookings command with a test, as a separate pull request.
• The cut-off uses local time without a time zone. Find out what happens on the night when the clocks change, and write it as an issue.

Knowledge check

Check your understanding

Ruth saw the wrong slots, but the slot code was correct. Explain the path from the command to the symptom, and name two pieces of evidence that pointed to the cause.

Show the answer

cmd_slots asks zone_for for the zone, then asks slots_for for that zone’s slots. zone_for stopped at the first district that matched the start of the postcode, and LS1 is the first row of data/zones.csv. So LS12 2AB got city, and slots_for correctly returned the city zone’s slots. The evidence: the first output line, LS12 2AB is in zone city., and the two-row input, which gives city in one order and west in the other. git blame then showed that the lookup came from the fix of issue 3.

Check your understanding

'Longest first' passes all 39 tests. Why is that not enough to merge it, and what showed the problem?

Show the answer

The tests check the districts that their authors thought of. None of them has a district that Larkfield does not serve but that starts like a served one. “Longest first” still matches LS19 7XB to LS1, so it would book a city van for a place outside the zones. The district table showed it. Its 12 rows come from the data file and the two issues, not from any fix, and one of them is LS19 7XB, expected “refused”.

Check your understanding

The assistant's review listed seven postcodes that could still go wrong. What did Amira do with that list, and why did she not ask the assistant to fix them all?

Show the answer

She ran them, on the fix and on the old code. The claims were true, and the old code gave a zone to every one of those postcodes too. So the fix did not cause them: they are a separate problem, the postcode format. The issue did not ask for it, and the code belongs to another team. She added tests for the two behaviours that the fix changed on purpose (ls122ab, LS19 7XB). She wrote the format problem down as a new issue for the owners.

Recap

• Map first, yourself, then with the assistant, and check its map against the code.
• Targeted questions name one function and one input. Every answer is a claim: check it with a run.
• Reproduce, read the history, shrink: git blame led to issue 3; two rows of data showed that the first match depended on their order.
• Failing test first (4 failed, 35 passed), then the assistant’s fix inside written limits (39 passed).
• Compare under one measure from the requirement: 12 of 12 for both full fixes, 11 for “longest first”, which passed every test.
• End to end on new input showed what the fix cannot do: three bookings were already on the wrong van.

Next, the other case study: move a small project across a breaking change of a dependency. Check each suggested change against the changelog and the tests.

Next: Upgrade a dependency safely