Reproduce first

What you will be able to do

Make a bug happen again on purpose, with the smallest input that still shows it, before you or an assistant change any code. Then check that the cause is in the code and not in your computer’s setup.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• Grace’s bug report and its two-row input, from Supply relevant context.
• A minimal failing example and a regression test, from Read failures and Focused tests in the Python course.
• A virtual environment made from requirements-lock.txt, from Isolate dependencies.

What you need on your computer

• The project ~/projects/ticket-cleaner-assist at the end of Module 3: python -m pytest prints 34 passed, and git status says working tree clean. Its environment is active. If the project is missing or broken, follow the reset steps of Context and boundaries with the snapshot end-of-m03.
• The labs clone ~/projects/nextia-learning-labs, and about 40 MB of free space for a second environment.
• No account, no key and no money. This lesson calls no assistant.

The Wi-Fi that does not work

Your laptop says “No internet”. Before you phone the help line, you do three things almost without thinking.

First, you try again: you open a second web page. If it loads, the problem was a moment, not a fault. Second, you make the question small: does every page fail, or only one site? Third, you try another device. If your phone has no internet either, the laptop is fine and the router is the problem. If the phone works, the problem is in the laptop.

Only then do you phone. And you can say exactly what happens: “No page loads on the laptop. The phone works on the same Wi-Fi.” The person at the help line starts at the right place.

The problem: a fix that you cannot check

Tomás wants to save time. He gives an assistant one line: “Grace says the report is wrong for the new web form export. Fix it.” The course team recorded that request with the smaller model (transcript bug-fix-blind-small). The assistant asked to see parsing.py and the export, got them, and changed one line.

Was the fix right? Look at what the session’s own checks show:

The project's tests, before the session and after it

tests: 24 passed
tests: 24 passed

The result is the same before and after. Nobody ran Grace’s command before the change, so nobody saw the bug with their own eyes. And no test fails without the fix. In this run the change was right: the course team’s hidden checks of the bug passed. But nothing in the session shows it. If the line had been wrong, these two lines would look exactly the same.

There is a second risk. Suppose the report was wrong only on Grace’s computer, because of an old copy of the program. Then any change to the code is a change for nothing, and the real cause stays.

The idea: reproduce first

To reproduce a bug is to make it happen again on purpose, with a command and an input that anyone can run. The result is a reproduction: the steps, the input, the observed output and the expected output. Developers often say “a repro”.

A good reproduction answers three questions, in this order:

1. Can I see it? Run the reporter’s own command, on their own input, and compare with what they wrote.
2. What is the smallest input that still shows it? That is the minimal failing input. The Python course called it a minimal failing example.
3. Is it the code, or the environment? Run the minimal input in a fresh setup. If the result is the same, the cause is in the code.

This is the Wi-Fi check, for a program. You do it before the assistant writes any code, for two reasons. You can then tell whether a change helps. And the reproduction is the best context that you can give the assistant: it is short, exact and checkable.

A tiny example: halve the input

Grace’s export has 12 rows. To find a small input, you do not need to understand the bug yet. Use a simple method: cut the input in half, run the program, and keep the half that still shows the bug. Then cut again.

The bug shows when one team has two lines in by_category. The diagram has each run of the course team, with its real output:

Halve, keep the part that still fails, then make each value plain. The bug survives every cut that keeps two spellings of one team.

Text description of the diagram

The diagram is a table of six runs of the program, each with its real by_category:

1. All 12 rows of Grace’s export: {"Billing": 1, "LOGIN": 1, "Login": 1, "account": 1, "billing": 2, "login": 2, "shipping": 2}. The bug shows: login and billing are split.
2. Rows 1 to 6 (T-3001 to T-3006): {"Login": 1, "billing": 2, "login": 2}. Still split, so keep this half.
3. Rows 1 to 3 (login, Sign-in, Login): {"Login": 1, "login": 2}. Still split, so keep these.
4. Rows 1 and 2 (login, Sign-in): {"login": 2}. That is correct: the bug is not in these two rows, so try the other part.
5. Rows 2 and 3 (Sign-in, Login): {"Login": 1, "login": 1}. Split, with two rows: keep them.
6. The same two rows, with Sign-in changed to login: {"Login": 1, "login": 1}. Still split, so the web form’s name is not needed. The bug report uses LOGIN, with the same result.

At the bottom: 12 rows became 2 rows in 5 runs. Red rows show the bug; the green row does not.

Two moves made the input small. Halving removed rows. Plain values removed details: the web form’s name Sign-in was not part of the problem. With one row, Login alone, the report has one line, {"Login": 1}. The name is already wrong there, but Grace’s symptom, two lines for one team, needs two rows.

Five runs took about two minutes. The result is an input that a person reads in five seconds, and an assistant in a few tokens. Halving is good when you cannot guess the cause. When you can guess, as Amira could in Supply relevant context, start from the guess, and use a run to check it.

Reproduce Grace’s bug in your project

Your project has changed since Grace wrote her email: it now has the --category option. So reproduce the bug in the code that you have today, not in the code that she used.

1. Check your starting point:

   git status
   python -m pytest

   Output

   On branch feature/category-option
   nothing to commit, working tree clean
   ============================== 34 passed in 0.05s ==============================

2. Run Grace’s own command on her export:

   python -m ticket_cleaner data/web-form-export.csv

   Output

   12 rows: 12 valid, 0 rejected.
   Report: reports/summary.json

   Open reports/summary.json. by_category has the seven lines of her email: Billing, LOGIN and Login next to billing and login. You see what she saw.

3. Now try the new option, with the login team:

   python -m ticket_cleaner data/web-form-export.csv --category login

   The report has "selected": 2 and "by_category": {"login": 2}. The export has four login tickets, all open. So the same bug also breaks your new option: it misses Login and LOGIN. Nobody reported this yet. You found it because you reproduced the bug in today’s code.

4. In your editor, make a new file minimal.csv in the project folder, with the three lines of the bug report:

   minimal.csv

   id,status,category,priority,subject
   T-1,open,login,1,One
   T-2,open,LOGIN,1,Two

   Run the program on it, with its own report file:

   python -m ticket_cleaner minimal.csv --output reports/minimal.json

   Output

   2 rows: 2 valid, 0 rejected.
   Report: reports/minimal.json

   reports/minimal.json has "by_category": {"LOGIN": 1, "login": 1}. Expected: {"login": 2}. You can see the bug, and the input is two rows.

Environment or code?

The environment of a program is everything around its code. That is the Python version, the packages, the folder that you run it from, the environment variables and the operating system. An environment problem goes away when you fix the setup. A code defect stays in every correct setup, because the cause is in the code itself.

The two need different fixes. A code change cannot repair a missing package. A new Python cannot repair a wrong line. So, before anyone changes code, run the minimal input in a clean setup. That is a fresh copy of the code, with a new environment from the lock file.

5. Check that you did not change any code yourself:

   git status --short

   Output

   ?? minimal.csv

   Only the new input file. No file of the program is changed.

6. Make a fresh copy of the snapshot end-of-m03 next to your project. The name ticket-cleaner-check keeps it apart from your work:

   macOS and Linux

   cd ~/projects
   cp -R nextia-learning-labs/C15/snapshots/end-of-m03 ticket-cleaner-check
   cd ticket-cleaner-check

   Windows

   cd ~\projects
   Copy-Item -Recurse nextia-learning-labs\C15\snapshots\end-of-m03 ticket-cleaner-check
   cd ticket-cleaner-check

7. Leave your project’s environment, and make a new one in the copy, from the lock file:

   macOS and Linux

   deactivate
   python3 -m venv .venv
   source .venv/bin/activate
   python -m pip install -q -r requirements-lock.txt
   python -m pytest

   Windows

   deactivate
   python -m venv .venv
   .venv\Scripts\Activate.ps1
   python -m pip install -q -r requirements-lock.txt
   python -m pytest

   Output (the last line)

   ============================== 34 passed in 0.05s ==============================

8. Copy the minimal input into the fresh copy, and run it there:

   macOS and Linux

   cp ../ticket-cleaner-assist/minimal.csv .
   python -m ticket_cleaner minimal.csv --output reports/minimal.json

   Windows

   Copy-Item ..\ticket-cleaner-assist\minimal.csv .
   python -m ticket_cleaner minimal.csv --output reports/minimal.json

   reports/minimal.json has {"LOGIN": 1, "login": 1} again. New copy, new environment, same wrong result.

The course team also ran the two rows with Python 3.12.13 and 3.13.16, each with the same lock file. Both printed {'LOGIN': 1, 'login': 1}. So the bug does not depend on your edits, your environment or the Python version: it is a code defect.

What does an environment problem look like? You can make one in a few seconds.

9. Go up one folder, to ~/projects, and run the program from there:

   cd ..
   python -m ticket_cleaner ticket-cleaner-check/minimal.csv

   Output (macOS)

   ~/projects/ticket-cleaner-check/.venv/bin/python: No module named ticket_cleaner

   Python looks for the package ticket_cleaner in the current folder, and ~/projects does not have it. The message is about the setup, not about categories. Fix the setup, and the error goes away.

The diagram puts the five runs side by side:

A code defect gives the same wrong answer everywhere. An environment problem gives an error about the setup, and it changes when the setup changes.

Text description of the diagram

The diagram is a table of five runs of the two-row input. Expected: {"login": 2}.

1. Your project, with your .venv: {"LOGIN": 1, "login": 1}. The bug shows.
2. A fresh copy of end-of-m03, with a new .venv from requirements-lock.txt: 34 passed, then {"LOGIN": 1, "login": 1}. The same, so it is not your edits and not your environment.
3. Python 3.12.13 and 3.13.16, run by the course team: {"LOGIN": 1, "login": 1}. The same, so it is not the Python version.
4. The wrong folder, ~/projects: No module named ticket_cleaner. An environment problem: the command cannot find the code.
5. No environment active, with macOS’s own Python 3.9.6: ModuleNotFoundError: No module named 'httpx'. An environment problem: a package is missing. This run is from the course team’s computer; yours can print something else.

At the bottom: the same wrong result in every clean environment is a code defect. An error about the setup, which changes when the setup changes, is an environment problem.

Look at the shape of each result. The code defect gives a normal-looking report with a wrong value in it, and exit code 0. The environment problems stop the program with an error that names a module, a file or a version. That is a strong hint, but not a rule. Some environment problems give wrong values too: for example, an old copy of a data file, or a different setting in .env. The fresh copy catches those as well, because it has none of your local files.

10. Go back to your project and its environment. Delete the input file: the bug report keeps its three lines.

    macOS and Linux

    cd ticket-cleaner-assist
    source .venv/bin/activate
    rm minimal.csv
    git status --short

    Windows

    cd ticket-cleaner-assist
    .venv\Scripts\Activate.ps1
    Remove-Item minimal.csv
    git status --short

    git status --short prints nothing. Keep ticket-cleaner-check: the module practice at the end of Choose meaningful checks uses it.

This lesson changes no file in your project. Its result is knowledge: a reproduction that you ran yourself, a second symptom, and the answer “code, not environment”.

Larkfield: Amira answers Grace

Amira writes back before she changes anything:

I can see it: two rows, login and LOGIN, give two lines in the report. It also happens in a fresh copy with a new environment, so it is in the program, not on your computer. It also affects the new --category option: --category login finds 2 of the 4 login tickets. I will send you the fix with a test.

Grace now knows three things: the bug is real, it is not her fault, and the team found a second effect. Amira also has the first part of her prompt for the assistant. The bug report’s “Environment” section is now a fact that she checked, not a guess.

When to reproduce first, and what it costs

Reproduce every bug before you change code, and before you ask an assistant to change it. It costs minutes: for Grace’s bug, about ten. Most of that time comes back later, because the reproduction becomes the regression test of the next lesson.

Some bugs are hard to reproduce. A failure can depend on timing, on the network or on data that you may not copy, such as real customers’ tickets. Then collect more facts: the exact command, the versions, the settings, the time. Make a synthetic input with the same shape, as Grace’s export is. If you still cannot reproduce it, say so in the report. Do not ask an assistant to fix a bug that nobody can see. It will change something, and you cannot tell whether that helped.

A reproduction does not find the cause. It tells you what fails, and what the failure does not depend on. Finding why is the next step, with a hypothesis and a targeted check.

Common mistake

'The environment is broken'

Symptom: Tomás wants to check the bug report. He opens a new terminal in ~/projects, activates an environment, and runs:

python -m ticket_cleaner ticket-cleaner-check/minimal.csv
~/projects/ticket-cleaner-check/.venv/bin/python: No module named ticket_cleaner

He decides that the environment is broken. He is about to ask an assistant to “fix the imports”.

Cause: the command ran in the wrong folder. Python finds ticket_cleaner only from the project folder, as in the Python course. The error says nothing about categories, so it is not Grace’s bug at all. Tomás reproduced a different problem: his own.

Fix: compare the symptom with the report before you go on. Grace saw a report with wrong lines, not an error. Run the command from the project folder, as the bug report says, and the program prints the split report. A code change for “the imports” would hide a setup mistake inside the code.

Your turn: the smallest input for the second symptom

In step 3, --category login found 2 of the 4 login tickets. Find the smallest input that still shows this symptom. Predict first: do you need a new file, or does minimal.csv already show it? Then run it in ticket-cleaner-check. Can one row show it?

Show the answer

minimal.csv already shows it:

python -m ticket_cleaner minimal.csv --output reports/minimal.json --category login

The report has "selected": 1 and "by_category": {"login": 1}. Expected: 2. The ticket T-2 with LOGIN is missing.

One row is enough too. A file with only the header and T-2,open,LOGIN,1,Two gives "selected": 0 with --category login, and an empty by_category. Expected: 1. The course team ran both.

So the two symptoms have the same minimal input. That is a strong hint that they have one cause, so you do not need a second bug report. Add one line to the observation of the first one, as Amira’s email did.

See it end to endFix a bug in an unfamiliar codebaseCase study

The case study reproduces a customer’s bug in an unfamiliar project, and shrinks it to two rows of data and one postcode.

Knowledge check

Check your understanding

What are the three questions of a good reproduction? Give the answer for Grace's bug for each.

Show the answer

• Can I see it? Yes: python -m ticket_cleaner data/web-form-export.csv gives Billing, LOGIN and Login next to billing and login, as in Grace’s email.
• What is the smallest input that still shows it? Two rows, login and LOGIN: {"LOGIN": 1, "login": 1} instead of {"login": 2}.
• Is it the code or the environment? The code. A fresh copy with a new environment from the lock file gives the same result. So do Python 3.12 and 3.13.

Check your understanding

Sort each symptom into 'probably the environment' or 'probably the code', and say how you would check: (a) ModuleNotFoundError: No module named 'httpx'; (b) a report with 'LOGIN': 1 next to 'login': 1, exit code 0; (c) the report is right on Amira's computer and wrong on Grace's, with the same file.

Show the answer

• (a) Probably the environment. A package is missing: the virtual environment is not active, or it was not made from the lock file. Check with python -m pip show httpx, or activate the environment and run again.
• (b) Probably the code. The program runs normally and gives a wrong value. Check in a fresh copy with a new environment: if the result is the same, it is a code defect.
• (c) Probably the environment, in the wide sense: a different version of the program, a different setting, or a different copy of the data. Compare the versions (git log -1, python --version, the lock file), and run the same minimal input on both computers.

Check your understanding

Tomás says: 'The assistant fixed it, and the tests still pass.' In the recorded small-model session, the tests printed 24 passed before and after the fix. What does that tell you about the fix, and what was missing?

Show the answer

It tells you almost nothing about the fix. The tests passed before the fix too, so none of them tests this bug. “24 passed” after the change only shows that the change broke none of them. What was missing is a reproduction: Grace’s command or the two-row input, run before and after the change. Better still is a test that fails before the fix and passes after it. That is the subject of the next two lessons.

Recap

• To reproduce a bug is to make it happen again on purpose. A reproduction has a command, a minimal failing input, and the observed and the expected output.
• Halve the input and keep the part that still fails; then make each value plain. Grace’s 12 rows became 2 in 5 runs.
• A code defect gives the same wrong result in every clean environment. An environment problem gives an error about the setup, which changes when the setup changes. Check with a fresh copy and a new environment from the lock file.
• Reproduce in today’s code: it showed a second symptom, --category login finds 2 of 4.
• Your project is unchanged. The copy ticket-cleaner-check stays for the module practice.

Next, you look for the cause. You ask the assistant for a hypothesis and a check, not for a fix.

Next: Ask for evidence