Document decisions

What you will be able to do

Write down how to use the project, what it cannot do, and why it is built the way it is. Put each fact in one place. Give each claim its evidence: a test, a command that you ran, or the name of the person who must check it.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• What a README contains, from Quality and handoff in Python for Practical AI Engineering.
• How to write a commit message, from Track a first change in Git and Team Development Essentials.
• The decisions of this course so far: the task brief, the rejected candidates of Module 3, and the fix of Module 4. Also the change of Refactor selectively.

What you need on your computer

• The project ~/projects/ticket-cleaner-assist after the last lesson, with its environment active. python -m pytest prints 40 passed, and git status --short prints nothing.
• The labs folder ~/projects/nextia-learning-labs, with C15/snapshots/end-of-m05/.
• If the project is missing or broken, use the reset steps under Module check, then do the last lesson again.

A note on the recipe card

A family recipe card for a cake says “100 g sugar”. In the margin, in pencil: “Not 200: the children found it too sweet. Honey burns, do not use it.”

Years later, someone bakes the cake for the first time. Without the margin note, they might think that 100 g is a typing mistake and use 200. Or they might try honey, because it sounds healthier. The note saves them from repeating two old experiments. It also says why, so they can decide again if the reason changes.

A recipe without the note works. A recipe with a wrong note is worse than no note. “Bake for 20 minutes”, when the cake needs 40, sends people into a failure that they trust.

The problem: documents that nobody checked

Look at a paragraph of README.md in your project. It came with the teammate’s change in the snapshot start:

README.md (the paragraph that came with the teammate's change)

The new web form uses other names for some categories, for example `Sign-in` and `Invoice`. The report counts them under its own names (`login`, `billing`); `ticket_cleaner/records.py` lists them.

Every word is true for the two examples. But a reader understands “one line per team”. Grace’s export had three lines for one team: LOGIN, Login and login. Nobody checked the README’s promise against the export, and no test did either. The document looked finished, so nobody asked.

An assistant makes this easier to get wrong, because it writes confident summaries. In the session refactor-open, it said that it would “make JSON writes safer”. That may be true. But nothing in the session measured it, and no test checks it. Copy that sentence into a document, and the document now contains a claim that nobody checked.

The opposite problem is just as real: a decision with no record. Tomás reads the README’s limitation about biling, and he has an idea:

“Let’s add a fuzzy-matching package, so that biling counts as billing. One line, and the report is cleaner.”

Amira has two reasons against it. The report must not guess, because a wrong guess moves a ticket to another team. And in Verify dependencies, a fuzzy-matching package already failed the question “is it needed?”. If these reasons live only in her memory, the team has the same discussion every few months. Or someone adds the package while she is on holiday.

The idea: two files, two jobs, and evidence for each claim

A project needs two kinds of documentation, for two kinds of readers:

• The README tells a user or a new developer what the project does and how to set it up and run it. It also says what the project cannot do. A limitation is something that the project does not do, said plainly, so that nobody relies on it.
• A decision record tells the next developer why the project is built this way. Each entry has the choice, the reason, how it was checked, and the alternatives that were rejected. Many teams call these architecture decision records. In ticket-cleaner, they are one short file, DECISIONS.md.

One rule holds for both: write no claim that you did not check. Each sentence that says what the program does needs evidence. A test is the best evidence, because it checks the claim again after every change. A command that you ran is good evidence, for a claim that rarely changes. And when only a person can check a claim, write who that person is.

Each kind of knowledge has one place. The README links to DECISIONS.md; it does not copy it.

Text description of the diagram

Five places, each with one job:

• README.md: how to set up, run and use the program, and what it cannot do. For users and new developers.
• DECISIONS.md: why. Each important choice, how it was checked, and what was rejected.
• docs/tasks/: what was asked. The brief with its criteria, and the bug report.
• Commit messages: what one step changed.
• Docstrings: what one function promises.

Put a fact in one place, and link to it from the others. Two copies of a fact drift apart.

A tiny example: one decision, every part visible

Here is decision 3 of DECISIONS.md, about the change of the last lesson. Read it as four parts:

DECISIONS.md (decision 3)

## 3. Category names are cleaned in one place

`clean_category` (`ticket_cleaner/parsing.py`) cleans a category in the same
way as the status and the ID: no spaces around it, small letters. Then it
changes the web form's names to ours. `--category` uses the same function, so
`--category Sign-in` selects the login tickets.

- Grace's bug: `clean_category` looked up the alias in small letters but kept
  the original text for other names, so `LOGIN` and `login` were two
  categories. The fix is one line: clean the text first.
- Checked by: `test_capital_letters_do_not_make_a_new_category` (it fails on
  the code before the fix), `test_web_form_export_has_one_line_per_team` and
  `test_web_form_names_work_in_the_category_option`.

1. The choice, in the heading: one place for the cleaning rule.
2. The reason: the status and the ID are cleaned the same way, and Grace’s bug came from a second rule.
3. The evidence: three tests, by name. Anyone can run them: python -m pytest -k web_form_names_work prints 1 passed, 41 deselected.
4. The rejected alternative: decision 3 has none, because nobody proposed one. Decisions 1, 2 and 4 each name a rejected candidate from Module 3, with the reason.

It takes two minutes to read. It would take an hour to find out again from the code and the history.

Larkfield: Amira documents the change

Amira writes the documents last, when the code is final. Otherwise she documents things that change again. She also starts with a test, so that the README cannot drift without someone noticing.

1. Make the README checkable. Copy the README test from the snapshot, and read it:

   macOS and Linux

   cp ../nextia-learning-labs/C15/snapshots/end-of-m05/tests/test_readme.py tests/

   Windows

   Copy-Item ..\nextia-learning-labs\C15\snapshots\end-of-m05\tests\test_readme.py tests\

   tests/test_readme.py

   """The README's claims are checked, so that it stays true after a change."""
   
   import re
   from pathlib import Path
   
   from ticket_cleaner.cli import main, make_parser
   
   README = Path("README.md").read_text(encoding="utf-8")
   
   
   def test_the_readme_shows_the_real_output(tmp_path, capsys):
       assert main(["data/tickets.csv", "--output", str(tmp_path / "s.json")]) == 0
       first_line = capsys.readouterr().out.splitlines()[0]
       assert first_line == "15 rows: 9 valid, 6 rejected."
       assert first_line in README
   
   
   def test_every_option_of_the_command_is_in_the_readme():
       options = set(re.findall(r"--[a-z-]+", make_parser().format_help()))
       options.discard("--help")
       documented = set(re.findall(r"^\| `(--[a-z-]+)`", README, re.MULTILINE))
       assert options == documented

   The first test runs the program and checks that the README shows its real first line. The second compares the options in --help with the rows of the README’s options table.

2. Run it:

   python -m pytest tests/test_readme.py -v

   Output (the end)

   tests/test_readme.py::test_the_readme_shows_the_real_output PASSED       [ 50%]
   tests/test_readme.py::test_every_option_of_the_command_is_in_the_readme PASSED [100%]
   ============================== 2 passed in 0.03s ===============================

3. See it catch a drift. In your editor, delete the --category row of the options table in README.md, and save. Predict the result, then run the test again:

   python -m pytest tests/test_readme.py

   Output (part)

   E       AssertionError: assert {'--category'...', '--status'} == {'--output', '--status'}
   E
   E         Extra items in the left set:
   E         '--category'
   FAILED tests/test_readme.py::test_every_option_of_the_command_is_in_the_readme
   ========================= 1 failed, 1 passed in 0.03s ==========================

   “The left set” is the options of --help. Put the row back:

   git restore README.md

4. Update the README. Make four edits, and check each claim before you write it. The diff shows them:

   git diff README.md (part)

   The new web form uses other names for some categories, for example `Sign-in` and `Invoice`. The report counts them under its own names (`login`, `billing`); `ticket_cleaner/records.py` lists them.
   Categories are counted in small letters, without spaces around them: `LOGIN`, `Login` and `login` are one category. The new web form uses other names for some categories, for example `Sign-in` and `Invoice`. The report counts them under its own names (`login`, `billing`); `ticket_cleaner/records.py` lists them. `--category` cleans the names that you give in the same way, so `--category Sign-in` selects the login tickets.

   git diff README.md (part)

   ## Limitations
   
   +- A category name with a typing mistake (`biling`) is a category of its own. The report does not guess what was meant.
   +- `--category` with a name that no ticket has gives an empty selection, not an error.
   +- The list of the web form's names is in `ticket_cleaner/records.py`. A new name on the form needs a new line there.
   
   `DECISIONS.md` explains the main choices.

   The other two edits are small. One sentence about ASSISTANT-SCOPE.md goes under “Work on the code”. Four new lines go in “Project layout”: docs/, ASSISTANT-SCOPE.md, DECISIONS.md and LICENSE. To save typing, copy the README from the snapshot, then read git diff README.md line by line:

   macOS and Linux

   cp ../nextia-learning-labs/C15/snapshots/end-of-m05/README.md .

   Windows

   Copy-Item ..\nextia-learning-labs\C15\snapshots\end-of-m05\README.md .

5. Check the limitations with real commands, not from memory:

   python -m ticket_cleaner data/web-form-export.csv --category biling

   The exit code is 0, and the report has "selected": 0, "by_category": {} and "categories": ["biling"]. The first two limitations are true. Then check the third:

   python -c "from ticket_cleaner.parsing import clean_category; print(clean_category('Log in'), clean_category('Log-In'), clean_category('Signin'))"

   Output

   log in login signin

   Log-In is on the list, so it becomes login. Log in with a space and Signin are not, so each would be its own category. The third limitation says exactly that: a new name needs a new line.

6. Add the decision record. Copy DECISIONS.md from the snapshot, and read it all. It has five decisions and one list of open questions:

   macOS and Linux

   cp ../nextia-learning-labs/C15/snapshots/end-of-m05/DECISIONS.md .

   Windows

   Copy-Item ..\nextia-learning-labs\C15\snapshots\end-of-m05\DECISIONS.md .

   Decision	The rejected alternative
   1. --category filters the selection, not the file	candidate C, which filtered before the summary
   2. The report without --category stays exactly the same	candidate A, with "category": null in every report
   3. Category names are cleaned in one place	none
   4. No new dependency	candidate D, with Unidecode
   5. Not done, on purpose	fuzzy matching, a warning for an unknown category, a --clean option

   The last section, “Still needs a human review”, lists what no test can check: the alias list, and real exports.

7. Run every test that DECISIONS.md names. -k selects the tests whose names contain one of the words:

   python -m pytest -v -k "whole_file_counts or report_for_some_categories or baseline or categories_key_only or capital_letters or one_line_per_team or web_form_names_work"

   Output (the end)

   ======================= 9 passed, 33 deselected in 0.03s =======================

   Each named test exists, and each one passes. A claim of “checked by” is now checked.

8. Run all the checks, and commit the three files:

   python -m pytest
   ruff check .
   ruff format --check .
   git add -A
   git commit -m "Document the cleaning rule, the limitations and the decisions"

   Output (the end of each command)

   ============================== 42 passed in 0.04s ==============================
   All checks passed!
   23 files already formatted
   [feature/category-option 51b4ddf] Document the cleaning rule, the limitations and the decisions
    3 files changed, 104 insertions(+), 1 deletion(-)
    create mode 100644 DECISIONS.md
    create mode 100644 tests/test_readme.py

   Your commit hash is different. ruff format --check . now counts 23 files, not 21: ruff also counts Markdown files, so tests/test_readme.py and DECISIONS.md add one each.

Now map each claim of the two documents to its evidence:

Each claim has a test, a command, or a named person. The one claim without any evidence stayed out of the documents.

Text description of the diagram

Seven claims and their evidence:

• “LOGIN, Login and login are one category”: test_capital_letters_do_not_make_a_new_category, which fails on the code before the fix.
• “--category Sign-in selects the login tickets”: test_web_form_names_work_in_the_category_option.
• “Every option is in the options table”: test_every_option_of_the_command_is_in_the_readme, which fails when a row is deleted.
• “15 rows: 9 valid, 6 rejected.”: test_the_readme_shows_the_real_output.
• “A typing mistake (biling) is a category of its own”: a command, --category biling → selected 0, exit code 0.
• “The alias list matches the web form”: no test can check it. Grace’s team owns the form, so it is under “Still needs a human review”.
• “JSON writes are safer”, from an assistant’s summary: no evidence, so it is not in any document.

When to document, and what it costs

Write a decision when a reasonable person could choose differently, or has already proposed to. Every rejected candidate of Module 3 is such a case. So is each thing that you chose not to do. Do not record decisions that nobody would question, such as “we use pytest”. A decisions file that lists everything is as hard to use as none.

Update the README in the same commit as the change that it describes. A README that is one commit behind is wrong for that commit. When a claim can be a test, make it a test, as tests/test_readme.py does. A test cannot check every sentence, but it catches the drift that happens most: a new option, or a changed output line.

Do not document what you did not check. If an assistant drafts a README section, treat each sentence as a claim in a diff. Check it, or delete it, or move it to “Still needs a human review” with a name.

The cost is real. Documents take time to write, and every claim that you keep is a claim that you must keep true. That is why the documents here are short: five decisions, three limitations. Short documents get read and kept up to date.

Common mistake

DECISIONS.md names a test that does not exist

Symptom: a decision says “Checked by: test_category_option_accepts_web_form_names”. You run it by name, and pytest runs nothing:

python -m pytest -k test_category_option_accepts_web_form_names

Output (the end)

collected 42 items / 42 deselected / 0 selected

============================ 42 deselected in 0.06s ============================

The exit code is 5: pytest found no test to run. It printed no error in red, so it is easy to miss.

Cause: the test was renamed, or it was never written. An assistant can also invent a test name that sounds right. Either way, the decision now cites evidence that does not exist.

Diagnose: list the tests with their names, and search for the words:

python -m pytest --collect-only -q

Fix: put the real name in DECISIONS.md, here test_web_form_names_work_in_the_category_option. Then run it by name: 1 passed, 41 deselected. Do this for every name each time that you change the file, as in step 7.

Your turn: check a draft paragraph

Imagine that an assistant drafted this paragraph for the README (constructed for the course). Before you accept it, mark each of its four sentences: checked by a test, checked by a command, needs a person, or false. Run commands to decide.

A draft paragraph (constructed)

Category names are cleaned, so every spelling of a team becomes one category.
--category accepts the web form's names, such as Sign-in and Invoice.
The cleaning is fully tested.
A misspelt category, such as biling, is corrected automatically.

Then write one new entry for DECISIONS.md: Tomás’s proposal of a fuzzy-matching package for biling. Use the four parts: the choice, the reason, the evidence, and the rejected alternative.

Show a worked solution

1. False. clean_category('Log in') gives 'log in', a category of its own. Only the spellings on the list become one category. Write instead: “Capital letters and spaces around a name do not matter, and the web form’s names in records.py become ours.”
2. Checked by a test. test_web_form_names_work_in_the_category_option checks Sign-in. The command --category Invoice selects 3 billing tickets in Grace’s export.
3. Not a claim that anyone can check. “Fully tested” has no measure. Delete it, or name what the tests cover.
4. False. --category biling gives "selected": 0. The README’s limitation says the opposite, and it is the true one.

A possible entry:

DECISIONS.md (a possible entry)

## 6. Misspelt categories are not corrected

A misspelt category (`biling`) stays a category of its own. The report must
not guess: a wrong guess moves a ticket to another team, and nobody sees it.
A separate line shows the mistake, and Grace's team can fix the source.

- Checked by: `python -m ticket_cleaner data/web-form-export.csv --category
  biling` gives `"selected": 0`, exit code 0.
- Rejected: a fuzzy-matching package (proposed by Tomás). It guesses, it is a
  new dependency, and the alias list in `records.py` already handles the
  known names.

Decision 5 of the snapshot already says “No fuzzy matching”, in one line. Your longer entry is fine for practice. In the project, add the reason to decision 5 instead of a second entry for the same choice.

See it end to endUpgrade a dependency safelyCase study

The case study records an upgrade in DECISIONS.md: each code change next to its changelog entry, the checks, and what was changed or not taken.

Knowledge check

Check your understanding

What is the difference between the job of README.md and the job of DECISIONS.md? Give one sentence from each file of ticket-cleaner that could not move to the other.

Show the answer

The README tells a user how to set up, run and use the program, and what it cannot do. DECISIONS.md tells the next developer why it is built this way, how each choice was checked, and what was rejected. “A category name with a typing mistake (biling) is a category of its own” belongs in the README. A user needs it to read the report. “Rejected: a candidate change that removed the other categories’ tickets in cli.py, before the summary” belongs in DECISIONS.md. A user does not need it, but a developer who wants to change the filter does.

Check your understanding

The README of start said that the report counts the web form's names under its own names. Every word was true. Why was it still a problem, and what would have caught it?

Show the answer

A reader understands it as “one line per team”, and that was false: Grace’s export had LOGIN, Login and login as three lines. The claim was true only for the examples that it named. A test that runs the program on an export with mixed spellings would have caught it. That test exists now: test_web_form_export_has_one_line_per_team. Without such a test, a document is a promise that nobody keeps checking.

Check your understanding

Predict: you add a new option --since to the command, but you do not change the README. Which test fails, and with what message?

Show the answer

test_every_option_of_the_command_is_in_the_readme fails. The options in --help now contain --since, and the README’s table does not. pytest reports the difference as “Extra items in the left set” with '--since', as it did for --category in step 3. The test cannot check that your description of the option is right, only that it exists. You still read your own sentence against the code.

Recap

• The README says how to use the project and what it cannot do (its limitations). A decision record, DECISIONS.md, says why: the choice, the reason, the evidence, and the rejected alternatives.
• Write no claim that you did not check. Evidence is a test, a command that you ran, or a named person under “Still needs a human review”.
• An assistant’s summary is a claim, not evidence. “Safer” without a measurement stays out of the documents.
• tests/test_readme.py keeps two README claims true after every change. Run every test that DECISIONS.md names, by name.
• Your project: 42 passed, with the README, DECISIONS.md and the README test in one commit.

Next, you explain every consequential line of the whole change, and you open a pull request that a reviewer can check without you.

Next: Independent comprehension