Verify dependencies

What you will be able to do

Before a package goes into the project, check four things. It exists, the project needs it, it is maintained and fits your versions, and its licence fits. Check every function that a change calls, too. Do all of it before you install anything.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• What a package, pip, requirements.txt and the lock file are, from Isolate dependencies in Python for Practical AI Engineering.
• How to review a diff with the checklist, from Inspect the diff.
• Why a model can write a name that does not exist, from Assistant interaction modes.

What you need on your computer

• The project ~/projects/ticket-cleaner-assist after the last lesson, on feature/category-option, with its environment active. python -m pytest prints 34 passed, and git status says working tree clean.
• A web browser, and an internet connection for two read-only lookups on pypi.org.
• No account and no money. You install nothing in this lesson.

A part for your bicycle

Your bicycle makes a noise. Someone in a forum writes: “Fit a Hartwell X2 chain. That fixes it.” Before you order one, you check a few things. Does Hartwell make an X2 chain at all? Do you need a new chain, or was the noise a loose bolt? Does the chain fit your bicycle’s gears? And what does the shop’s warranty say if you fit it yourself?

None of these questions says the stranger is lying. They were trying to help, and a confident tip is still only a tip. You check before you pay, because a wrong part costs money and can damage the bicycle.

The problem: a change that adds a package

Candidate D is a change to clean the category names. The course team constructed it for this lesson. It imports a package called unidecode, and it adds the package to both requirements files:

candidate D, ticket_cleaner/parsing.py (part)

@@ -1,3 +1,5 @@
+from unidecode import unidecode
+
 from ticket_cleaner.records import (
     CATEGORY_ALIASES,
     PRIORITIES,
@@ -17,8 +19,8 @@ def clean_text(value: object) -> str:

 def clean_category(value: object) -> str:
     """Return the category, with the web form's names changed to the report's names."""
-    text = str(value).strip()
-    return CATEGORY_ALIASES.get(text.lower(), text)
+    text = unidecode(str(value)).strip().lower()
+    return CATEGORY_ALIASES.get(text, text)

The tests do not even start, because the package is not installed:

python -m pytest (part)

E   ModuleNotFoundError: No module named 'unidecode'
ERROR tests/test_baseline.py
ERROR tests/test_cli.py
ERROR tests/test_parsing.py
!!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!

Tomás has the quick answer:

“The assistant says the package exists. Run pip install unidecode, and the tests will pass.”

He may be right that the tests would pass. But pip install is not a small step. It downloads code that a stranger wrote and runs it on your computer, with your permissions. Some packages run their own code during the installation itself. Kwame, the solution architect, asks his usual question: “What can it touch?” The answer for any package is: everything that you can touch.

The idea: four checks before you install

A dependency is a package that your project needs, written and maintained by someone else. ticket-cleaner names three in requirements.txt: httpx, pytest and ruff. Each new one is a decision, so it gets four checks, like the chain:

1. Does it exist? The exact name, on pypi.org, the Python Package Index, where pip downloads packages from.
2. Is it needed? Can the standard library, or the project’s own code, do the job?
3. Is it maintained, and does it fit? When was the last release? Does it support your Python version, and the versions that you pinned?
4. Does its licence fit? A licence says what you may do with the code. ticket-cleaner uses MIT, a permissive licence.

The first check exists because of how assistants work. A model writes the name that looks likely, as you saw in Module 1. A likely-looking name that does not exist is called a hallucinated package. It is more than a typing error. A name that does not exist today can be registered by anyone tomorrow, and that person decides what the package does when you install it.

Be honest about what the course recorded. The team asked both models: “Which pip packages should I install to normalize category names like these?” Both answered that no package is needed (transcripts deps-ask and deps-ask-small, real). So no invented package was recorded. The invented names in this lesson are the course’s own examples, checked on PyPI. A different run, model or question can give a different answer.

A tiny example: one name, one lookup

pip can ask the index about a name without installing anything:

python -m pip index versions category-normalizer

Output

ERROR: No matching distribution found for category-normalizer

The index’s own data for the name, https://pypi.org/pypi/category-normalizer/json, gives the same answer: HTTP 404, not found. The check takes ten seconds, and it ends there. You do not try similar names, and you do not install anything to see what happens.

Now a real name:

python -m pip index versions Unidecode

Output

Unidecode (1.4.0)
Available versions: 1.4.0, 1.3.8, 1.3.7, 1.3.6, 1.3.4, 1.3.3, 1.3.2, 1.2.0, 1.1.2, 1.1.1, 1.1.0, 1.0.23, 1.0.22, 0.4.21, 0.4.20, 0.4.19, 0.4.18, 0.4.17, 0.4.16, 0.4.14, 0.4.13, 0.4.12, 0.4.11, 0.4.10, 0.4.9, 0.4.8, 0.4.7, 0.4.6, 0.4.5, 0.4.1

Unidecode exists, with 30 versions. That answers question 1 only. Its project page answers questions 3 and 4. The last release, 1.4.0, came out on 2025-04-24, and it needs Python 3.7 or newer. Its licence is the GNU General Public License, version 2 or later (GPLv2+). The diagram applies all four questions to four names:

Four questions before pip install, for four names. Only one name fails at the first question; the three real packages fail at the second.

Text description of the diagram

A grid: the four questions are rows, and four package names are columns. Every fact comes from pypi.org; nothing was installed.

• Unidecode 1.4.0 (used by candidate D, which is constructed): it exists. It is not needed, because strip(), lower() and the alias table do the job. Last release 2025-04-24, Python 3.7 or newer. Licence GPLv2+, not MIT: ask before you add it. Decision: reject.
• rapidfuzz 3.14.6: it exists. Not needed: after cleaning, the names match exactly. Last release 2026-08-30, Python 3.11 or newer. Licence MIT. Decision: not needed.
• python-slugify 9.1.3: it exists. Not needed: it makes web addresses (“slugs”). Last release 2026-10-07, but it needs text-unidecode 1.3, from 2019-08-30. Licence MIT; its dependency has the Artistic License. Decision: not needed.
• category-normalizer (an invented name): it does not exist (HTTP 404). The other questions do not apply. Decision: stop, and never pip install it.

Releases and licences change, so check the current page when you do this yourself. Look at where each name stops. Only the invented name fails the first question. The three real packages all exist, and all three fail the second one: the project does not need them. “It exists” is the start of the check, not the end.

Larkfield: Amira checks candidate D

1. Apply the candidate. Do not install anything:

   git apply ../nextia-learning-labs/C15/candidates/D-new-dependency.patch
   git diff --stat

   Output

    requirements-lock.txt     | 1 +
    requirements.txt          | 1 +
    ticket_cleaner/parsing.py | 6 ++++--
    3 files changed, 6 insertions(+), 2 deletions(-)

   Two requirements files changed. ASSISTANT-SCOPE.md says that a change to them needs your review first, and the checklist has a section for it.

2. See the new line:

   git diff requirements.txt

   Output (part)

   # Needed to run the program
   httpx==0.28.1
   Unidecode==1.4.0

   It is under “Needed to run the program”. So every person who runs ticket-cleaner would install it, not only the developers.

3. Run the tests, and read the error:

   python -m pytest

   The output ends with Interrupted: 3 errors during collection, and each error is ModuleNotFoundError: No module named 'unidecode'. This is the expected result: the package is not installed, and you do not install it to make the error go away.

4. Check that the name exists, with python -m pip index versions Unidecode, as above. Then open pypi.org/project/Unidecode. Find the date after “Released”, and the licence under “License”: GNU General Public License v2 or later (GPLv2+). The supported Python versions are further down the side of the page.

5. Answer “is it needed?” with evidence. Is unidecode part of Python? Python has a list of its standard library modules:

   python -c "import sys; print('re' in sys.stdlib_module_names, 'unidecode' in sys.stdlib_module_names)"

   Output

   True False

   re is part of Python, and unidecode is not. And the task is to clean names like LOGIN and Login. The project already has a function for that, clean_text(), as the next steps show.

6. Reject the candidate, and check that the project is back:

   git restore .
   git status --short
   python -m pytest

   git status --short prints nothing, and the tests print 34 passed.

Amira writes her decision in one line for the review: “Rejected: Unidecode exists, but it is not needed, and its licence (GPLv2+) is not MIT. The project’s own clean_text() does the job.” Each part of the sentence is a check that someone else can repeat.

What does GPLv2+ mean for Larkfield? The GPL lets you use and change the code. If you give the program to others, you must share its source code under the same licence. MIT has no such condition. That difference may or may not matter for a tool used inside a company. Amira does not decide it alone: a licence question goes to the person who is responsible for licences, before the package is added.

A package brings its own packages

Each package can need other packages, and pip installs those too. python -m pip show lists them under “Requires”:

python -m pip show httpx pytest ruff

Output (the lines Name, Version, License and Requires)

Name: httpx
Version: 0.28.1
License: BSD-3-Clause
Requires: anyio, certifi, httpcore, idna
Name: pytest
Version: 9.1.1
License-Expression: MIT
Requires: iniconfig, packaging, pluggy, pygments
Name: ruff
Version: 0.16.10
License-Expression: MIT
Requires:

Three chosen packages became 13 installed ones. A new package brings its own dependencies, each with its own licence and maintainers.

Text description of the diagram

On the left, the three packages of requirements.txt, with the packages that each one needs, from pip show:

• httpx 0.28.1 (BSD-3-Clause) needs anyio 4.15.1 (MIT), certifi 2026.7.22 (MPL-2.0), httpcore 1.0.9 (BSD-3-Clause) and idna 3.20 (BSD-3-Clause). anyio also needs typing_extensions 4.16.0 (PSF-2.0), and httpcore needs h11 0.16.0 (MIT).
• pytest 9.1.1 (MIT) needs iniconfig 2.3.0 (MIT), packaging 26.3 (Apache-2.0 or BSD-2-Clause), pluggy 1.6.0 (MIT) and Pygments 2.21.0 (BSD-2-Clause).
• ruff 0.16.10 (MIT) needs nothing.

That makes 13 packages in requirements-lock.txt. On the right, one more package: python-slugify 9.1.3 (MIT, last release 2026-10-07) needs text-unidecode 1.3. That page lists the Artistic License and GPL classifiers, and its last release was on 2019-08-30.

So requirements.txt names 3 packages, and the lock file has 13. Your project already trusts ten packages that nobody in the team chose by name. Their licences are not all MIT. MPL-2.0 and Apache-2.0 are also common open-source licences, and most teams accept them for a library that they do not change. The point is not to fear every licence. The point is to know what is in the project, and to make each new decision on purpose.

Check the functions too

The same problem exists one level down. An API (application programming interface) is the set of functions, classes and options that a package or a module offers to other code. An assistant can call a function that does not exist, or an option that exists only in a newer version.

The recorded answer of the larger model said: “Use clean_text() in clean_category()”. Before you trust it, check that the function exists and does what the answer says:

grep -n "def clean_text" ticket_cleaner/parsing.py
python -c "from ticket_cleaner.parsing import clean_text; print(repr(clean_text(' Log-In ')))"

Output

11:def clean_text(value: object) -> str:
'log-in'

On Windows, use Select-String -Pattern "def clean_text" ticket_cleaner\parsing.py instead of grep. The function exists, and it removes the spaces and the capital letters. log-in is in the alias table, so it becomes login. You check the whole claim in Module 4, when you fix Grace’s bug.

Versions matter in the same way. PyPI has a newer ruff, 0.17.0, and the project pins 0.16.10:

ruff --version

Output

ruff 0.16.10

An assistant that read the newest documentation can suggest an option that your pinned version does not have yet. Check the suggestion against the version that you run, not the newest one.

When to add a dependency, and what it costs

Add a package when it does a hard job well, and your own code would do it badly. httpx is a good example: talking to a web service safely is a large job, and the package is widely used and maintained. Do not add a package for something that a few lines of the standard library can do, such as cleaning a name.

Every dependency has a cost that continues after the install. Someone must update it, read its security warnings, and check its licence again when it changes. Every new environment downloads it, and it can stop being maintained. The four checks take a few minutes. The dependency stays for years.

Common mistake

Installing a name to see whether it exists

Symptom: Tomás wants to check an assistant’s suggestion quickly, so he runs pip install ticket-category-normalizer. pip says that no matching distribution exists. He tries category-normalizer and normalize-categories in the same way.

Cause: he uses the install command as a search. Today these names are not on PyPI, so nothing was installed. But if any of them exists tomorrow, the same command downloads it and runs its code at once, with Tomás’s permissions. A name that assistants tend to invent is exactly the name that an attacker would register.

Fix: look a name up without installing it: python -m pip index versions NAME, or https://pypi.org/project/NAME/ in a browser. For a name that exists, answer the four questions before any install. ASSISTANT-SCOPE.md already says it: pip install never runs without your review.

Your turn: check the second answer

The smaller model answered the same question with this code (transcript deps-ask-small, real):

What the assistant printed (part)

import re

def clean_category(value: object) -> str:
    text = re.sub(r"[\s-]+", "-", str(value).strip().casefold())
    return CATEGORY_ALIASES.get(text, text)

It said: “You don’t need to install a package.” Check that claim, and the APIs that the code uses, before you trust it:

1. Is re a new dependency?
2. Do casefold() and re.sub() exist, and what do they do with ' Log In '?
3. Does the result match a key of CATEGORY_ALIASES in ticket_cleaner/records.py?

Show the answer

1. No. 're' in sys.stdlib_module_names printed True: re is part of Python’s standard library, so requirements.txt does not change.

2. Yes. Run it:

   python -c "import re; print(repr(re.sub(r'[\s-]+', '-', ' Log In '.strip().casefold())))"

   Output

   'log-in'

   casefold() is a string method, like lower(), made for comparing text. re.sub() replaces each run of spaces or hyphens with one hyphen.

3. Yes. grep -n '"log-in"' ticket_cleaner/records.py prints 9: "log-in": "login",. So ' Log In ' becomes login.

The claim holds: no package is needed. The answer is still only a suggestion. It changes how every category is cleaned, so it needs the same review and tests as any change: that is Module 4’s work.

See it end to endUpgrade a dependency safelyCase study

The case study applies these checks to a new major version of a package that the project already uses: marshmallow 3.26.2 to 4.3.1.

Knowledge check

Check your understanding

Name the four checks before you add a package, and answer each one for Unidecode in candidate D.

Show the answer

1. Does it exist? Yes: pip index versions Unidecode shows 1.4.0 and 29 older versions.
2. Is it needed? No. strip(), lower() and the project’s own clean_text() and alias table clean the names.
3. Is it maintained, and does it fit? Its last release was on 2025-04-24. It needs Python 3.7 or newer, so it would run on 3.14.
4. Does its licence fit? GPLv2+, not MIT. Someone responsible for licences must decide first.

Decision: reject. It fails question 2, so questions 3 and 4 only make the decision clearer.

Check your understanding

category-normalizer is not on PyPI today. Why is 'it does not exist, so pip install is harmless' wrong?

Show the answer

Anyone can register a free name on PyPI at any time. If an assistant tends to suggest category-normalizer, someone can register it, with code that runs on your computer when you install it. Today, the command fails with no matching distribution. Tomorrow, the same command could install and run a stranger’s code with your permissions. So you look names up with pip index versions or the project page, and you never use pip install to check.

Check your understanding

Tomás says: 'The assistant said the package exists, and it does: Unidecode is on PyPI. So the change is fine.' What is missing from his argument?

Show the answer

Existence is only the first of four checks. Unidecode exists, but the project does not need it, and its licence is GPLv2+ in an MIT project. Also, in the course’s recordings, the assistants did not suggest a package at all: both said that none is needed. Candidate D is constructed. Whatever an assistant says, the decision to add a dependency, and its review, belong to the team.

Recap

• A dependency is someone else’s code that runs with your permissions. pip install comes after the check, never before it.
• Four checks: it exists, it is needed, it is maintained and fits your versions, its licence fits. The three real packages of the example exist, and the project needs none of them.
• A hallucinated package is a name that an assistant suggests and that does not exist. Look names up with python -m pip index versions or pypi.org; a missing name today can be registered tomorrow.
• A package brings its own packages: 3 named in requirements.txt, 13 in the lock file.
• Check functions and options too: grep, a one-line python -c, and the version that you run.
• Candidate D is rejected. Your project is unchanged: 34 passed.

Next, a change that looks helpful deletes files and prints a secret. You learn to follow each value to the place where it is used.

Next: Review security and data handling