Automated endpoint tests

What you will be able to do

Test every path through the API in less than a second, including the one that the Postman collection cannot reach: the classifier is down. The test shows that the API answers with a controlled error that a client can act on, not a crash.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• Fixtures and parametrize, from Focused tests: Test the contract: normal, boundary and error cases.
• How the route gets its classifier, from Separate concerns: Dependencies: give the route its classifier.
• Exception handlers, from Validation and error responses: Exception handlers.

What you need on your computer

• pytest and httpx2 in the project’s environment, from Keep the documentation true: Install the test tools.
• ticket_api/export_contract.py, contract/openapi.json and tests/test_contract.py, from Keep the documentation true: Worked example: the reviewed contract. python -m pytest passes. If the contract test fails, read the difference and export again, as in Catch an accidental change.
• If your project is missing or broken, copy the snapshot end-of-m05, the project at the end of Module 5. Use a snapshot has the steps.

Tested with Python 3.14.6, FastAPI 0.142.2, pytest 9.1.1 and httpx2 2.13.1 on macOS.

TestClient: requests without a server

TestClient sends HTTP requests to your application object directly, in memory. There is no server, no port and no network, so a test takes about a millisecond, and the results are always the same. It has the same methods as httpx: client.get(...), client.post(..., json=...), and the response has .status_code, .json() and .headers.

from fastapi.testclient import TestClient

from ticket_api.main import app

client = TestClient(app)
response = client.get("/health")
assert response.status_code == 200

You used it in Module 4 for the contract tests. Now you test the behaviour of each endpoint, with the three kinds of case from the Python course: normal, boundary and error.

Worked example: the normal and the invalid cases

tests/test_classify.py

import pytest
from fastapi.testclient import TestClient

from ticket_api.classifier import ClassifierUnavailable
from ticket_api.main import app
from ticket_api.routes import get_classifier

client = TestClient(app)


def test_health():
    response = client.get("/health")
    assert response.status_code == 200
    assert response.json() == {"status": "ok"}


def test_billing_ticket():
    response = client.post(
        "/v1/classify",
        json={"subject": "Charged twice for March", "body": "Same invoice."},
    )
    assert response.status_code == 200
    assert response.json() == {
        "category": "billing",
        "priority": 1,
        "confidence": 0.9,
        "model_version": "keywords-1.0",
    }


@pytest.mark.parametrize("subject", ["x", "x" * 120])
def test_subject_length_limits_are_accepted(subject):
    assert client.post("/v1/classify", json={"subject": subject}).status_code == 200


@pytest.mark.parametrize(
    ("payload", "field"),
    [
        ({"subject": ""}, "subject"),
        ({"subject": "x" * 121}, "subject"),
        ({"body": "no subject"}, "subject"),
        ({"subject": "Hi", "colour": "red"}, "colour"),
    ],
)
def test_invalid_payload_names_the_field(payload, field):
    response = client.post("/v1/classify", json=payload)
    assert response.status_code == 422
    assert response.json()["error"]["code"] == "invalid_request"
    assert field in response.json()["error"]["fields"]


def test_malformed_json():
    response = client.post(
        "/v1/classify",
        content='{"subject": "x"',
        headers={"Content-Type": "application/json"},
    )
    assert response.status_code == 422
    assert response.json()["error"]["fields"] == ["body"]

Test	Kind	What it proves
test_billing_ticket	Normal	The whole body is exactly right, not only the category.
test_subject_length_limits_are_accepted	Boundary	1 and 120 characters are both accepted: the limits are not off by one.
test_invalid_payload_names_the_field	Error	Four invalid bodies, one test function. Each one gets 422 and names the right field.
test_malformed_json	Error	content= sends text that is not valid JSON, which json= cannot do.

parametrize with two names, ("payload", "field"), gives the test two values for each case. Each row of the list is one case, so it reads like the failure rows of your contract table.

The case that only a test can reach: a failing dependency

A real classifier can fail: a model server is down, or out of memory. What does the API answer then? You cannot test this with Postman, because your classifier never fails. With pytest, you can replace it.

FastAPI keeps a dictionary, app.dependency_overrides. If it has an entry for a dependency, FastAPI calls the replacement instead. The route asks for get_classifier, as in Separate concerns, so a test can give it a classifier that always fails.

First, a name for this kind of failure, in classifier.py:

ticket_api/classifier.py (add before Prediction)

class ClassifierUnavailable(Exception):
    """The classifier cannot answer now. The request may work later."""

Then the test. A fixture installs the override before the test, and removes it after the test, with yield in the middle. So the next test gets the real classifier again:

tests/test_classify.py (add at the end)

class BrokenClassifier:
    version = "broken"

    def predict(self, text: str):
        raise ClassifierUnavailable("the classifier is not available")


@pytest.fixture
def broken_classifier():
    app.dependency_overrides[get_classifier] = BrokenClassifier
    yield
    app.dependency_overrides.clear()


def test_unavailable_classifier_gives_503(broken_classifier):
    response = TestClient(app, raise_server_exceptions=False).post(
        "/v1/classify", json={"subject": "Refund"}
    )
    assert response.status_code == 503
    assert response.json()["error"]["code"] == "classifier_unavailable"
    assert "Traceback" not in response.text

raise_server_exceptions=False makes the test client behave like a real client. It receives the response that the server sends, instead of seeing the exception inside the test. Amira runs the tests:

Output (shortened)

>       assert response.status_code == 503
E       assert 500 == 503
FAILED tests/test_classify.py::test_unavailable_classifier_gives_503 - assert...
========================= 1 failed, 11 passed in 0.20s =========================

The test found a real gap: the API answers 500 internal_error. That tells the client “the server has a bug”, and it is wrong. The request is fine, and the same request can work later. The correct answer is 503, from Module 1. She adds a handler in errors.py, next to the others:

ticket_api/errors.py (changes)

from ticket_api.classifier import ClassifierUnavailable


    # in add_error_handlers, before the handler for Exception:
    @app.exception_handler(ClassifierUnavailable)
    async def classifier_unavailable(request: Request, exc: ClassifierUnavailable):
        logger.warning("classifier unavailable: %s", exc)
        return error_response(
            503,
            "classifier_unavailable",
            "The classifier is not available. Try again later.",
        )

The test passes. A 503 is part of the contract too, so she documents it on the route, next to the 422:

ticket_api/routes.py (in responses=)

        503: {
            "model": ErrorResponse,
            "description": "The classifier is not available.",
        },

The contract test fails, as it should: the contract changed. She reads the difference (a new 503 response). She decides that it is not breaking (a new documented response for a case that existed before), and runs python -m ticket_api.export_contract. Now everything passes:

Output

tests/test_classify.py ..........                                        [ 83%]
tests/test_contract.py ..                                                [100%]

============================== 12 passed in 0.12s ==============================

Guided practice: test every path

1. Make tests/test_classify.py with the normal and invalid cases. Run python -m pytest. They all pass.
2. Add ClassifierUnavailable to classifier.py, and the broken-classifier test. Run the tests. The new test fails with assert 500 == 503.
3. Add the handler to errors.py. Run the tests. All of them pass, but check that the contract test still passes too. (It does: the route does not document 503 yet.)
4. Document the 503 on the route. The contract test fails. Read its difference.
5. Run python -m ticket_api.export_contract, and run the tests again. 12 pass.

Your turn: two more cases

Add a test for each, and predict the result before you run it:

1. GET /v1/categories returns exactly the five categories, in order.
2. A ticket whose subject has an urgent word, but no category word: {"subject": "I cannot do anything"}. What category and priority do you expect?

Show the answers

def test_categories():
    response = client.get("/v1/categories")
    assert response.json() == {
        "categories": ["billing", "login", "shipping", "account", "other"]
    }


def test_urgent_word_without_category():
    response = client.post("/v1/classify", json={"subject": "I cannot do anything"})
    assert response.json()["category"] == "other"
    assert response.json()["priority"] == 3

The second one may surprise you: the classifier returns other with priority 3 as soon as no category matches, before it looks for urgent words. Is that the right behaviour? That is a product decision, and the test now records it. If the help desk wants urgent other tickets to get priority 1, the test is where you write that requirement first.

Common mistake

One test changes another test

Symptom: a test passes alone (python -m pytest -k billing), but fails when you run all of them, or the result depends on the order. For example, test_billing_ticket gets 503.

Cause: a test changed shared state and did not put it back. Here, app.dependency_overrides belongs to the one app object that every test uses. If a test sets an override and does not clear it, every later test gets the broken classifier.

Fix:

1. Change shared state only in a fixture, and undo it after yield, as broken_classifier does. The code after yield runs even when the test fails.
2. Never set an override in a test function body.
3. In the next lesson, each test gets its own new application, which removes most shared state.

Check your understanding

Why does the broken-classifier test replace the classifier with dependency_overrides, instead of stopping a real classifier service?

Show the answer

There is no classifier service to stop: the classifier is in the same program. And even with a real one, a test that depends on stopping a server is slow and unreliable. The override makes the failure happen on purpose, every time, in a millisecond, and the route cannot tell the difference. This works because the route gets its classifier through a dependency, not by making it itself.

Recap

• TestClient(app) sends requests to the application in memory: fast, and the same every time.
• Test normal cases, both sides of every limit, invalid bodies and malformed JSON. parametrize turns contract rows into test cases.
• app.dependency_overrides replaces a dependency in a test. Install it in a fixture, and clear it after yield.
• A dependency that is down gives 503 with its own code, not 500. Document it, and update the reviewed contract.

Next, you move the settings into one module, and add logs that connect each request to its response.

Next: Configuration and logging