Persistent data

What you will be able to do

Keep the data that matters when containers are replaced, and lose only what does not matter. You add a history of the classifications in PostgreSQL, and you run the API and its database together with one command.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• Containers on a user-defined network find each other by name, from Ports and service access: Container to container.
• Secret files and API_KEY_FILE, from Runtime configuration: Secret files.
• Liveness and readiness (/health and /ready), from Publish safely: Liveness and readiness.
• The reviewed contract and python -m ticket_api.export_contract, from Keep the documentation true: Worked example: the reviewed contract.
• requirements-run.txt, made in a container, from Dockerfile essentials: Make requirements-run.txt in a container.
• Optional: a SQL SELECT, from Select and filter: A query on six rows. The lesson explains the one query that it uses.

What you need on your computer

• The project with the new config.py and main.py, secrets/api_key.txt and the image ticket-api:1.1.0-dev, from Runtime configuration: Guided practice: settings and a secret file.
• The virtual environment .venv in the project, from Build and Deploy Your First API (Environment and first endpoint: Step 1: make the project), to run the tests.
• An internet connection: the lesson downloads one Python file and the PostgreSQL image (about 167 MB).
• If your project is missing or broken, copy the snapshot end-of-m02, as Use a snapshot says. Then do steps 1 to 6 of the guided practice of Runtime configuration.

Tested with Docker Desktop 4.81.0 (Docker Engine 29.6.1, Compose v5.2.0) on macOS, Apple silicon. The Windows and Linux steps are not tested.

Note

On Windows, in PowerShell, type curl.exe where the lesson says curl. Where other commands differ, a tab shows the PowerShell version.

Container files are disposable

Amira starts the API, writes a file in the container, and looks at it:

Terminal

docker run -d --name ticket-api ticket-api:1.1.0-dev
docker exec ticket-api sh -c "echo hello > /tmp/note.txt"
docker exec ticket-api cat /tmp/note.txt
docker diff ticket-api

The output (after the container ID):

hello
C /tmp
A /tmp/note.txt

docker diff lists what changed in the container’s writable layer, the thin layer on top of the image (Images and running containers): C is changed, A is added. Now she replaces the container with a new one from the same image, as every update does:

Terminal

docker rm -f ticket-api
docker run -d --name ticket-api ticket-api:1.1.0-dev
docker exec ticket-api cat /tmp/note.txt

The output (after the container name and the new ID):

cat: /tmp/note.txt: No such file or directory

The file was in the old container’s writable layer, and that layer was deleted with the container. A container is disposable: you replace it at every new version, every change of a setting and every restart on a new server. Anything that must survive must be stored outside it.

Terminal

docker rm -f ticket-api

Where data belongs

Storage	What it is	Survives a new container?	Use it for
The writable layer	Files that a container writes in its own file system	No	Temporary files
Named volume	Storage that Docker manages, with a name, outside every container	Yes, until you delete the volume	A database’s files on your computer
Bind mount	A file or folder of your computer, seen by the container	Yes: it is your file	A secret file, as in the last lesson
External database	A database server outside the app’s container, often a managed cloud service	Yes	Records, such as the history, in production
Object storage	A cloud service for files, such as Azure Blob Storage	Yes	Uploaded files, reports, trained model files

A volume is storage for containers that lives outside them. A container mounts it at a path, and the files at that path go into the volume. A named volume stays on one computer, though. A cloud service that starts your container on any of many servers cannot use it. In the cloud, records go into a managed database, and files go into object storage (Module 4).

This lesson adds a history to the API: one row in a database for each classification. The database is PostgreSQL, in its own container, with its files in a named volume.

The history keeps the result (category, priority, confidence, model version, time and request ID), not the ticket’s text. A ticket can contain names, addresses or order numbers. The history is for counting and for following up a request ID, and neither needs the text. Data that you do not store cannot leak.

Add the history

The code for the history is given: the course is about containers, not about Python. You add one module and connect it in four files.

Step 1: get history.py

Download the file into ticket_api:

macOS

curl -fsSL -o ticket_api/history.py https://raw.githubusercontent.com/Nextia-Intelligence/nextia-learning-labs/main/C18/files/history.py

Windows

curl.exe -fsSL -o ticket_api\history.py https://raw.githubusercontent.com/Nextia-Intelligence/nextia-learning-labs/main/C18/files/history.py

Linux

curl -fsSL -o ticket_api/history.py https://raw.githubusercontent.com/Nextia-Intelligence/nextia-learning-labs/main/C18/files/history.py

-f makes curl fail on an HTTP error instead of saving the error page. If the download does not work, copy the code below into ticket_api/history.py.

Show ticket_api/history.py

ticket_api/history.py

"""Keep a record of the classifications in PostgreSQL.

The record has the result of each classification, never the ticket's text.
History is optional: without DATABASE_URL the API classifies tickets and
keeps no record.
"""

import psycopg
from psycopg.rows import dict_row

CREATE_TABLE = """
CREATE TABLE IF NOT EXISTS classifications (
    id            bigserial PRIMARY KEY,
    request_id    text        NOT NULL,
    category      text        NOT NULL,
    priority      smallint    NOT NULL,
    confidence    real        NOT NULL,
    model_version text        NOT NULL,
    created_at    timestamptz NOT NULL DEFAULT now()
)
"""


class HistoryUnavailable(Exception):
    """History is off, or the database does not answer."""


class History:
    def __init__(self, url: str):
        self.url = url
        self.table_ready = False

    def connect(self) -> psycopg.Connection:
        try:
            return psycopg.connect(self.url, connect_timeout=3, row_factory=dict_row)
        except psycopg.OperationalError as error:
            raise HistoryUnavailable("the database does not answer") from error

    def setup(self) -> None:
        """Make the table if it does not exist."""
        with self.connect() as conn:
            conn.execute(CREATE_TABLE)
        self.table_ready = True

    def check(self) -> None:
        with self.connect() as conn:
            conn.execute("SELECT 1")

    def add(self, request_id: str, result: dict) -> None:
        if not self.table_ready:
            self.setup()
        with self.connect() as conn:
            conn.execute(
                "INSERT INTO classifications"
                " (request_id, category, priority, confidence, model_version)"
                " VALUES (%s, %s, %s, %s, %s)",
                (
                    request_id,
                    result["category"],
                    result["priority"],
                    result["confidence"],
                    result["model_version"],
                ),
            )

    def recent(self, limit: int) -> list[dict]:
        if not self.table_ready:
            self.setup()
        with self.connect() as conn:
            return conn.execute(
                "SELECT request_id, category, priority, confidence, model_version,"
                " created_at FROM classifications ORDER BY id DESC LIMIT %s",
                (limit,),
            ).fetchall()

What it does: it connects to PostgreSQL with psycopg, the PostgreSQL driver for Python, and waits at most 3 seconds. It makes the table classifications if it does not exist. add writes one row, recent reads the newest rows, and check is for /ready. Every connection problem becomes HistoryUnavailable, so the rest of the app needs to know only one error.

Step 2: the new package and the version

Add psycopg to requirements.txt, under the packages that the API needs to run:

requirements.txt

# Needed to run the API
fastapi[standard]==0.142.2
psycopg[binary]==3.3.6

In pyproject.toml, change the version and add the same package:

pyproject.toml

version = "1.0.0"
version = "1.1.0"
description = "Classifies support tickets into a category and a priority."
requires-python = ">=3.12"
dependencies = ["fastapi[standard]==0.142.2"]
dependencies = ["fastapi[standard]==0.142.2", "psycopg[binary]==3.3.6"]

[binary] installs a ready-made version of psycopg with the PostgreSQL client library inside, so the slim image needs no extra system packages.

Step 3: the models and the error

In ticket_api/models.py, add an import at the top, and two models after CategoryList:

ticket_api/models.py

from datetime import datetime
from typing import Literal

ticket_api/models.py (add after CategoryList)

class HistoryItem(BaseModel):
    """One classification that the API made. It has no ticket text."""

    request_id: str = Field(description="The request that made the classification.")
    category: Category
    priority: int = Field(ge=1, le=3)
    confidence: float = Field(ge=0, le=1)
    model_version: str
    created_at: datetime = Field(description="When the API made it (UTC).")


class HistoryList(BaseModel):
    items: list[HistoryItem] = Field(description="The newest classification first.")

In ticket_api/errors.py, import the error, and add a handler after the one for ClassifierUnavailable:

ticket_api/errors.py

from ticket_api.classifier import ClassifierUnavailable
from ticket_api.history import HistoryUnavailable
from ticket_api.models import ErrorDetail, ErrorResponse

ticket_api/errors.py (add after the ClassifierUnavailable handler)

    @app.exception_handler(HistoryUnavailable)
    async def history_unavailable(request: Request, exc: HistoryUnavailable):
        logger.warning("history unavailable: %s id=%s", exc, request.state.request_id)
        return error_response(
            request,
            503,
            "history_unavailable",
            "The history is not available. Try again later.",
        )

Step 4: save each result, and list them

In ticket_api/routes.py, three changes: the imports, the end of classify, and a new endpoint at the end of the file:

ticket_api/routes.py (the top)

import asyncio
from typing import Annotated, get_args

from fastapi import APIRouter, Depends, Request
import logging

from fastapi import APIRouter, Depends, Query, Request
from fastapi.concurrency import run_in_threadpool

from ticket_api.classifier import ClassifierUnavailable, KeywordClassifier
from ticket_api.history import HistoryUnavailable
from ticket_api.models import (
    Category,
    CategoryList,
    Classification,
    ErrorResponse,
    HistoryList,
    TicketIn,
)
from ticket_api.security import require_api_key

logger = logging.getLogger("ticket_api")

router = APIRouter(prefix="/v1", tags=["tickets"])

ticket_api/routes.py (the end of classify, and the new endpoint)

    return Classification(
    result = Classification(
        category=prediction.category,
        priority=prediction.priority,
        confidence=prediction.confidence,
        model_version=classifier.version,
    )
    history = request.app.state.history
    if history is not None:
        try:
            await run_in_threadpool(
                history.add, request.state.request_id, result.model_dump()
            )
        except HistoryUnavailable:
            logger.warning("history not saved id=%s", request.state.request_id)
    return result


@router.get(
    "/history",
    tags=["history"],
    summary="List recent classifications",
    description=(
        "Returns the newest classifications first. History is off when the "
        "server has no DATABASE_URL."
    ),
    responses={
        401: {
            "model": ErrorResponse,
            "description": "The API key is missing or wrong.",
        },
        503: {
            "model": ErrorResponse,
            "description": "History is off, or the database does not answer.",
        },
    },
    dependencies=[Depends(require_api_key)],
)
async def list_history(
    request: Request,
    limit: Annotated[int, Query(ge=1, le=100, description="How many to return.")] = 10,
) -> HistoryList:
    history = request.app.state.history
    if history is None:
        raise HistoryUnavailable("history is off: DATABASE_URL is not set")
    items = await run_in_threadpool(history.recent, limit)
    return HistoryList(items=items)

Look at the try in classify. If the database does not answer, the classification still returns 200, and the log gets a warning. This is a design choice. Classifying is the API’s main job. The history is a side record, so its failure must not stop the main job. But the failure must be visible: in the log, and in /ready (next step). /v1/history needs the key, like /v1/classify.

Step 5: connect it in main.py

ticket_api/main.py

from ticket_api.config import Settings, SettingsError, load_settings
from ticket_api.errors import add_error_handlers, error_response
from ticket_api.history import History, HistoryUnavailable
from ticket_api.models import ErrorResponse, Health, Readiness

ticket_api/main.py (in create_app)

app = FastAPI(
    title="Ticket Classifier API",
    version="1.0.0",
    version="1.1.0",
    description=DESCRIPTION,
    openapi_tags=[
        {"name": "tickets", "description": "Classify support tickets."},
        {"name": "history", "description": "See recent classifications."},
        {"name": "health", "description": "Check that the service runs."},
    ],

ticket_api/main.py (after make_classifier)

app.state.classifier = make_classifier(settings.classifier_mode)
app.state.history = None
if settings.database_url:
    app.state.history = History(settings.database_url)
    try:
        app.state.history.setup()
    except HistoryUnavailable:
        logger.warning("the database does not answer yet: history is not ready")
else:
    logger.info("DATABASE_URL is not set: history is off")

ticket_api/main.py (in ready)

def ready() -> Readiness:
    app.state.classifier.predict("readiness check")
    if app.state.history is not None:
        app.state.history.check()
    return Readiness(status="ready")

Without DATABASE_URL, the API works as before, with no history. If the database is not ready when the API starts, the API still starts, and makes the table later, at the first request.

Step 6: the tests

Add the file tests/test_history.py. It has 8 tests: 4 for the history and 4 for the settings of the last lesson. FakeHistory keeps the rows in a Python list, so the tests need no database.

Show tests/test_history.py

tests/test_history.py

from datetime import datetime, timezone

import pytest

from ticket_api.config import SettingsError, load_settings
from ticket_api.history import HistoryUnavailable


class FakeHistory:
    """Keeps the history in a list, so the tests need no database."""

    def __init__(self, down=False):
        self.rows = []
        self.down = down

    def check(self):
        if self.down:
            raise HistoryUnavailable("down")

    def add(self, request_id, result):
        self.check()
        created_at = datetime.now(timezone.utc)
        self.rows.append({"request_id": request_id, **result, "created_at": created_at})

    def recent(self, limit):
        self.check()
        return list(reversed(self.rows))[:limit]


@pytest.fixture
def history_client(make_client):
    def make(down=False):
        client = make_client()
        client.app.state.history = FakeHistory(down)
        return client

    return make


def test_history_is_off_without_database_url(client):
    response = client.get("/v1/history")
    assert response.status_code == 503
    assert response.json()["error"]["code"] == "history_unavailable"


def test_classification_is_recorded_without_the_text(history_client):
    client = history_client()
    response = client.post("/v1/classify", json={"subject": "Parcel lost"})
    items = client.get("/v1/history").json()["items"]
    assert len(items) == 1
    assert items[0]["category"] == "shipping"
    assert items[0]["request_id"] == response.headers["X-Request-ID"]
    assert "subject" not in items[0]


def test_history_limit(history_client):
    client = history_client()
    for _ in range(3):
        client.post("/v1/classify", json={"subject": "Refund"})
    assert len(client.get("/v1/history?limit=2").json()["items"]) == 2
    assert client.get("/v1/history?limit=0").status_code == 422


def test_database_down_still_classifies_but_is_not_ready(history_client):
    client = history_client(down=True)
    assert client.post("/v1/classify", json={"subject": "Refund"}).status_code == 200
    assert client.get("/ready").status_code == 503
    assert client.get("/v1/history").status_code == 503


def test_api_key_from_a_file(tmp_path, monkeypatch):
    key_file = tmp_path / "api_key"
    key_file.write_text("from-a-file\n")
    monkeypatch.setenv("API_KEY_FILE", str(key_file))
    assert load_settings().api_key == "from-a-file"


def test_missing_secret_file_stops_the_app(monkeypatch):
    monkeypatch.setenv("API_KEY_FILE", "/no/such/file")
    with pytest.raises(SettingsError):
        load_settings()


def test_required_api_key(monkeypatch):
    monkeypatch.delenv("API_KEY", raising=False)
    monkeypatch.delenv("API_KEY_FILE", raising=False)
    monkeypatch.setenv("REQUIRE_API_KEY", "true")
    with pytest.raises(SettingsError):
        load_settings()


def test_secrets_are_not_in_the_settings_repr(monkeypatch):
    monkeypatch.setenv("API_KEY", "do-not-print")
    monkeypatch.setenv("DATABASE_URL", "postgresql://u:do-not-print@db/x")
    assert "do-not-print" not in repr(load_settings())

In the virtual environment, Amira installs the new package, makes the lock file again, and runs the tests:

Terminal

python -m pip install -r requirements.txt
python -m pip freeze > requirements-lock.txt
python -m pytest

The output (the end):

FAILED tests/test_contract.py::test_openapi_matches_the_reviewed_contract - A...
1 failed, 24 passed in 0.24s

Predict: why does the contract test fail? The API has a new endpoint and a new version, so the OpenAPI document changed. The change is intended, so she exports the contract again, as in the API course, and runs the tests again:

Terminal

python -m ticket_api.export_contract
python -m pytest

The output (the end):

Wrote contract/openapi.json
...
25 passed in 0.30s

Step 7: the packages of the image

The image installs from requirements-run.txt, so that file must have psycopg too. Make it again in a container, as in Dockerfile essentials: Make requirements-run.txt in a container, now with psycopg in the command:

Terminal

docker run --rm python:3.14.8-slim-trixie sh -c 'pip install -q --root-user-action=ignore --disable-pip-version-check "fastapi[standard-no-fastapi-cloud-cli]==0.142.2" "psycopg[binary]==3.3.6" && pip freeze' > requirements-run.txt

Check the result: wc -l requirements-run.txt and grep psycopg requirements-run.txt (PowerShell: (Get-Content requirements-run.txt).Count and Select-String psycopg requirements-run.txt). The output:

      52 requirements-run.txt
psycopg==3.3.6
psycopg-binary==3.3.6

52 lines: the 50 of Module 2, and the two psycopg packages.

Write compose.yaml

Now the project needs two containers that work together: the API and PostgreSQL. You could start them with two docker run commands, a network, a volume and many options. Docker Compose does all of that from one file. Compose is a part of Docker that starts a group of containers from a file called compose.yaml. Each container in the file is a service.

First, the database password. It goes into .env, which stays out of Git. A template that has no real secret goes into Git, so that others know which values to set. Make .env.example:

.env.example

# Copy this file to .env and change the value. Never commit .env.
POSTGRES_PASSWORD=change-me-local-only

Then make .env from it. This replaces the .env of the last lesson: the API’s settings are now in compose.yaml.

macOS

cp .env.example .env

Windows

Copy-Item .env.example .env

Linux

cp .env.example .env

Open .env and change the password to a new one. The password protects only the database on your computer, but use one that you do not use anywhere else.

Then make compose.yaml in the project folder:

compose.yaml

# The ticket API and its database, on your computer.
# Start:  docker compose up -d --build
# Stop:   docker compose down          (keeps the data)
# Reset:  docker compose down --volumes (deletes the data)

services:
  api:
    build: .
    image: ticket-api:1.1.0
    ports:
      - "127.0.0.1:8000:8000"
    environment:
      API_KEY_FILE: /run/secrets/api_key
      DATABASE_URL: postgresql://tickets:${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}@db:5432/tickets
      REQUIRE_API_KEY: "true"
    secrets:
      - api_key
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres:18.6-trixie
    environment:
      POSTGRES_USER: tickets
      POSTGRES_DB: tickets
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}
    volumes:
      - pgdata:/var/lib/postgresql
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "tickets", "-d", "tickets"]
      interval: 2s
      timeout: 3s
      retries: 15

secrets:
  api_key:
    file: ./secrets/api_key.txt

volumes:
  pgdata:

Line by line:

Lines	Meaning
build: . and image: ticket-api:1.1.0	Build the image from the Dockerfile in this folder, and tag it ticket-api:1.1.0.
ports: "127.0.0.1:8000:8000"	The same as -p 127.0.0.1:8000:8000: only this computer can connect. The quotes stop YAML from reading the value as a number.
environment:	The API’s settings, as with -e. REQUIRE_API_KEY: "true": here the key is required.
DATABASE_URL: ...@db:5432/tickets	The database’s address. The host name is db, the service’s name. Compose puts both services on one network, where each service has its name, as in Ports and service access.
${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}	Compose replaces this with the value from .env. :? means required: without the value, Compose stops with that message.
secrets: - api_key	Mount the Compose secret api_key in the container, read-only, at /run/secrets/api_key. API_KEY_FILE points there.
depends_on: db: condition: service_healthy	Start the API only after the database’s health check passes.
image: postgres:18.6-trixie	The official PostgreSQL image, PostgreSQL 18.6 on Debian 13, pinned to an exact tag.
POSTGRES_USER, POSTGRES_DB, POSTGRES_PASSWORD	The image makes this user and this database the first time that it starts with an empty volume.
volumes: - pgdata:/var/lib/postgresql	Mount the named volume pgdata at /var/lib/postgresql, where PostgreSQL 18 keeps its files. See The PostgreSQL 18 mount path.
healthcheck:	A health check is a command that Docker runs in the container again and again, to see if the service works. pg_isready asks PostgreSQL if it accepts connections: every 2 s, at most 15 times.
secrets: api_key: file:	The Compose secret comes from ./secrets/api_key.txt, the file of the last lesson.
volumes: pgdata:	Declare the named volume. Compose makes it the first time.

The .env file is for Compose, not for the containers. Compose reads it to fill in ${...} in compose.yaml. The containers get only what environment: lists, so the API container does not get POSTGRES_PASSWORD, only the URL that contains it.

Predict: what happens if .env is missing? Amira tried it before she made the file:

error while interpolating services.api.environment.DATABASE_URL: required variable POSTGRES_PASSWORD is missing a value: Set POSTGRES_PASSWORD in .env

Compose stopped before it made anything.

Start, check and stop

The API and its database in Compose: one network, one published port on 127.0.0.1, and two kinds of storage outside the containers.

Text description of the diagram

Your computer is the host. curl or a browser on it calls http://127.0.0.1:8000, the published port. Docker forwards it to port 8000 of the api container. A phone on the Wi-Fi cannot connect, because the port is published only on 127.0.0.1. The api container (ticket-api:1.1.0, listening on 0.0.0.0:8000) and the db container (postgres:18.6-trixie) are on the Compose network ticket-api_default. The API calls the database at db:5432. Port 5432 is not published, so only containers on this network can reach it. Two things are outside the containers: the host file secrets/api_key.txt, mounted read-only in api at /run/secrets/api_key, and the named volume ticket-api_pgdata, mounted in db at /var/lib/postgresql. Containers are replaced. The volume and the file on the host stay.

Amira starts both services. --build builds the API’s image first:

Terminal

docker compose up -d --build

The output (the build lines shortened):

 Image ticket-api:1.1.0 Building
...
 Image ticket-api:1.1.0 Built
 Network ticket-api_default Creating
 Network ticket-api_default Created
 Volume ticket-api_pgdata Creating
 Volume ticket-api_pgdata Created
 Container ticket-api-db-1 Creating
 Container ticket-api-db-1 Created
 Container ticket-api-api-1 Creating
 Container ticket-api-api-1 Created
 Container ticket-api-db-1 Starting
 Container ticket-api-db-1 Started
 Container ticket-api-db-1 Waiting
 Container ticket-api-db-1 Healthy
 Container ticket-api-api-1 Starting
 Container ticket-api-api-1 Started

The first time, Compose also downloads postgres:18.6-trixie. The names come from the project name, which is the folder’s name: ticket-api. So the network is ticket-api_default, the volume is ticket-api_pgdata, and the containers are ticket-api-db-1 and ticket-api-api-1. Look at the order: the database first, then Waiting until it is Healthy, then the API.

depends_on with condition: service_healthy makes the API wait for a database that answers, not only for a container that runs.

Text description of the diagram

You run docker compose up -d --build. Compose builds ticket-api:1.1.0 and makes the network and the volume. It starts db with the volume at /var/lib/postgresql. Then, every 2 seconds and at most 15 times, it runs the health check pg_isready in db, which answers “not ready” and then “ready”. When db is healthy, Compose starts api, with the secret at /run/secrets/api_key. The API connects to db:5432 and makes the table. Compose prints that ticket-api-api-1 started. Your curl 127.0.0.1:8000/ready gets {"status":"ready"}.

Terminal

docker compose ps

The output:

NAME               IMAGE                  COMMAND                  SERVICE   CREATED         STATUS                   PORTS
ticket-api-api-1   ticket-api:1.1.0       "fastapi run --host …"   api       3 seconds ago   Up Less than a second    127.0.0.1:8000->8000/tcp
ticket-api-db-1    postgres:18.6-trixie   "docker-entrypoint.s…"   db        3 seconds ago   Up 2 seconds (healthy)   5432/tcp

The database’s port 5432 is not published: only the API, on the same network, can reach it. docker compose logs api shows the API’s log, with the same start lines as before. She checks readiness and sends two tickets, then reads the history:

Terminal

curl -s http://127.0.0.1:8000/ready
curl -s -X POST http://127.0.0.1:8000/v1/classify -H "Content-Type: application/json" -H "X-API-Key: local-dev-key" -d @requests/billing.json
curl -s -X POST http://127.0.0.1:8000/v1/classify -H "Content-Type: application/json" -H "X-API-Key: local-dev-key" -d '{"subject":"Parcel lost"}'
curl -s http://127.0.0.1:8000/v1/history -H "X-API-Key: local-dev-key"

The output (one line per command; the last one is long):

{"status":"ready"}
{"category":"billing","priority":1,"confidence":0.9,"model_version":"keywords-1.0"}
{"category":"shipping","priority":1,"confidence":0.7,"model_version":"keywords-1.0"}
{"items":[{"request_id":"1fe5dfeafae4","category":"shipping","priority":1,"confidence":0.7,"model_version":"keywords-1.0","created_at":"2026-10-07T16:58:28.691060Z"},{"request_id":"d2ef963e6b97","category":"billing","priority":1,"confidence":0.9,"model_version":"keywords-1.0","created_at":"2026-10-07T16:58:28.663288Z"}]}

The newest first, with the time in UTC, and no ticket text. PowerShell can change the quotes in -d '{"subject":"Parcel lost"}' before curl.exe gets them. If you get 422, save the JSON in a file outside the project, for example $HOME\parcel.json, and use -d "@$HOME\parcel.json". (Not tested on Windows.)

To look into the database, she runs psql, PostgreSQL’s command-line client, in the db container. docker compose exec is docker exec with the service’s name:

Terminal

docker compose exec db psql -U tickets -d tickets -c "SELECT id, request_id, category, priority, created_at FROM classifications;"

The output:

 id |  request_id  | category | priority |          created_at
----+--------------+----------+----------+-------------------------------
  1 | d2ef963e6b97 | billing  |        1 | 2026-10-07 16:58:28.663288+00
  2 | 1fe5dfeafae4 | shipping |        1 | 2026-10-07 16:58:28.69106+00
(2 rows)

The same two rows, in the order they were written. The SELECT asks for five columns of every row in the table classifications.

Last, what Docker keeps about the API container:

Terminal

docker inspect ticket-api-api-1 --format "{{json .Config.Env}}"

The output (shortened):

["REQUIRE_API_KEY=true","API_KEY_FILE=/run/secrets/api_key","DATABASE_URL=postgresql://tickets:change-me-local-only@db:5432/tickets",...]

The key is a file, as planned. But the database password is in DATABASE_URL, in plain text. Here it is a local password that protects only a database without a published port. In the cloud, the password would be a platform secret, or the app would use an identity instead of a password (Module 4). The API can also read the URL from a file with DATABASE_URL_FILE, in the same way as API_KEY_FILE.

To stop everything:

Terminal

docker compose down
docker volume ls --filter name=ticket-api

The output:

 Container ticket-api-api-1 Stopping
 Container ticket-api-api-1 Stopped
 Container ticket-api-api-1 Removing
 Container ticket-api-api-1 Removed
 Container ticket-api-db-1 Stopping
 Container ticket-api-db-1 Stopped
 Container ticket-api-db-1 Removing
 Container ticket-api-db-1 Removed
 Network ticket-api_default Removing
 Network ticket-api_default Removed
DRIVER    VOLUME NAME
local     ticket-api_pgdata

down removes the containers and the network, but not the volume. The next docker compose up -d makes new containers, and the history is still there.

The PostgreSQL 18 mount path

Many older tutorials mount PostgreSQL’s volume at /var/lib/postgresql/data. Since PostgreSQL 18, the official image keeps its files in /var/lib/postgresql/18/docker, a folder for each major version. A later upgrade can then keep the old and the new files side by side. The image declares the folder above them, /var/lib/postgresql, as its volume:

Terminal

docker image inspect postgres:18.6-trixie --format "{{json .Config.Volumes}}"
docker compose exec db ls /var/lib/postgresql/18

The output:

{"/var/lib/postgresql":{}}
docker

Predict: what happens with the old path, pgdata:/var/lib/postgresql/data? Amira tried it in a copy of compose.yaml, with a new, empty volume:

 Container ticket-api-db-1 Waiting
 Container ticket-api-db-1 Error dependency db failed to start
dependency failed to start: container ticket-api-db-1 exited (1)

The database container stopped, and the API was never started (docker compose ps -a shows it as Created). The database’s log:

db-1  | Error: in 18+, these Docker images are configured to store database data in a
db-1  |        format which is compatible with "pg_ctlcluster" (specifically, using
db-1  |        major-version-specific directory names).  This better reflects how
db-1  |        PostgreSQL itself works, and how upgrades are to be performed.
...
db-1  |        Counter to that, there appears to be PostgreSQL data in:
db-1  |          /var/lib/postgresql/data (unused mount/volume)
...
db-1  |        The suggested container configuration for 18+ is to place a single mount
db-1  |        at /var/lib/postgresql which will then place PostgreSQL data in a
db-1  |        subdirectory, allowing usage of "pg_upgrade --link" without mount point
db-1  |        boundary issues.

The message talks about old data, but the volume was empty: the image refuses any mount at the old path. Mount the volume at /var/lib/postgresql, as in this course’s compose.yaml. The image fails loudly, which is good: a database that started and wrote its files outside the volume would lose them without a message.

Guided practice: the API and its database

1. Do steps 1 to 7 of Add the history. Expect 25 passed and 52 lines in requirements-run.txt.
2. Make .env.example, .env and compose.yaml as in Write compose.yaml. Change the password in .env.
3. Run git status. Check that .env and secrets/ are not in the list, and that .env.example and compose.yaml are.
4. Start the services: docker compose up -d --build.
5. Run docker compose ps. Wait until db shows (healthy) and api shows Up.
6. Call /ready. Expect {"status":"ready"}. If curl shows (52) Empty reply from server, the API is still starting: wait a few seconds.
7. Send two tickets to /v1/classify with the key, then call /v1/history with the key. Expect two items, the newest first.
8. Run the psql query in the db service. Expect the same two rows.
9. Replace the project’s README.md with the version below.
10. Leave the services running for the module practice.

Show README.md

README.md

# ticket-api

A small web API that classifies support tickets. It runs in a container, alone or with a PostgreSQL database that keeps a history of the classifications.

## Run it in a container

You need Docker. The API listens on port 8000 in the container.

```sh
docker build -t ticket-api:1.1.0 .
docker run --rm -p 127.0.0.1:8000:8000 -e API_KEY=local-dev-key ticket-api:1.1.0
```

Check it: `curl http://127.0.0.1:8000/health` gives `{"status":"ok"}`.

## Run it with the database

```sh
cp .env.example .env                         # then change the password in .env
mkdir -p secrets
printf 'local-dev-key' > secrets/api_key.txt
docker compose up -d --build
```

- `docker compose down` stops and removes the containers. The data stays in the volume `ticket-api_pgdata`.
- `docker compose down --volumes` also deletes the data.

## Settings

| Setting | Required | Default | What it does |
|---|---|---|---|
| `API_KEY` or `API_KEY_FILE` | When `REQUIRE_API_KEY` is `true` | none | The key that clients send in `X-API-Key`. `API_KEY_FILE` is the path of a file that holds it. |
| `REQUIRE_API_KEY` | No | `false` | `true`: the API does not start without a key. |
| `DATABASE_URL` or `DATABASE_URL_FILE` | No | none | A PostgreSQL URL. Without it, history is off. |
| `ALLOWED_ORIGINS` | No | none | Sites that may call the API from a browser, separated by commas. |
| `SHOW_DOCS` | No | `true` | `false` hides `/docs`, `/redoc` and `/openapi.json`. |
| `CLASSIFIER_MODE`, `CLASSIFIER_TIMEOUT`, `LOG_LEVEL` | No | `keywords`, `2.0`, `INFO` | As in the API course. |

Never put a key or a password in the Dockerfile, in `compose.yaml` or in Git. `.env` and `secrets/` are in `.gitignore` and `.dockerignore`.

## Work on the code

```sh
python3 -m venv .venv                # Windows: python -m venv .venv
source .venv/bin/activate            # Windows: .venv\Scripts\Activate.ps1
python -m pip install -r requirements-lock.txt
python -m pytest
```

`requirements-run.txt` is the exact list of packages in the image. Make it again inside a container, so that it is the same on every computer:

```sh
docker run --rm python:3.14.8-slim-trixie sh -c 'pip install -q --root-user-action=ignore --disable-pip-version-check "fastapi[standard-no-fastapi-cloud-cli]==0.142.2" "psycopg[binary]==3.3.6" && pip freeze' > requirements-run.txt
```

Your turn: the database stops

Change one condition: stop only the database, with docker compose stop db. Before you test, predict the status code of /health, /ready and POST /v1/classify (with the key). Then test, read the last lines of docker compose logs api, and start the database again with docker compose start db.

Show the answer

{"status":"ok"}
200
{"error":{"code":"history_unavailable","message":"The history is not available. Try again later.","request_id":"57da74011ff3","fields":[]}}
503
{"category":"billing","priority":1,"confidence":0.7,"model_version":"keywords-1.0"}
200

These are /health, /ready and a classification of {"subject":"Charged twice for March"}, each with -w "\n%{http_code}\n". The process runs (/health 200). It is not ready, because the history does not answer (/ready 503). It still classifies (200). The log shows the side effect:

WARNING ticket_api: history unavailable: the database does not answer id=57da74011ff3
INFO ticket_api: GET /ready 503 18ms id=57da74011ff3
...
WARNING ticket_api: history not saved id=5ecc59a5fbbf
INFO ticket_api: POST /v1/classify 200 8ms id=5ecc59a5fbbf

After docker compose start db, /ready gives {"status":"ready"} again. The classification that was made during the outage is not in the history: the API does not keep it for later. This is why /ready must show the problem. In Module 5, the cloud platform reads /ready and sends no traffic to a copy that is not ready, but it does not restart it.

Common mistake

The history is empty after docker compose down and up

Symptom: the history worked, and it even survived a restart of the database container. After docker compose down and docker compose up -d, /v1/history gives {"items":[]}. docker volume ls shows a volume with a long name of numbers and letters, and no ticket-api_pgdata.

Cause: the data was never in a named volume. In this test, Amira removed the two volumes: parts from compose.yaml. The PostgreSQL image declares /var/lib/postgresql as a volume, so Docker made an anonymous volume: a volume without a name, for that one container. docker compose up -d --force-recreate db reuses it, so the data seemed safe. docker compose down removes the container, and the next up makes a new container with a new anonymous volume. The old one stays on your computer, unused. With no volume at all, the data goes even earlier. Files in a container’s writable layer go with the container.

Fix:

1. Give every piece of data that must survive a named volume, in both places: - pgdata:/var/lib/postgresql under the service, and pgdata: under volumes: at the end of the file.
2. Check after docker compose up -d: docker volume ls --filter name=ticket-api must show ticket-api_pgdata.
3. Find the old anonymous volume with docker inspect <container> --format "{{json .Mounts}}" before you remove the container. Remove it with docker volume rm <its name>. Remove only volumes that you know are yours.

For a real AI service: the same mistake loses more than a history. Think of a service that writes feedback, labels or a vector index into its container’s files. It works perfectly in a demonstration, and loses everything at the first update. Decide for each file: disposable, a volume on one computer, or a database or object storage that outlives every container.

Check your understanding

Which commands remove the containers, and which command also deletes the history?

Show the answer

docker compose down stops and removes the containers and the network. The named volume ticket-api_pgdata stays, so the history survives. docker compose down --volumes also removes the volume, and with it the history. docker compose up -d --force-recreate api replaces only the API’s container. Neither the volume nor the database is touched.

Check your understanding

Why does DATABASE_URL use the host name db, and not localhost or 127.0.0.1?

Show the answer

The URL is used inside the API container. There, localhost and 127.0.0.1 mean the API container itself, which has no database. Compose puts both services on the network ticket-api_default, where each service can be found by its name, so db:5432 reaches the database container. Port 5432 is not published, so programs outside the network cannot reach the database.

Module practice

Worked example. Amira has the API and its database running from the guided practice. She proves which data survives when the API’s container is replaced:

Terminal

docker compose exec api sh -c "echo hello > /tmp/note.txt"
docker compose exec api cat /tmp/note.txt
docker compose up -d --force-recreate api
docker compose exec api cat /tmp/note.txt
curl -s http://127.0.0.1:8000/v1/history -H "X-API-Key: local-dev-key"

The output (the history shortened):

hello
 Container ticket-api-db-1 Running
 Container ticket-api-api-1 Recreate
 Container ticket-api-api-1 Recreated
 Container ticket-api-db-1 Waiting
 Container ticket-api-db-1 Healthy
 Container ticket-api-api-1 Starting
 Container ticket-api-api-1 Started
cat: /tmp/note.txt: No such file or directory
{"items":[{"request_id":"1fe5dfeafae4","category":"shipping",...},{"request_id":"d2ef963e6b97","category":"billing",...}]}

--force-recreate api makes a new API container even though nothing changed. The file in the old container is gone. The two classifications are still there, because they are in the database, in the volume. She does the same with docker compose down, then docker compose up -d: still two items. Then, on purpose, she deletes the data:

Terminal

docker compose down --volumes
docker compose up -d
curl -s http://127.0.0.1:8000/v1/history -H "X-API-Key: local-dev-key"

The output (the last lines of down, then the new history):

 Volume ticket-api_pgdata Removing
 Network ticket-api_default Removing
 Volume ticket-api_pgdata Removed
 Network ticket-api_default Removed
...
{"items":[]}

Your task. Run the API with its dependency, and recreate both containers. Then show with evidence which data survived:

1. Start the services, and send three tickets with different subjects.
2. Write a file in the API container.
3. Recreate both containers with docker compose up -d --force-recreate api db.
4. Show that the file is gone, that the history still has three items, and that the database has three rows (SELECT count(*) FROM classifications;).
5. Write two sentences: which state is disposable, which state survives, and why.

Show a model answer

docker compose up -d
curl -s -X POST http://127.0.0.1:8000/v1/classify -H "Content-Type: application/json" -H "X-API-Key: local-dev-key" -d '{"subject":"Refund for order 1182"}'
curl -s -X POST http://127.0.0.1:8000/v1/classify -H "Content-Type: application/json" -H "X-API-Key: local-dev-key" -d '{"subject":"Cannot log in"}'
curl -s -X POST http://127.0.0.1:8000/v1/classify -H "Content-Type: application/json" -H "X-API-Key: local-dev-key" -d '{"subject":"Parcel lost"}'
docker compose exec api sh -c "echo hello > /tmp/note.txt"
docker compose up -d --force-recreate api db
docker compose exec api cat /tmp/note.txt
curl -s "http://127.0.0.1:8000/v1/history?limit=5" -H "X-API-Key: local-dev-key"
docker compose exec db psql -U tickets -d tickets -c "SELECT count(*) FROM classifications;"

The results in the test run: the three tickets gave billing (priority 2), login and shipping. Both containers were recreated (Recreate, Recreated, then Healthy for db). cat: /tmp/note.txt: No such file or directory. The history had three items, shipping first. The count was:

 count
-------
     3
(1 row)

The file was in the API container’s writable layer, which is deleted with the container: disposable. The classifications were in PostgreSQL’s files, in the named volume ticket-api_pgdata, which is outside every container: they survive until docker compose down --volumes.

Module check

Check your understanding

Name the -p option that lets only your own computer reach port 8000 of a container, and the setting that makes the ticket API refuse to start without a key.

Show the answer

-p 127.0.0.1:8000:8000 (in Compose: "127.0.0.1:8000:8000" under ports:). REQUIRE_API_KEY=true: with it, a missing key gives Settings error: REQUIRE_API_KEY is true, but API_KEY is not set and exit code 1.

Check your understanding

You run docker compose down, then docker compose up -d. Predict what is still there: the file /tmp/note.txt in the API container, the history, the network, the containers.

Show the answer

The history is still there: it is in the named volume ticket-api_pgdata, which down does not remove. The file is gone, because it was in the old container’s writable layer. The network and the containers are new: down removed them, and up made them again with the same names. Only down --volumes deletes the history.

Check your understanding

Completion check: confirm, with commands and their results, that the required data survives while the disposable container state does not.

Show the answer

Evidence for a reviewer, from the module practice:

1. Before: /v1/history (with the key) lists the classifications, and docker compose exec api cat /tmp/note.txt prints hello.
2. docker compose up -d --force-recreate api db replaces both containers.
3. After: cat /tmp/note.txt gives No such file or directory (disposable state is gone). /v1/history has the same items, and SELECT count(*) gives the same count (required data survived).
4. docker volume ls --filter name=ticket-api shows ticket-api_pgdata: the place where the data survives.
5. The explanation: the writable layer belongs to one container, and the volume belongs to no container.

Demonstrated: all five, with real output. Partially demonstrated: the commands without the before-and-after outputs, or no explanation.

Recap

In this module, you:

• published the API’s port on 127.0.0.1 only, found the 0.0.0.0 bind address problem, and let containers find each other by name on a network;
• gave one image different settings at start, kept the key in a read-only secret file (API_KEY_FILE), and made a missing key stop the app (REQUIRE_API_KEY);
• added the history in PostgreSQL (version 1.1.0, 25 tests), and ran the API and its database with Compose. The data is in the named volume ticket-api_pgdata;
• proved that the history survives new containers, and that files in a container do not.

Reset. To stop everything and delete the practice data:

docker compose down --volumes
docker image rm ticket-api:1.1.0-dev
docker volume ls
docker system df

Keep ticket-api:1.1.0: Module 5 uses it. Check in docker volume ls that ticket-api_pgdata is gone. If your project is broken, copy the snapshot end-of-m03, then make .env and secrets/api_key.txt again, as its README.md says.

Next, you leave your computer: the building blocks that every cloud offers, and how the API’s needs map to them on Azure.

Next: Regions and service types