Structured records

What you will be able to do

Turn raw ticket records, as they arrive from a file, into checked Ticket objects, and report only on those. The program finds and names each record with a problem before any report uses it. So a typing mistake in the data cannot become a wrong number in the report.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every step, task and check of this lesson, with the instructions. Its first cells make the ticket-cleaner folder as at the start of this lesson, so it works without your local project. Read the explanations here, and run the code there. It needs no install.

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• Modules, packages and imports, from Modules and imports.
• The contract of a function, from Function contracts: The contract.
• Two names for one list, from Collections: Two names, one list. It explains why a ticket is frozen.

What you need on your computer (only for the local route; the notebook and the boxes on this page need none of it)

• The package ticket_cleaner/ with __init__.py and report.py, and count_tickets.py, which imports from it. You made them in Modules and imports: Guided practice. If they are missing or different, do that section again. You can also copy the files from the setup cell of this lesson’s notebook. It has every file as at the start of this lesson.
• A terminal in ~/projects/ticket-cleaner. You made the folder in Files and terminal: Guided practice.
• The project’s environment .venv, active in your terminal (the prompt starts with (.venv)). You made it in Isolate dependencies: Guided practice. If it is missing or broken, delete it and make it again from the lock file, as in Make the environment again from nothing.

Tested with Python 3.14.6 on macOS and Pyodide 314.0 (browser). The notebook was tested in Jupyter.

Raw records are not tickets yet

Until now, the tickets were perfect: the right keys, a status in small letters, a priority that is a number. Real data is not like that. When you read a file in Module 4, each record arrives as a dictionary of strings, typed by people:

{"id": "t-1003", "status": "Open ", "category": "login", "priority": "1"}
{"id": "T-1004", "status": "opne", "category": "billing", "priority": "2"}
{"id": "T-1005", "status": "open", "category": "billing", "priority": "high"}

The first record is valid after cleaning: small letters, no extra spaces. The second has a typing mistake in the status. The third has a priority that is not a number. If the report functions get these records, filter_by_status misses "Open " and "opne", and average_priority stops with an error at "high".

So the program has three steps, each in its own module:

Step	Module	Job
Validate	parsing.py	Find the problems in one raw record.
Parse	parsing.py	Turn a valid raw record into a clean Ticket.
Report	report.py	Calculate with Ticket objects only.

The report functions can then trust their input. Every check is in one place.

Type hints: the contract on the def line

A type hint says the type that a parameter expects, or that a function returns. It is written after a : for a parameter, and after -> for the return value:

def average_priority(tickets: list[Ticket]) -> float | None:

Read it as: “average_priority takes a list of Ticket objects, and returns a float or None”. The common hints are:

Hint	Meaning
str, int, float, bool	One value of that type.
list[str]	A list of strings.
dict[str, int]	A dictionary with string keys and integer values.
float | None	A float, or None. The | means “or”.
-> None	The function returns nothing useful (it prints, or saves a file).

Python does not check type hints when the program runs. average_priority("abc") still runs, and fails later. The hints are for people and for tools. VS Code reads them and suggests the right names as you type. It also underlines a call with the wrong type before you run anything.

Dataclasses: a type for a ticket

A dictionary can hold any keys. If you type ticket["stauts"], you find the mistake only when that line runs. For a record with a fixed set of fields, make your own type instead. A class defines a new type, and a dataclass is the shortest way to write a class that mainly holds data:

ticket_cleaner/records.py

from dataclasses import dataclass

STATUSES = {"open", "pending", "closed"}
PRIORITIES = {1, 2, 3}
REQUIRED_FIELDS = ("id", "status", "category", "priority")


@dataclass(frozen=True)
class Ticket:
    """One valid support ticket."""

    id: str
    status: str
    category: str
    priority: int

    def is_urgent(self) -> bool:
        """Return True for priority 1, the most urgent level."""
        return self.priority == 1

Part	Meaning
@dataclass	Tells Python to write the usual code for you: making a ticket from four values, showing it in a readable way, and comparing two tickets.
(frozen=True)	A ticket cannot change after you make it.
id: str and the next lines	The fields, each with a type hint. Each ticket has these four values.
def is_urgent(self)	A method: a function that belongs to the type. self is the ticket that you call it on.

You make a ticket by calling the class like a function, and read its fields with a dot:

ticket = Ticket(id="T-1003", status="open", category="login", priority=1)
print(ticket)                # Ticket(id='T-1003', status='open', category='login', priority=1)
print(ticket.status)         # open
print(ticket.is_urgent())    # True

Run Python in your browser

Make and use a Ticket

The last line has a typing mistake. Run the code, and read the last line of the error. Then fix the name, and add a line that tries to change the status: ticket.status = "closed". What happens?

Python code (you can change it)from dataclasses import dataclass @dataclass(frozen=True) class Ticket: """One valid support ticket.""" id: str status: str category: str priority: int def is_urgent(self) -> bool: """Return True for priority 1, the most urgent level.""" return self.priority == 1 ticket = Ticket(id="T-1003", status="open", category="login", priority=1) print(ticket) print(ticket.status, ticket.is_urgent()) same = Ticket(id="T-1003", status="open", category="login", priority=1) print("Equal:", ticket == same) print(ticket.stauts)

Why frozen=True? In Collections, two names for one list caused changes in unexpected places. A frozen ticket cannot change, so it is safe to share between lists and functions. To get a ticket with a different status, make a new one. The replace function from dataclasses copies a ticket with one field changed: replace(ticket, status="closed").

Note

A dataclass is one kind of class. You can also write a class by hand, with an __init__ method that stores each value, and other methods. In this course, dataclasses are enough. Use a full class when a type has important behaviour, not only data, and you see that a dataclass does not fit.

Worked example: validate, then parse

The validation function returns a list of problems. An empty list means that the record is valid. This is better than True or False, because the report can say why a record was rejected. It also finds all the problems in a record at once, not only the first one.

ticket_cleaner/parsing.py

from ticket_cleaner.records import PRIORITIES, REQUIRED_FIELDS, STATUSES, Ticket


def clean_text(value: object) -> str:
    """Return the value as text with no outer spaces, in small letters."""
    if value is None:
        return ""
    return str(value).strip().lower()


def find_problems(raw: dict) -> list[str]:
    """Return a list of problems with one raw record. An empty list means valid."""
    problems = []
    for field in REQUIRED_FIELDS:
        if clean_text(raw.get(field)) == "":
            problems.append(f"missing {field}")

    status = clean_text(raw.get("status"))
    if status and status not in STATUSES:
        problems.append(f"unknown status {status!r}")

    priority = clean_text(raw.get("priority"))
    if priority and not (priority.isdigit() and int(priority) in PRIORITIES):
        problems.append(f"priority must be 1, 2 or 3, not {priority!r}")
    return problems


def to_ticket(raw: dict) -> Ticket:
    """Make a Ticket from a raw record that has no problems."""
    return Ticket(
        id=clean_text(raw["id"]).upper(),
        status=clean_text(raw["status"]),
        category=clean_text(raw["category"]),
        priority=int(clean_text(raw["priority"])),
    )

Some details:

• clean_text accepts any value (object), because a JSON file can contain numbers, and str(2) is "2". raw.get(field) gives None for a missing key, and clean_text(None) gives "", so a missing key and an empty value are the same problem.
• {status!r} puts quotes around the value in the message: unknown status 'opne'. Then a value with a space, such as 'open ', is visible.
• priority.isdigit() is True only if every character is a digit. The and stops before int(priority) when it is False, so int("high") never runs.
• to_ticket does not check anything. Its contract says that the record has no problems. The caller must call find_problems first.

The report functions get type hints and use the fields with a dot instead of ["..."]:

ticket_cleaner/report.py

from ticket_cleaner.records import Ticket


def filter_by_status(tickets: list[Ticket], status: str = "open") -> list[Ticket]:
    """Return the tickets that have the given status."""
    return [ticket for ticket in tickets if ticket.status == status]


def count_by_category(tickets: list[Ticket]) -> dict[str, int]:
    """Return the number of tickets in each category, sorted by category."""
    counts = {}
    for ticket in tickets:
        counts[ticket.category] = counts.get(ticket.category, 0) + 1
    return dict(sorted(counts.items()))


def average_priority(tickets: list[Ticket]) -> float | None:
    """Return the average priority, or None when there are no tickets."""
    if not tickets:
        return None
    return round(sum(ticket.priority for ticket in tickets) / len(tickets), 1)

sum(ticket.priority for ticket in tickets) adds up the priorities in one line. It works like a list comprehension, without the square brackets.

This diagram shows how main() uses the five functions, and the type that goes in and out of each one:

The call graph: main() calls each function in order. The types show what each function needs and what it gives back.

Text description of the diagram

main() first calls find_problems for each raw row (a dict), which returns a list[str]. For each row with no problems, it calls to_ticket, which returns a Ticket. Then it calls three functions in report.py with lists of tickets. filter_by_status returns the selected list[Ticket]. count_by_category returns a dict[str, int]. average_priority returns a float or None. Only main() prints.

Guided practice: build the three modules

1. Make ticket_cleaner/records.py with the code from the dataclass section.

2. Make ticket_cleaner/parsing.py with the code from the worked example.

3. Replace the contents of ticket_cleaner/report.py with the typed version.

4. Replace count_tickets.py with this version. The data now arrives as raw strings, as from a file:

   count_tickets.py

   from ticket_cleaner.parsing import find_problems, to_ticket
   from ticket_cleaner.records import Ticket
   from ticket_cleaner.report import average_priority, count_by_category, filter_by_status
   
   SAMPLE_ROWS = [
       {"id": "T-1001", "status": "open", "category": "billing", "priority": "2"},
       {"id": "T-1002", "status": "closed", "category": "login", "priority": "3"},
       {"id": "t-1003", "status": "Open ", "category": "login", "priority": "1"},
       {"id": "T-1004", "status": "opne", "category": "billing", "priority": "2"},
       {"id": "T-1005", "status": "open", "category": "billing", "priority": "high"},
       {"id": "T-1006", "status": "closed", "category": "shipping", "priority": "1"},
   ]
   
   
   def main() -> None:
       tickets: list[Ticket] = []
       for raw in SAMPLE_ROWS:
           problems = find_problems(raw)
           if problems:
               print(f"Skipped {raw['id']}: {'; '.join(problems)}")
           else:
               tickets.append(to_ticket(raw))
   
       status = "open"
       selected = filter_by_status(tickets, status)
       print(f"Tickets with status '{status}': {len(selected)}")
       if not selected:
           print("No tickets to count.")
           return
       for category, count in count_by_category(selected).items():
           print(f"  {category}: {count}")
       print(f"Average priority: {average_priority(selected)}")
   
   
   if __name__ == "__main__":
       main()

   "; ".join(problems) joins the strings in a list into one string, with "; " between them.

5. Run it:

   python count_tickets.py

   Output

   Skipped T-1004: unknown status 'opne'
   Skipped T-1005: priority must be 1, 2 or 3, not 'high'
   Tickets with status 'open': 2
     billing: 1
     login: 1
   Average priority: 1.5

   t-1003 with "Open " is counted as an open ticket, because to_ticket cleaned it. The two records with problems are named, not lost.

Your turn: one more rule

The help desk says that every ticket ID must start with T-. Add this rule to find_problems in the box below. For an ID that does not start with T- (after cleaning), add the problem "id must start with 'T-'". A missing ID must give only "missing id", not both problems.

Run Python in your browser

Your turn: a rule for the ID

{**good, "id": "X-9"} makes a copy of good with a different ID. The box checks all three cases. This rule is for practice. Do not add it to your project, because the next modules expect the rules above.

Python code (you can change it)STATUSES = {"open", "pending", "closed"} PRIORITIES = {1, 2, 3} REQUIRED_FIELDS = ("id", "status", "category", "priority") def clean_text(value: object) -> str: if value is None: return "" return str(value).strip().lower() def find_problems(raw: dict) -> list[str]: problems = [] for field in REQUIRED_FIELDS: if clean_text(raw.get(field)) == "": problems.append(f"missing {field}") status = clean_text(raw.get("status")) if status and status not in STATUSES: problems.append(f"unknown status {status!r}") priority = clean_text(raw.get("priority")) if priority and not (priority.isdigit() and int(priority) in PRIORITIES): problems.append(f"priority must be 1, 2 or 3, not {priority!r}") # Your rule here. return problems good = {"id": "t-1003", "status": "open", "category": "login", "priority": "1"} print(find_problems(good)) print(find_problems({**good, "id": "X-9"})) print(find_problems({**good, "id": ""}))

Show one answer

    ticket_id = clean_text(raw.get("id"))
    if ticket_id and not ticket_id.startswith("t-"):
        problems.append("id must start with 'T-'")

clean_text makes the ID small letters, so compare with "t-". The check if ticket_id and ... skips an empty ID, which already has the problem missing id.

Common mistake

A field name with a typing mistake

Symptom: after you change the dictionaries to Ticket objects, a report function stops with an error like this:

TypeError: 'Ticket' object is not subscriptable

or, with a typing mistake in a field name:

AttributeError: 'Ticket' object has no attribute 'stauts'. Did you mean: 'status'?

Cause: the first error means that the code still uses dictionary access, ticket["status"], on a Ticket. A dataclass has fields, not keys. “Subscriptable” means “can be read with [ ]”. The second error is a field name that does not exist. Python suggests the closest name.

Fix:

1. Read the line number in the error, and change ticket["status"] to ticket.status.
2. For a misspelled name, use the name that Python suggests.
3. To find these mistakes before you run the code, look for underlined names in VS Code. The editor knows the fields of Ticket from its type hints.

Check your understanding

Why does find_problems return a list of strings instead of True or False?

Show the answer

A list says why a record is not valid, and it can hold more than one problem. The program can show the reasons in the report, so a person can fix the data. An empty list still works as “valid”, because an empty list counts as false: if problems:.

Check your understanding

Module check: check to_ticket and find_problems separately. Give one assert for each, and explain the contract of to_ticket in plain words.

Show the answer

For example: assert find_problems({"id": "T-1", "status": "opne", "category": "x", "priority": "2"}) == ["unknown status 'opne'"], and assert to_ticket({"id": " t-7 ", "status": "Open", "category": "Login", "priority": "1"}) == Ticket(id="T-7", status="open", category="login", priority=1). Contract of to_ticket: it takes a raw record that has no problems, and returns a clean Ticket. The text is cleaned, and the priority is an integer. It does not check the record. The caller must call find_problems first.

Recap

• Type hints show a function’s contract on its def line: def f(tickets: list[Ticket]) -> float | None:. Python does not check them when it runs, but your editor does.
• A dataclass is a short way to define a type for a record. Read its fields with a dot. frozen=True stops changes.
• Validate first, and return a list of problems, so that the report can say why a record was rejected.
• Parse only valid records, into clean objects. The report functions then work with clean data only.
• Keep each step in its own module: records.py, parsing.py and report.py.

You can now organise code into functions and modules with clear contracts. In the next module, the raw records come from real CSV and JSON files, and from a web service.

Next: Reading and writing files