Translate a need

What you will be able to do

Turn a business need into a prediction problem before you write any model code. Say who acts on the prediction, for which unit, at which moment and from which facts. A model for an unclear need can get a good score and still help nobody, and nothing warns you.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook is a first look at the data in the terms of the decision. It shows the rows, the escalation rate, the senior team’s capacity per day and one ticket at prediction time. Its setup cells install pandas and scikit-learn and download the practice data, so it needs no local setup. About 20 minutes.

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• What a model, training and a prediction are, from What machine learning is.
• Features and labels, from Supervised learning: Features and labels.
• Classification and regression, and when a rule is enough, from Choose the learning task.
• The target escalated_72h, the prediction time and leakage, from Features and target timing in SQL and Data Preparation for AI. Which columns are known when a ticket is created, from Features and target timing: What each column knows.
• The split by time into train, validation and test sets, from Choose a split in the SQL course.
• Filtering rows and groupby in pandas, from Transform with Python in the SQL course.

What you need on your computer (only for the local route; the notebook needs none of it)

• Nothing yet. You make the project folder ~/projects/ticket-model in the next lesson, in Define a baseline: Set up your project. Until then, use the notebook in Colab or Kaggle.
• If you already made the project, its data must be complete. Run python get_data.py in the project folder. The last line must be Ready: train 16939 rows (1983 escalated), valid 1179 (183), test 1002 (157). If it is different, run python get_data.py --reset.

The notebook was tested in Jupyter on macOS. It used Python 3.14.6, pandas 3.0.6 and scikit-learn 1.9.1.

From a need to a question

A baker says: “I throw away too much bread every evening.” That is a real need, but nobody can build a model from it yet. A model gives answers, so first you must know which question it answers, and what the baker will do with the answer.

So you ask the baker a few questions. What will you do differently? “I will bake fewer loaves on quiet days.” So the prediction must say how many loaves of each kind will sell on a given day. When do you need the answer? “At five in the morning, before I start the ovens.” So the prediction can use only what is known at five: the day of the week, the weather forecast, a holiday. It cannot use the number of loaves sold by noon, because at five that number does not exist yet.

Without these questions, it is easy to build the wrong thing. Suppose someone builds a model that predicts the number of loaves thrown away, from the day’s sales. It may get an excellent score. But the baker cannot use it at five in the morning, because the day’s sales are not known then. The score is good, the model is useless, and nothing in the code says so.

This lesson is about these questions. Before any model exists, you translate a need into five answers: the decision, the prediction unit, the target, the prediction time and the features. Each answer limits what the model may do, so you write them down first.

Larkfield’s model, and the data for it

You met Larkfield in What machine learning is. It is a fictional online shop for home and garden products. Customers contact support, and each contact is a ticket. In the SQL course, you prepared Larkfield’s help-desk data for a model. In this course, you train that model, test it honestly and hand it over. The data is made up for this course; it has no real people.

The data is bigger now. The SQL course used a sample export of 240 customers over six months. That was enough to practise data preparation, but its validation month had only 45 escalations. With so few, any difference between two models was smaller than the noise. So Mei, the data engineer, ran the same preparation on the full export: about 1,000 customers over two years, July 2024 to June 2026. The columns, the target and the split rule are the same as at the end of the SQL course. Ticket IDs now have six digits (T-100001) and customer IDs five (C-00001), so you cannot mix up the two datasets.

You met most of the team in Module 1. They are the same people as in the SQL course. Priya guides you through the course, and the others each bring their own point of view:

Person	Role	In this course
Priya	Data scientist	Trains and evaluates the model. She guides you through the course.
Grace	Support lead	Owns the decision. She knows the senior team’s capacity and what an error costs.
Mei	Data engineer	Prepared the data and owns get_data.py.
Tomás	AI engineer	Wants the most complex model and the highest score. He often suggests a shortcut.
Omar	Product manager	Asks what a number means for customers and for the business.

Start from the decision

Grace describes the problem in her own words:

“Some tickets need a senior agent. When we find out late, the customer has already waited a day in the normal queue. I want those tickets to go to the senior team when they arrive.”

Like the baker’s words, this is a need, not yet a prediction problem. Priya turns it into five answers, one by one.

First, the decision: the action that a person takes differently because of the prediction. Here, Grace’s team sends a flagged ticket straight to the senior team, instead of the normal queue. A prediction that changes no action has no value, however good its score. The decision also brings a limit: the senior team can review at most 1 in 5 new tickets (20%).

Second, the prediction unit: the thing that one prediction is about. Grace’s team acts on one ticket at a time, so the unit is one ticket, and the data has one row per ticket.

Third, the target: the exact thing that the model learns to predict. In Supervised learning, you called it the label. Here it is escalated_72h: 1 if a senior agent took the ticket within 72 hours after it was created, otherwise 0. The 72 hours make the target precise, so that everyone counts the same tickets.

Fourth, the prediction time: the moment when the model must give its answer. Grace wants the ticket routed when it arrives, so the prediction time is the moment when the ticket is created. Later is too late for the decision.

Fifth, the features: the facts that the model may use as input. Every feature must be known at the prediction time. That rule gives the 10 feature columns from the SQL course, which all exist when a ticket is created.

The five answers fit in one table:

Question	Larkfield’s answer
Decision: what does a person do differently because of the prediction?	Grace’s team sends a flagged ticket straight to the senior team. The senior team can review at most 1 in 5 new tickets (20%).
Prediction unit: one prediction is about one …	ticket.
Target: what exactly do we predict?	escalated_72h: 1 if a senior agent took the ticket within 72 hours after it was created, otherwise 0.
Prediction time: when must the prediction exist?	When the ticket is created.
Features: which facts may the model use?	Only facts known at the prediction time: the 10 feature columns from the SQL course.

One ticket at the prediction time

To make the prediction time concrete, look at one real row of the data. Ticket T-116944 is the first ticket of May 2026 that escalated. A customer sent it through the web form at 07:47 on 1 May. The table shows each value and whether it exists at 07:47:

Column	Value	Known at 07:47 on 1 May?
created_at	2026-05-01 07:47:30	Yes: the prediction time
channel, team, priority	web_form, warranty, 2	Yes
segment, region	trade, west	Yes: from the customer record
order_value, word_count	549.72, 86	Yes
customer_tenure_days, prior_tickets_90d, created_hour	1887, 2, 7	Yes
escalated_72h	1	No. Known only after 72 hours.

At 07:47, the model sees every row except the last one. From those facts, it must say “send to the senior team” or “normal queue”. Note the priority: 2, not 1. A simple rule “send priority 1 tickets to the senior team” would not flag this ticket.

From the decision to the metric

Once the decision is clear, it also tells you how to judge the model. A metric is a number that measures how good the predictions are. Choose the metric from the decision, not from habit. The diagram follows the chain for Larkfield.

The decision fixes the unit, the time and the target. The two kinds of error, and their costs, choose the metrics.

Text description of the diagram

The chain starts with the decision: Grace’s team sends a flagged ticket to the senior team. The decision fixes the prediction: one ticket, when it is created. The prediction fixes the target, escalated_72h: was the ticket escalated within 72 hours? The model’s output becomes a flag, yes or no, for each new ticket. A flag can be wrong in two ways. A wrong “no” is a missed escalation: the customer waits about a day more. A wrong “yes” is an unnecessary review: about 10 minutes of a senior agent. Recall counts the missed escalations, and precision counts the unnecessary reviews. Both lead to the metrics. The share of tickets flagged must be at most 20%, and recall must be at least 40%. Priya also reports the precision of the flags.

Each flag is either right or wrong, and a wrong flag can be wrong in two ways. A missed escalation is a ticket that the model did not flag, but that escalated. The customer with a serious problem waits in the normal queue, often a day more, and some of these customers leave Larkfield. An unnecessary review is a ticket that the model flagged, but that did not escalate. A senior agent then opens a ticket that a normal agent could solve, which costs about 10 minutes. Grace’s rough numbers, side by side:

Error	What happens	Cost
Missed escalation (not flagged, but it escalated)	The customer with a serious problem waits in the normal queue.	Often a day more. Some of these customers leave Larkfield.
Unnecessary review (flagged, but it did not escalate)	A senior agent opens a ticket that a normal agent could solve.	About 10 minutes of a senior agent.

One missed escalation costs much more than one unnecessary review. So Omar asks: “Why not flag every ticket? Then we miss nothing.” The answer is the capacity. The senior team can review at most 20% of new tickets, and every unnecessary review uses a place that a real escalation could have had. So the question for the model is this: within 20% of the tickets, how many escalations can it catch? Two numbers answer it, and each one counts one kind of error:

• Recall answers “of the tickets that escalated, how many did we flag?” A missed escalation lowers it.
• Precision answers “of the tickets that we flagged, how many really escalated?” An unnecessary review lowers it.

In the next lesson, you measure both for simple rules, and you see where the 40% in the diagram comes from. Classification metrics explains them fully, with their formulas.

Worked example: the first look at the data

Priya now looks at the data through the five answers. Each code block below is a cell of the lesson notebook, whose setup cells have already loaded pandas as pd. First, she loads the three parts of the split. Her function read() keeps the text unknown as a category, so that only an empty cell becomes a missing value. (In Module 4, read() moves into the file ticket_model.py.)

def read(path):
    """Read a ticket file. Only an empty cell is missing: "unknown" stays a category."""
    return pd.read_csv(path, parse_dates=["created_at"], dtype={"customer_id": "string"},
                       keep_default_na=False, na_values=[""])


train = read("data/train.csv")
valid = read("data/valid.csv")
test = read("data/test.csv")
for name, part in [("train", train), ("valid", valid), ("test", test)]:
    first, last = part["created_at"].min().date(), part["created_at"].max().date()
    print(f"{name}: {len(part)} tickets, {part['escalated_72h'].sum()} escalated "
          f"({part['escalated_72h'].mean():.1%}), {first} to {last}")

The output:

train: 16939 tickets, 1983 escalated (11.7%), 2024-07-01 to 2026-04-30
valid: 1179 tickets, 183 escalated (15.5%), 2026-05-01 to 2026-05-31
test: 1002 tickets, 157 escalated (15.7%), 2026-06-01 to 2026-06-27

About 1 in 8 train tickets escalated. In May and June, the rate is higher, because a new warranty policy started on 1 May 2026 (the SQL course met it first). The split uses the same rule as the SQL course: train is the past, the validation set is May, and the test set is June. June 28 to 30 is not in the data, because those targets were not final at the export. Train, validation and test sets explains the job of each part.

Next, Priya turns Grace’s capacity into tickets per day, for May. A share such as 20% is hard to picture, and a number of tickets per day is what the senior team feels:

days = valid["created_at"].dt.date.nunique()
per_day = len(valid) / days
print("May:", days, "days,", round(per_day, 1), "new tickets per day")
print("Capacity, 20% of new tickets:", round(0.2 * per_day, 1), "tickets per day")
print("Escalations per day in May:", round(valid["escalated_72h"].sum() / days, 1))

The output:

May: 31 days, 38.0 new tickets per day
Capacity, 20% of new tickets: 7.6 tickets per day
Escalations per day in May: 5.9

The senior team can review about 7 or 8 new tickets a day, and about 6 tickets a day escalate. So a perfect list of flags would fit. A real model makes mistakes, though, and each unnecessary review uses one of the 7 or 8 places.

Last, Priya checks the columns that are not features. The file extra_features.csv has four more columns for each ticket. For T-116944:

extra = pd.read_csv("data/extra_features.csv")
extra[extra["ticket_id"] == "T-116944"].T

The output:

                             16943
ticket_id                 T-116944
prior_escalations_90d            0
order_value_band              high
customer_escalation_rate    0.1739
priority_now                     1

For each column, she asks one question: when is this value written? If it is written before the ticket or when the ticket is created, the model may use it. If it is written later, the model at 07:47 cannot know it:

Column	When it is written	Feature?
prior_escalations_90d	Before the ticket: the customer’s escalations in the 90 days before it.	Yes. Select useful features tests it.
order_value_band	At creation: none, low, mid or high, from order_value.	Allowed, but it repeats order_value.
customer_escalation_rate	After the export: the customer’s rate over all their tickets, including later ones.	Never. It contains later targets.
priority_now	During the ticket’s life. The help desk sets it to 1 when a ticket is escalated.	Never. The escalation writes it.

priority_now is 1 for T-116944, but its priority at creation was 2. The export knows the future of every ticket. The model at 07:47 does not.

Guided practice: the first look

Do the worked example in the notebook. You do not need a local project.

1. Open the notebook from the box Do this lesson in a notebook at the top of this page.
2. Run the setup cells, from the top. The last line of the data cell is Ready: train 16939 rows (1983 escalated), valid 1179 (183), test 1002 (157).
3. In section 2, write your prediction for the escalation rate, then run the cell. You see 11.7% for train and 15.5% for valid.
4. Run the capacity cell. You see 7.6 tickets per day.
5. In section 3, run the two cells for T-116944. You see priority 2 in its row and priority_now = 1 in the extra columns.
6. In the Your turn cell, put the names of the extra columns that are known at creation in known_at_creation.
7. Run the check cell. It prints Check passed when your list is right.

Show the answer to step 6

known_at_creation = ["prior_escalations_90d", "order_value_band"]

Both values exist when the ticket is created. customer_escalation_rate and priority_now are written later.

Your turn: a different decision

Change one thing: the decision. Omar, the product manager, asks: “How many tickets will arrive tomorrow? I plan the agents’ shifts the evening before.” Write the five answers for Omar’s decision. In the notebook, section 5, load data/daily_tickets.csv and compute the mean number of tickets per day.

Show the answer

Question	Omar’s decision
Decision	Omar plans how many agents work tomorrow.
Prediction unit	One day, not one ticket.
Target	The number of tickets created on that day (tickets in daily_tickets.csv).
Prediction time	The evening before the day.
Features	Facts known the evening before: the weekday, whether it is a weekend, the month. Not the day’s own tickets.

The mean is 26.35 tickets per day, over 730 days. The cost of an error changes too: too few agents means long waits, and too many means paid hours with no work. So a new decision gives a new unit, a new target and new costs. The target is now a number, not a yes or no, so this is a regression task, as in Choose the learning task.

Common mistake

A column from the future

Symptom: Tomás counts the escalations for each value of priority_now in train. Every one of the 1,983 escalations has priority_now = 1. “This column finds all of them. Let’s use it.”

              size   sum  mean
priority_now
1             3814  1983  0.52
2             5140     0  0.00
3             7985     0  0.00
Escalations with priority_now = 1: 1983 of 1983

Cause: priority_now does not predict the escalation; it records it. The help desk sets it to 1 when a senior agent takes the ticket. At the prediction time, nobody has escalated the ticket yet, so priority_now is the same as priority. In May, only 61 of the 183 escalated tickets had priority 1 at creation.

Fix: for every column, ask “when is this value written?” Keep only the columns that are known at the prediction time. A column that finds every case is a warning sign, not a gift.

Consequence in AI work: a model with this column gets an excellent score in evaluation. In real use, the column has no information, so the model finds about one escalation in three. That is no better than a simple rule, and everyone expected much more. This is leakage, which you met in the SQL course. In Select useful features, you measure what it does to a model’s score.

Knowledge check

Check your understanding

Give Larkfield's five answers: the decision, the prediction unit, the target, the prediction time and the features.

Show the answer

• Decision: Grace’s team sends a flagged ticket straight to the senior team, which can review at most 20% of new tickets.
• Prediction unit: one ticket.
• Target: escalated_72h, 1 if a senior agent took the ticket within 72 hours after creation, otherwise 0.
• Prediction time: when the ticket is created.
• Features: only facts known at creation: the 10 feature columns from the SQL course.

Check your understanding

Predict: Tomás adds customer_escalation_rate (the customer's rate over all their tickets) as a feature. Is it known at the prediction time? What happens to the evaluation score, and what happens in real use?

Show the answer

It is not known at the prediction time. It comes from all the customer’s tickets, including tickets created later. So it contains their targets, and sometimes the target of the ticket itself. The evaluation score goes up, because the column carries part of the answer. In real use, the later tickets do not exist yet, so the model works worse than its score said. Select useful features measures this: the score rises far above the honest one.

Check your understanding

What action does a flag enable at Larkfield? What does each kind of error cost, and why can Grace not simply flag every ticket?

Show the answer

A flag sends the ticket straight to the senior team, so a serious problem gets help on the first day. A missed escalation leaves the customer in the normal queue, often a day more, and some customers leave. An unnecessary review costs about 10 minutes of a senior agent. A miss costs more, but the senior team can review at most 20% of new tickets, about 7.6 a day in May. Each unnecessary review uses a place that a real escalation could have had.

Recap

• A need is not yet a prediction problem. Before any model, write five answers: the decision, the prediction unit, the target, the prediction time and the features.
• For Larkfield: a flagged ticket goes to the senior team, at most 20% of new tickets. One ticket, at creation, target escalated_72h, with the 10 features from the SQL course.
• The full export has 16,939 train tickets (11.7% escalated), 1,179 in May (15.5%) and 1,002 in June (15.7%). The senior team can review about 7.6 May tickets a day.
• A missed escalation costs a customer about a day; an unnecessary review costs about 10 minutes. The capacity limits the reviews. So the questions are: within 20% flagged, how many escalations do we catch (recall), and how many flags are right (precision)?
• A column written after the prediction time, such as priority_now, can look perfect. At creation, it finds only 61 of 183 escalations.

Next, you measure the simplest predictions, without a model, and set the result that a model must beat.

Next: Define a baseline