Regression intuition

What you will be able to do

Predict a number, such as tomorrow’s ticket count, with a straight-line model. Then check whether the model still works on days it has not seen, before anyone plans with it.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has every piece of code, task and check of this lesson, with the instructions. Its setup cells install pandas and scikit-learn and download the practice data, so it needs no local setup. Read the explanations and the charts here, and run the code there.

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 60 minutes

What you need to know

• Features, labels and regression, from Supervised learning: Features and labels.
• The train set and the test set, and why a split by time copies real use, from Train, validation and test sets.
• The error of a prediction, MAE, RMSE and the mean error, from Regression metrics.
• The loss as a mean squared error, from Loss and gradients: How wrong is w = 6? The loss in Essential Mathematics and Statistics for AI.

What you need on your computer (only for the local route; the notebook needs none of it)

• The project folder ~/projects/ticket-model, with its .venv environment (pandas 3.0.6, scikit-learn 1.9.1, matplotlib 3.11.2) and the data in data/. You made them in Define a baseline: Set up your project.
• data/daily_tickets.csv (730 days). If it is missing or changed, run python get_data.py --reset in the project folder. If get_data.py is missing too, download it again: curl -fsSL -O https://raw.githubusercontent.com/Nextia-Intelligence/nextia-learning-labs/main/C05/get_data.py (curl.exe on Windows).
• Run the code of this lesson in the lesson notebook, opened in VS Code with the .venv kernel (see the box after the guided practice).

Tested on macOS with Python 3.14.6, pandas 3.0.6, scikit-learn 1.9.1 and matplotlib 3.11.2. The Windows and Linux steps are not tested. The notebook was tested in Jupyter.

A taxi fare is a straight line

Think of a taxi in a city. The price list on the door says: $3 to start, then $2 for each kilometre. So a 5 km trip costs 3 + 2 × 5 = $13, and a 10 km trip costs $23. Every extra kilometre adds the same $2. If you draw the fare against the distance, you get a straight line. It starts at $3 for 0 km and goes up by $2 for each kilometre.

Now turn the situation around. You do not have the price list. You only have six receipts from earlier trips:

Distance (km)	2	3	5	6	8	10
Fare ($)	7.50	8.50	12.50	15.50	19.00	23.00

Can you find the start fee and the price per kilometre from these receipts? The points do not sit exactly on one line, because a trip sometimes waits in traffic and sometimes does not. But they are close to a line, so you can draw the line that passes closest to all six points. Its height at 0 km is the start fee. Its rise for each kilometre is the price per kilometre.

Six taxi receipts and the straight line that passes closest to them. The line starts at $3 for 0 km and rises by $2 for each kilometre.

Text description of the chart

Six dots show the receipts: 2 km for $7.50, 3 km for $8.50, 5 km for $12.50, 6 km for $15.50, 8 km for $19.00 and 10 km for $23.00. A straight line goes from $3 at 0 km to $27 at 12 km. Four dots are 50 cents above or below the line, and the last two are exactly on it.

This is the whole idea of this lesson. A linear regression is a model that predicts a number with a straight line. The line is a starting value plus a fixed amount for each unit of a feature. Learning the model means finding these two numbers from examples, as you just did with the receipts. The starting value is the intercept (a), the prediction when the feature is 0. The amount per unit is the coefficient (b), also called the slope. The prediction is a + b × feature.

The name has two parts. “Linear” means “in a straight line”. “Regression” means a model that predicts a number, as you saw in Supervised learning.

Omar’s question: how many tickets tomorrow?

Omar, the product manager, plans the agents’ shifts the evening before, as in Translate a need. If he plans too few agents, customers wait for hours. If he plans too many, agents sit without work and Larkfield pays for it. So he asks Priya: “How many tickets will arrive tomorrow?” The answer is a number, so this is a regression task.

In Regression metrics, you scored simple forecasts that need no model, such as “predict the mean of the train days”. Such a forecast gives every day the same number. But Omar knows more about tomorrow: is it a weekend? Which month is it? A linear regression can use these facts. It works like the taxi fare: a starting number of tickets, plus a fixed amount for each fact about the day.

The data is a second, small file of the full export: daily_tickets.csv, one row per day from July 2024 to June 2026. The data is made up for this course; it has no real people. The escalation model starts in the next lesson. Regression comes first because its errors are easy to see: they are tickets.

A line through seven days

Start small, with one week of real data: 1 to 7 July 2024. The column is_weekend is 1 on Saturday and Sunday, and 0 on the other days. The column tickets is the number of tickets that arrived that day.

Day	Mon	Tue	Wed	Thu	Fri	Sat	Sun
is_weekend	0	0	0	0	0	1	1
tickets	33	33	29	36	29	13	9

Here are two models for this week:

• Constant: predict the mean of the week every day: 182 ÷ 7 = 26 tickets.
• Line: predict tickets = a + b × is_weekend, with a = 32 and b = −21. A weekday gets 32 tickets. A weekend day gets 32 − 21 = 11.

Read the line like the taxi fare. The intercept, 32, is the prediction for a normal weekday. The coefficient, −21, is what a weekend adds: here it takes away 21 tickets. Because is_weekend is only 0 or 1, the “per unit” is “for a weekend day”.

To compare the two models, look at the residual of each day: the actual value minus the predicted value. It is the error of one prediction, as in Regression metrics. Statisticians call it a residual because it is what is left after the prediction. A positive residual means the model predicted too few; a negative one, too many. The loss is one number for how wrong the model is on all days together. Here it is the mean squared error (MSE): square each residual, then take the mean, as in the mathematics course.

Day	Actual	Constant	Residual	Squared	Line	Residual	Squared
Mon	33	26	7	49	32	1	1
Tue	33	26	7	49	32	1	1
Wed	29	26	3	9	32	−3	9
Thu	36	26	10	100	32	4	16
Fri	29	26	3	9	32	−3	9
Sat	13	26	−13	169	11	2	4
Sun	9	26	−17	289	11	−2	4
			Sum	674		Sum	44

The MSE of the constant is 674 ÷ 7 = 96.29. The MSE of the line is 44 ÷ 7 = 6.29. The line is much closer, because the weekend is quiet and the line knows it. In a formula:

$$\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2$$

Read it as “the mean, over all n days, of the squared residual”. Here y is the actual count and ŷ (“y hat”) is the predicted count.

• Actual tickets
• Line: 32 − 21 × is_weekend

One week of tickets and the line model. The vertical gap between the two dots of a day is that day's residual.

Text description of the chart

The horizontal axis shows the days from Monday 1 July to Sunday 7 July 2024. The actual tickets are 33, 33, 29, 36 and 29 on the weekdays, then 13 and 9 at the weekend. The line model predicts 32 on each weekday and 11 on each weekend day. The largest gap is Thursday: 36 actual against 32 predicted, a residual of 4.

Least squares: how the line is chosen

Why a = 32 and b = −21, and not some other pair? Least squares is the rule: choose a and b so that the MSE is as small as possible. Squaring makes every residual count, whatever its sign, and it makes a large miss count much more than a small one.

For a feature that is only 0 or 1, the answer is easy to see. The best a is the mean of the weekdays: 160 ÷ 5 = 32. The best a + b is the mean of the weekend days: 22 ÷ 2 = 11, so b = 11 − 32 = −21. Any other line has a larger MSE on this week. In the notebook, you compute the MSE of one other line, 33 − 22 × is_weekend, and compare.

The taxi receipts work the same way. Least squares finds fare = 3.00 + 2.00 × km: the price list, because the traffic delays happen to cancel out in these six receipts. Another line, 4 + 1.8 × km, has an MSE of 0.49 on the receipts, against 0.17 for the best line.

Fitting a model means finding its numbers from data. You do not need to do it by hand. scikit-learn’s LinearRegression does least squares for you:

import pandas as pd
from sklearn.linear_model import LinearRegression

daily = pd.read_csv("data/daily_tickets.csv", parse_dates=["date"])
week = daily.head(7)[["date", "weekday", "is_weekend", "tickets"]]

model = LinearRegression()
model.fit(week[["is_weekend"]], week["tickets"])
print("a (intercept_):", round(model.intercept_, 2))
print("b (coef_):", model.coef_.round(2))
print("predictions:", model.predict(week[["is_weekend"]]).round(2))

The output:

Output

a (intercept_): 32.0
b (coef_): [-21.]
predictions: [32. 32. 32. 32. 32. 11. 11.]

fit takes the features as a table (week[["is_weekend"]], with two pairs of brackets), even when there is only one feature. It takes the target as one column (week["tickets"]). After fit, the learned numbers are in intercept_ and coef_, and predict uses them to make predictions.

Worked example: Priya’s forecast for May and June

Priya now uses all 730 days. She trains on the days before 1 May 2026 and tests on May and June 2026, the busy garden season. This is a split by time, as in Train, validation and test sets. The model always predicts days that come after the days it learned from, as it will in real use.

The month is a number from 1 to 12, but month 12 is not “twelve times” month 1. So she turns it into twelve columns of 0 and 1, one for each month (month_1 … month_12). Each month then gets its own coefficient, like a separate extra fee. This is safe here, because the twelve months are fixed and known in advance. (Ticket categories are different: Preprocess with a pipeline shows why.)

daily = daily.join(pd.get_dummies(daily["month"], prefix="month", dtype=int))
MONTHS = [f"month_{m}" for m in range(1, 13)]

train = daily[daily["date"] < "2026-05-01"].copy()
test = daily[daily["date"] >= "2026-05-01"].copy()
print(len(train), "train days,", len(test), "test days")
print("mean tickets per day in train:", round(train["tickets"].mean(), 2))

The output:

Output

669 train days, 61 test days
mean tickets per day in train: 25.32

She measures each model with the metrics of Regression metrics, on train and on test. MAE is the mean size of a miss, in tickets. RMSE is also in tickets, and it grows more when some misses are large. The mean test residual is the mean error on test: a positive value means that the model predicts too few on average. The function report prints all three:

from sklearn.metrics import mean_absolute_error, mean_squared_error


def report(name, train_predicted, test_predicted):
    """Print MAE and RMSE on train and test, and the mean test residual."""
    parts = []
    for part, predicted in [(train, train_predicted), (test, test_predicted)]:
        mae = mean_absolute_error(part["tickets"], predicted)
        rmse = mean_squared_error(part["tickets"], predicted) ** 0.5
        parts.append(f"MAE {mae:.2f}, RMSE {rmse:.2f}")
    residual = (test["tickets"] - test_predicted).mean()
    print(f"{name}: train {parts[0]} | test {parts[1]} | mean test residual {residual:+.2f}")

She compares three models. The first is the baseline that needs no model: predict the train mean, 25.32, every day. The second is the weekend line. The third is the weekend line plus the twelve month columns.

train_mean = train["tickets"].mean()
report("constant", [train_mean] * len(train), [train_mean] * len(test))

weekend = LinearRegression().fit(train[["is_weekend"]], train["tickets"])
print(f"tickets = {weekend.intercept_:.2f} + ({weekend.coef_[0]:.2f}) x is_weekend")
report("is_weekend", weekend.predict(train[["is_weekend"]]), weekend.predict(test[["is_weekend"]]))

COLUMNS = ["is_weekend"] + MONTHS
season = LinearRegression().fit(train[COLUMNS], train["tickets"])
train["predicted"] = season.predict(train[COLUMNS])
test["predicted"] = season.predict(test[COLUMNS])
report("is_weekend + month", train["predicted"], test["predicted"])

The output:

Output

constant: train MAE 8.52, RMSE 10.43 | test MAE 14.95, RMSE 17.27 | mean test residual +12.27
tickets = 29.99 + (-16.45) x is_weekend
is_weekend: train MAE 5.86, RMSE 7.33 | test MAE 12.54, RMSE 14.08 | mean test residual +12.45
is_weekend + month: train MAE 4.41, RMSE 5.64 | test MAE 6.45, RMSE 8.19 | mean test residual +4.98

The same results in a table:

Model	Train MAE	Train RMSE	Test MAE	Test RMSE	Mean test residual
Constant (train mean)	8.52	10.43	14.95	17.27	+12.27
is_weekend	5.86	7.33	12.54	14.08	+12.45
is_weekend + month	4.41	5.64	6.45	8.19	+4.98

Read the table from top to bottom:

• The constant is wrong by about 15 tickets a day in May and June. May and June are busy, and the constant predicts too few.
• The weekend line, tickets = 29.99 − 16.45 × is_weekend, helps on train (5.86), but not on test (12.54). It knows the weekend, but nothing about the season.
• The season model cuts the test MAE to 6.45, less than half of the constant’s error. But its mean test residual is still +4.98: it predicts about 5 tickets a day too few.

Read the residuals

A model fits the train days when its error on them is small. It generalises when its error stays small on days it has not seen. The season model fits better than it generalises: its train MAE is 4.41 and its test MAE is 6.45. The two numbers say that something is missing, but not what. The residuals, drawn over time, show it.

• Actual tickets
• Model, fitted on train
• Model, predicting test

The last four months: the actual tickets and the season model. From 1 May, the actual line is above the model on most days.

Text description of the chart

The chart shows each day from 1 March to 30 June 2026. The actual line goes up and down every week: five busy weekdays, then two quiet weekend days. The model follows the same weekly shape, with one level for each month. In March and April (train), the actual line is sometimes above and sometimes below the model. In May and June (test), the actual line is above the model on most days. The mean residual is +5.24 in May and +4.72 in June.

The mean residual of each month makes the pattern clear:

• Mean residual per month
• No error (0)

The mean residual of each month for the season model. The first year is below zero, the second year above it, and the two test months are the highest.

Text description of the chart

Each dot is one month, from July 2024 to June 2026. From July 2024 to April 2025, every month is below zero, between −1.47 and −3.18: the model predicts too many. May and June 2025 are exactly 0. From July 2025 to April 2026, every month is above zero, between +1.47 and +3.08: the model predicts too few. The two test months, May and June 2026, are the highest: +5.24 and +4.72.

The second year is busier than the first, because Larkfield has more customers each month. But the model gives every July the same level. So it predicts too many tickets in the first year and too few in the second. May and June 2025 are exact because train has only one May and one June, and the model copies them. Then it predicts May 2026 from May 2025, and the shop has grown since. No column says “time”, so the model cannot see the growth.

This is the main lesson of residuals: a pattern in the residuals is something the model does not know. Residuals that jump randomly around zero are good. A residual that stays on one side for months points to a missing feature.

Now compare the model with a forecast from Regression metrics that needs no model: “the mean of the same weekday over the last 4 weeks”. On May and June it has an MAE of 5.08 and a mean error of +1.65. The season model has 6.45 and +4.98, so the simple forecast wins. It wins because the last 4 weeks already contain the growth that the model cannot see. This is not a failure of machine learning. It is the reason why you compare every model with simple forecasts: a model is worth its cost only when it beats them. In the independent variation below, a trend feature brings the model to an MAE of 5.01, about the same as the simple forecast.

When a straight line is the right tool

A linear regression assumes that each feature adds a fixed amount, the same everywhere, like the $2 per kilometre. When that is roughly true, it is a very good first model. It learns from little data and trains in a fraction of a second. Each coefficient has a plain meaning in the unit of the target: “a weekend day has about 16 fewer tickets”. Omar can check that number against what he knows. Any more complex model for the daily tickets must beat it.

The same assumption shows its limits:

• Curves. If the effect of a feature grows or shrinks along the way, a straight line misses it. You can add features that bend it, as the month columns did.
• Effects that depend on each other. If the weekend drop is larger in summer than in winter, one weekend coefficient cannot show it.
• Values outside the data. A line keeps going. The taxi line says that a 300 km trip costs 3 + 2 × 300 = $603, although no receipt was longer than 10 km. A trend line, which you add below, assumes that growth continues forever.
• Large misses. Least squares squares the residuals, so a few extreme days pull the line toward them.

Guided practice: residuals by month

Compute the mean residual of the test months yourself. The lesson notebook has every cell; the box at the top of this page has the Colab and Kaggle links.

1. Open the notebook in Colab or Kaggle.
2. Run the setup cells. You should see pandas 3.0.6, scikit-learn 1.9.1 and the Ready: line.
3. In section 1, run the taxi cell. You should see fare = 3.00 + 2.00 x km.
4. In section 2, predict which model has the lower MSE. Then run the cell. You should see MSE constant: 96.29 and MSE line: 6.29.
5. Do the first Your turn task: the MSE of 33 − 22 × is_weekend. The check cell prints Check passed when your answer is right.
6. In section 4, run the three models. Compare each line with the table above.
7. In section 5, do the Your turn task: the mean test residual for each month, in by_month.
8. Run the next cell. It prints the train residuals by month. Compare them with the second chart.

Show the answer to step 7

test["residual"] = test["tickets"] - test["predicted"]
by_month = test["residual"].groupby(test["month"]).mean().round(2)
print(by_month)

The output:

Output

month
5    5.24
6    4.72
Name: residual, dtype: float64

Both test months are under-predicted by about 5 tickets a day. In June, 30 days × 4.72 is about 140 tickets that Omar’s plan did not expect.

Run the notebook on your computer

Download the notebook with View on GitHub, then the download button. Save it in ~/projects/ticket-model/notebooks/. In VS Code, open it and select Select Kernel › Python Environments › .venv. The notebook makes its own folder ticket-model next to it and works there, so it never changes your project files.

Independent variation: add a trend

Change one thing: add a column days, the number of days since 1 July 2024. With it, the model can learn a steady growth, like a price that goes up a little every day. Before you run it, predict: will the mean test residual go up, go down, or stay near +5? In the notebook, add "days" to trend_columns in section 6, and run the cell.

Show the answer

daily["days"] = (daily["date"] - pd.Timestamp("2024-07-01")).dt.days
train["days"] = daily.loc[train.index, "days"]
test["days"] = daily.loc[test.index, "days"]

trend_columns = ["is_weekend"] + MONTHS + ["days"]
trend = LinearRegression().fit(train[trend_columns], train["tickets"])
trend_test = trend.predict(test[trend_columns])
report("with trend", trend.predict(train[trend_columns]), trend_test)
print("tickets per day added each day:", round(trend.coef_[trend_columns.index("days")], 4))

The output:

Output

with trend: train MAE 4.09, RMSE 5.22 | test MAE 5.01, RMSE 6.52 | mean test residual +0.54
tickets per day added each day: 0.0122

The mean test residual falls from +4.98 to +0.54, and the test MAE from 6.45 to 5.01. The model learned a growth of 0.0122 tickets per day, each day. After one year, that is about 4.5 more tickets per day (0.0122 × 365). The monthly residuals of both years are now close to zero, for example −0.96 in July 2024 and +0.96 in July 2025.

A trend is also a risk. Like the taxi line, it keeps going: the model assumes that the growth continues at the same rate, forever. It cannot know about a lost customer group or a new competitor. Check its residuals every month, and retrain it when they drift away from zero.

Common mistake

A random test set hides the growth

Symptom: Tomás, the AI engineer, takes 61 days at random as the test set, instead of May and June. The season model looks better:

random_test = daily.sample(n=61, random_state=0)
random_train = daily.drop(random_test.index)
model = LinearRegression().fit(random_train[COLUMNS], random_train["tickets"])
predicted = model.predict(random_test[COLUMNS])
print("random test MAE:", round(mean_absolute_error(random_test["tickets"], predicted), 2))
print("random test mean residual:", round((random_test["tickets"] - predicted).mean(), 2))

The output:

Output

random test MAE: 4.38
random test mean residual: -1.1

An MAE of 4.38 instead of 6.45, and almost no bias. Omar would plan May and June with it.

Cause: the random test days sit between train days of the same months and years. The model has already seen their neighbours, so the growth over time stays hidden. In real use, the model always predicts days after its train days.

Fix: split by time when the model predicts the future. Train on the past and test on the later days, as in Train, validation and test sets and Choose a split in SQL and Data Preparation for AI. Then read the residuals over time.

Consequence in AI work: a model that you test on random rows of a growing business looks accurate and is wrong in one direction. Here, Omar would plan for about 5 tickets a day too few, and the gap would grow every month.

See it end to endEstimate house pricesCase study

The house-price case study fits a linear model to the log of the price, and shows why the log helps.

Knowledge check

Check your understanding

The season model has a mean test residual of +4.98. What does the sign tell Omar, and what should he do with the forecast?

Show the answer

A residual is the actual count minus the predicted count. A positive mean residual means that the model predicts too few tickets: on average, about 5 tickets a day more arrive than it says. Omar should not plan the shifts exactly to the forecast. He can add about 5 tickets a day to it for now. Better, Priya adds a feature for the growth (the trend) and checks the residuals again.

Check your understanding

Predict and explain: the season model has a train MAE of 4.41 and a test MAE of 6.45. Why is the test error larger, and why does the residual chart help more than the two numbers?

Show the answer

The model’s numbers come from the train days, so it fits them well. The test days come later, and the shop has grown since, so the model predicts too few. The two MAE values only say that the test error is larger. The residual chart says why. The residuals stay below zero for the first year and above zero for the second, and the test months are the highest. A residual that stays on one side over time points to a missing feature: time, or the growth of the customer base.

Check your understanding

A taxi line is fare = 3 + 2 × km. Name the intercept and the coefficient, say what each one means, and explain why the line is not safe for a 300 km trip.

Show the answer

The intercept is 3: the predicted fare for 0 km, the start fee. The coefficient is 2: the fare goes up by $2 for each extra kilometre. For 300 km, the line predicts 3 + 2 × 300 = $603. But the receipts went only up to 10 km. A line keeps going in the same direction, and nothing in the data says that long trips follow the same price. Real long trips may have a fixed price or a different rate. A linear regression is only reliable near the values it learned from.

Recap

You can now predict a number with a linear regression and judge it honestly:

• A linear regression predicts a + b × feature, like a taxi fare: an intercept (the start) plus a coefficient for each unit of each feature.
• A residual is actual minus predicted. Least squares chooses the line with the lowest mean squared error; LinearRegression().fit() does it for you.
• The season model (is_weekend + month) reaches a test MAE of 6.45 and an RMSE of 8.19. The constant baseline has 14.95 and 17.27.
• It fits train better than it generalises to May and June (MAE 4.41 against 6.45). The residuals show why: the shop grows, and no feature says so. A trend column cuts the mean test residual from +4.98 to +0.54.
• A random test set hides the problem (MAE 4.38). Split by time when the model predicts the future.

Next, the escalation question: a yes or no for each ticket. A straight line cannot answer it well, and the next lesson shows why.

Next: Classification intuition