Skip to content
Français

Interactions, not ratings

  • Lesson 4
  • 35 min
  • Module 2: Data, implicit feedback and an honest split

Where we are

We ran our first baseline on the shop log. The log has views, baskets and orders, but no ratings. In this lesson, we treat these events as implicit feedback, give each one a weight, and see what a missing event means.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Turn what customers do (views, basket adds and orders) into one number per customer and product. Learn what a missing event can and cannot mean. Every method in this course starts from these numbers.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook has the code and the task of this lesson. It weights the events of the shop log, shows one customer’s products, and counts how many “not bought” pairs were bought later. Its setup cells install the packages and download the data, so it needs no local setup. About 20 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 35 minutes

What you need to know

  • The Larkfield shop log (products, customers, events) and the popularity baseline, from Meet the shop log.
  • Recall@10: the share of the products that a customer ordered which are in the list of 10, from What success means.

What you need on your computer

Nothing for the notebook. For the project: your larkfield-recs folder from Meet the shop log, with its virtual environment active. recs check prints Setup check passed.

The Larkfield shop log is synthetic: our code made every product, customer and event with a model of customer behaviour. The course team measured the numbers on an Apple M4 laptop. Tested on macOS; the Windows commands are not tested.

Think of a good shop assistant in a small home and garden shop. Customers do not fill in a form about what they like. Nobody gives stars to a lamp. But the assistant watches. A woman picks up a floor lamp and looks at it for a while. She puts it down, walks around, and comes back to it. She carries it to the till. Next to it, she glances at a desk lamp for one second.

The assistant learns a lot from this. The floor lamp is a strong signal: she came back and she paid. The desk lamp is a weak signal: she looked, but that is all. And the hundreds of products she walked past tell the assistant almost nothing. Perhaps she did not see them. Perhaps she already has one at home.

A recommender must learn in the same way. It must read what people do, because people rarely say what they like.

Tomás, the team’s AI engineer, read about recommenders that predict star ratings: “this customer would give this film 4 stars”. He wants to do the same for Larkfield. Priya, the data scientist, opens the shop log. There is no rating column. There are three kinds of event, and each event is one customer doing one thing with one product.

The whole shop log (synthetic): 805,316 events from 1 February 2025 to 5 July 2026. Views are most of them.
Text description of the chart

Three bars. Views: 572,465. Basket adds: 149,577. Orders: 83,274 order lines, in 57,418 orders. There are about 7 views for each order line.

So a rating model has nothing to learn from. And if Priya keeps only the orders, she throws away most of the log. Count what that costs. Before the test period (22 June 2026), 38,377 customers did something in the shop, but only 22,857 of them ever ordered. For the other 15,520 customers, the orders say nothing at all, while their views say what they looked at. In the test period, 952 of the 2,118 customers who ordered had never ordered before. For them, views and baskets are the only clues.

Explicit feedback is what people say on purpose: a rating, a review, a thumbs-up. Implicit feedback is what people do: they view, they add to the basket, they buy, they come back. The word “implicit” means “not said directly”. The customer does not tell us “I like this lamp”. We read it from the order.

Explicit feedback Implicit feedback
Example 4 stars for a lamp a view, a basket add, an order
How much of it little: few people rate a lot: every visit makes events
What it says like and dislike interest only, of different strength
Main trap people who rate are not typical a missing event is not a “no”

Most shops, apps and media services have mostly implicit feedback. That is why this course is about it. The machine learning ideas are the same as in the machine learning course, but the data has no labels of “like” and “dislike”. It has only signals of interest.

An order says more than a basket add, and a basket add says more than a view. So we give each kind of event a weight, a number that says how strong the signal is. Larkfield’s project uses these weights:

Event Weight Why
view 1 weak: people look at many things
basket 3 stronger: they thought about buying
order 5 strongest: they paid

Then, for each customer and product, we add the weights of all their events. The sum is the customer’s interaction with the product: one number for one pair. Here is a real customer from the log, C100134, on one day in April 2025:

Seven event boxes for customer C100134 on 13 April 2025, each with its weight: view task lamp +1, view floor lamp +1, basket floor lamp +3, view glass-column lamp +1, view floor lamp +1, basket floor lamp +3, order floor lamp +5. An arrow labelled add leads to three cells of the customer's row: floor lamp 13, glass-column lamp 1, task lamp 1.
Seven events become three numbers: the sum of the weights for each product. Every other product stays empty.
Text description of the diagram

On the left are the customer’s seven events, in time order, each with its weight. The customer viewed a task lamp (+1) and a floor lamp (+1). The customer added the floor lamp to the basket (+3) and viewed a glass-column lamp (+1). Then the customer viewed the floor lamp again (+1), added it to the basket again (+3) and ordered it (+5). On the right is the customer’s row: one number for each product. The floor lamp has 13 (1 + 3 + 1 + 3 + 5), the glass-column lamp 1 and the task lamp 1. Every other product has an empty cell: no event, which is not the same as a zero rating.

The floor lamp gets 13, a strong signal. The two other lamps get 1 each: weak, but not nothing. The customer compared three lamps and chose the floor lamp.

Nobody measured that an order is worth exactly five views. The weights 1, 3 and 5 are an assumption, a reasonable start. Other shops use other numbers. Some count only orders. Some use the logarithm of the count, so that 50 views of one product do not shout louder than one order. Later, in ALS and BPR with implicit, a method turns these sums into a “confidence”, and we tune how strongly it trusts them. For now, keep in mind: the weights are a setting, and a setting can be tested.

“Not bought” does not mean “not liked”

Section titled ““Not bought” does not mean “not liked””

Now look at the cells that stay empty. For C100134, that is 2,991 of the 2,994 products. Does an empty cell mean “this customer does not like this product”? Think about the stories that can sit behind it.

Four boxes: never saw it (the product was on page 9), not now (a winter coat in July), bought it elsewhere (in another shop), does not want it (the only real no). Arrows from all four lead to one dashed box: an empty cell, no event, the matrix cannot tell which story is true.
Four different stories, one empty cell. Only one of the stories is a real no.
Text description of the diagram

Four stories lead to the same empty cell. The customer never saw the product, because it was on page 9 of the results. The customer does not need it now, like a winter coat in July. The customer bought it in another shop. Or the customer saw it and does not want it. Only the last story is a real “no”, and the data cannot tell the four stories apart.

Implicit feedback has positive signals only. We see interest when it happens. We never see a clear “no”. A view without an order is weaker, but it is still a sign of interest. An empty cell is unknown, not negative.

Measured: what each kind of pair says about the next two weeks

Section titled “Measured: what each kind of pair says about the next two weeks”

We can test this on Larkfield’s data. Take the 2,118 customers who ordered in the test period (22 June to 5 July 2026). For each customer and each product, look at what happened before the test period. Then count how often the customer ordered that product in the test period.

Measured on the synthetic log. A pair with only views was ordered 12 times as often as a pair with no event.
Text description of the chart

Four bars, for the 2,118 test customers. Pairs the customer had already ordered: 3.27% were ordered again in the test period (265 of 8,106). Pairs added to the basket but not ordered: 1.66% (83 of 5,014). Pairs only viewed: 0.62% (202 of 32,465). Pairs with no event, among the products on sale: 0.05% (3,193 of 6,295,707).

Three things stand out:

  1. A view is evidence. Pairs that were only viewed were ordered 0.62% of the time. Pairs with no event: 0.05%. A view makes an order about 12 times more likely.
  2. “Not bought” is often “not yet”. 285 pairs that were viewed or added to the basket, but never ordered, were ordered in the next two weeks. This is the number you count in the notebook.
  3. Most orders are new products for the customer. Of the 3,743 products ordered in the test period, 3,193 (85%) had no event at all from that customer before. A recommender cannot only repeat what a customer touched. It must find new products, and for that it uses what other customers did.

Common mistake

Treating every empty cell as a dislike

Tomás builds his first model with a simple rule: a filled cell is “liked” (1), an empty cell is “disliked” (0). Then he trains a classifier to separate the two. The model learns to push down every product that a customer has not touched yet. But 85% of the test orders were exactly such products. The model is trained to be wrong about the orders we care about most. The fix is not a better classifier. Treat empty cells as unknown. Methods for implicit feedback give them a low weight, or sample a few of them as weak negatives. You will see both in ALS and BPR with implicit.

Priya writes down the decision for the team:

  • Larkfield uses implicit feedback: views, basket adds and orders.
  • Each event has a weight: view 1, basket 3, order 5. The weights are a setting that the team can test.
  • The interaction of a customer with a product is the sum of the weights of their events before the test period.
  • An empty cell means “unknown”. No method may treat it as a dislike.

In the project, the weights are one line of larkfield_recs/data.py:

EVENT_WEIGHTS = {"view": 1.0, "basket": 3.0, "order": 5.0}

The next lesson puts these sums into one big table, the user-item matrix, and the command recs matrix builds it with any weights you give.

When to use implicit feedback, and what it costs

Section titled “When to use implicit feedback, and what it costs”
Use it when Be careful when
You have logs of what people do (almost every shop, app or media service). A product was shown more often than others: it gets more views because of where you put it, not because people like it more.
Ratings are rare, or come from a few unusual people. An event is cheap and noisy, such as a scroll or a hover. Give it a very small weight, or leave it out.
You want a signal for new customers who have not bought yet. The product is bought once in a lifetime (a sofa). An order then means “do not show me more sofas”, not “I want more”.

Explicit ratings are still useful where they exist: the case study Books for readers uses a data set with star ratings. And the costs of implicit feedback are real. The data is noisy, positive-only, and shaped by what the shop already shows. Feedback loops and popularity bias comes back to that last point: a recommender that learns only from its own suggestions can get stuck.

  1. Open the notebook of this lesson in Colab or Kaggle (the buttons at the top of the lesson), or in Jupyter on your computer.
  2. Run the setup cells. Wait for Ready: 3 files in larkfield-recs.
  3. Run section 1. It prints the number of customer-product pairs before the test period (467,160). It shows a table with the sum of the weights for each pair and the set of event kinds.
  4. Run the next cell. It shows one customer’s products, strongest first.
  5. Run section 2. It counts the pairs that were viewed or added to the basket, but never ordered.
Show the expected result

Section 1 prints 467,160 customer-product pairs before the test period. The second cell shows the products of one customer with their summed weights. Section 2 prints 393,567 pairs were viewed (or added to the basket) but not ordered: 341,546 pairs with views only and 52,021 with a basket add but no order.

Predict first: of the 393,567 pairs that were viewed but never ordered, how many were ordered in the next two weeks? Then write the cell of section 3 and run the check.

Show the answer

285 pairs. The check prints Check passed: later is 285. The solution is one line:

later = sum(1 for (c, p) in viewed_only.index if p in truth.get(c, set()))

285 looks small next to 393,567. But only 2,118 customers ordered in the test period. Most of these pairs belong to customers who did not come back in those two weeks. For the test customers, a viewed pair was ordered 12 times as often as a pair with no event (the chart above).

Check your understanding

Larkfield has views, basket adds and orders, but no ratings. Explain implicit feedback in your own words, and give one reason why we keep views instead of using orders only.

Show the answer

Implicit feedback is what people do, not what they say: views, basket adds and orders are signs of interest of different strength. We keep views because they carry real information. Before the test period, 15,520 customers had events but no order, and 952 of the 2,118 test customers had never ordered. For them, views and baskets are the only clues. And a viewed pair was ordered about 12 times as often as a pair with no event.

Check your understanding

A customer viewed a garden table, added it to the basket, viewed it again and ordered it. What is the interaction with the weights view 1, basket 3, order 5? And what is the interaction with a garden chair that the customer never viewed?

Show the answer

1 + 3 + 1 + 5 = 10 for the table. The chair has no event, so its cell is empty. It is not a 0 rating and not a dislike. The customer may not have seen it, may not need it yet, or may have bought one elsewhere.

Check your understanding

Tomás says: "An empty cell means the customer did not want the product, so I will train the model to predict 0 there." Use one measured number from this lesson to explain why this is a problem.

Show the answer

85% of the products ordered in the test period (3,193 of 3,743) had no event from that customer before. A model trained to predict 0 for empty cells learns to push down exactly these products. Empty cells are unknown, not negative. Methods for implicit feedback give them a low weight or sample a few as weak negatives.

Recap

  • Larkfield has no ratings. It has implicit feedback: 572,465 views, 149,577 basket adds and 83,274 order lines in the synthetic log.
  • We give each event a weight (view 1, basket 3, order 5) and add them for each customer and product: the interaction. The weights are an assumption, a setting that can be tested.
  • Implicit feedback has positive signals only. An empty cell is unknown, not a dislike: never saw it, not now, bought elsewhere, or a real no.
  • Measured: a viewed pair was ordered 12 times as often as a pair with no event (0.62% against 0.05%). 285 viewed-but-not-ordered pairs were ordered in the next two weeks.
  • 85% of the test orders were products the customer had never touched. A recommender must find new products, using what other customers did.

Next, we put every interaction into one table, the user-item matrix, and see how empty it is.

Next: The user-item matrix