Interactions, not ratings
- Lesson 4
- 35 min
- Module 2: Data, implicit feedback and an honest split
Where we are
We ran our first baseline on the shop log. The log has views, baskets and orders, but no ratings. In this lesson, we treat these events as implicit feedback, give each one a weight, and see what a missing event means.
Lesson 4 of 29 · Recommendation and Personalisation: Show Each Person What MattersInteractions, not ratings
1. The recommendation problem
2. Data, implicit feedback and an honest split
3. Measure recommendations offline
4. Neighbours and collaborative filtering
5. Matrix factorisation for implicit feedback
6. Content, hybrids and cold start
7. Two stages: candidates and ranking
8. From offline scores to the real shop
9. Run it in production
What you will be able to do
Turn what customers do (views, basket adds and orders) into one number per customer and product. Learn what a missing event can and cannot mean. Every method in this course starts from these numbers.
NotebookDo this lesson in a notebook
The notebook has the code and the task of this lesson. It weights the events of the shop log, shows one customer’s products, and counts how many “not bought” pairs were bought later. Its setup cells install the packages and download the data, so it needs no local setup. About 20 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- The Larkfield shop log (products, customers, events) and the popularity baseline, from Meet the shop log.
- Recall@10: the share of the products that a customer ordered which are in the list of 10, from What success means.
What you need on your computer
Nothing for the notebook. For the project: your larkfield-recs folder from Meet the shop log, with its virtual environment active. recs check prints Setup check passed.
The Larkfield shop log is synthetic: our code made every product, customer and event with a model of customer behaviour. The course team measured the numbers on an Apple M4 laptop. Tested on macOS; the Windows commands are not tested.
A shop assistant who watches
Section titled “A shop assistant who watches”Think of a good shop assistant in a small home and garden shop. Customers do not fill in a form about what they like. Nobody gives stars to a lamp. But the assistant watches. A woman picks up a floor lamp and looks at it for a while. She puts it down, walks around, and comes back to it. She carries it to the till. Next to it, she glances at a desk lamp for one second.
The assistant learns a lot from this. The floor lamp is a strong signal: she came back and she paid. The desk lamp is a weak signal: she looked, but that is all. And the hundreds of products she walked past tell the assistant almost nothing. Perhaps she did not see them. Perhaps she already has one at home.
A recommender must learn in the same way. It must read what people do, because people rarely say what they like.
The problem: Larkfield has no ratings
Section titled “The problem: Larkfield has no ratings”Tomás, the team’s AI engineer, read about recommenders that predict star ratings: “this customer would give this film 4 stars”. He wants to do the same for Larkfield. Priya, the data scientist, opens the shop log. There is no rating column. There are three kinds of event, and each event is one customer doing one thing with one product.
Text description of the chart
Three bars. Views: 572,465. Basket adds: 149,577. Orders: 83,274 order lines, in 57,418 orders. There are about 7 views for each order line.
So a rating model has nothing to learn from. And if Priya keeps only the orders, she throws away most of the log. Count what that costs. Before the test period (22 June 2026), 38,377 customers did something in the shop, but only 22,857 of them ever ordered. For the other 15,520 customers, the orders say nothing at all, while their views say what they looked at. In the test period, 952 of the 2,118 customers who ordered had never ordered before. For them, views and baskets are the only clues.
Implicit feedback
Section titled “Implicit feedback”Explicit feedback is what people say on purpose: a rating, a review, a thumbs-up. Implicit feedback is what people do: they view, they add to the basket, they buy, they come back. The word “implicit” means “not said directly”. The customer does not tell us “I like this lamp”. We read it from the order.
| Explicit feedback | Implicit feedback | |
|---|---|---|
| Example | 4 stars for a lamp | a view, a basket add, an order |
| How much of it | little: few people rate | a lot: every visit makes events |
| What it says | like and dislike | interest only, of different strength |
| Main trap | people who rate are not typical | a missing event is not a “no” |
Most shops, apps and media services have mostly implicit feedback. That is why this course is about it. The machine learning ideas are the same as in the machine learning course, but the data has no labels of “like” and “dislike”. It has only signals of interest.
Give each event a weight
Section titled “Give each event a weight”An order says more than a basket add, and a basket add says more than a view. So we give each kind of event a weight, a number that says how strong the signal is. Larkfield’s project uses these weights:
| Event | Weight | Why |
|---|---|---|
| view | 1 | weak: people look at many things |
| basket | 3 | stronger: they thought about buying |
| order | 5 | strongest: they paid |
Then, for each customer and product, we add the weights of all their events. The sum is the customer’s interaction with the product: one number for one pair. Here is a real customer from the log, C100134, on one day in April 2025:
Text description of the diagram
On the left are the customer’s seven events, in time order, each with its weight. The customer viewed a task lamp (+1) and a floor lamp (+1). The customer added the floor lamp to the basket (+3) and viewed a glass-column lamp (+1). Then the customer viewed the floor lamp again (+1), added it to the basket again (+3) and ordered it (+5). On the right is the customer’s row: one number for each product. The floor lamp has 13 (1 + 3 + 1 + 3 + 5), the glass-column lamp 1 and the task lamp 1. Every other product has an empty cell: no event, which is not the same as a zero rating.
The floor lamp gets 13, a strong signal. The two other lamps get 1 each: weak, but not nothing. The customer compared three lamps and chose the floor lamp.
The weights are a choice
Section titled “The weights are a choice”Nobody measured that an order is worth exactly five views. The weights 1, 3 and 5 are an assumption, a reasonable start. Other shops use other numbers. Some count only orders. Some use the logarithm of the count, so that 50 views of one product do not shout louder than one order. Later, in ALS and BPR with implicit, a method turns these sums into a “confidence”, and we tune how strongly it trusts them. For now, keep in mind: the weights are a setting, and a setting can be tested.
“Not bought” does not mean “not liked”
Section titled ““Not bought” does not mean “not liked””Now look at the cells that stay empty. For C100134, that is 2,991 of the 2,994 products. Does an empty cell mean “this customer does not like this product”? Think about the stories that can sit behind it.
Text description of the diagram
Four stories lead to the same empty cell. The customer never saw the product, because it was on page 9 of the results. The customer does not need it now, like a winter coat in July. The customer bought it in another shop. Or the customer saw it and does not want it. Only the last story is a real “no”, and the data cannot tell the four stories apart.
Implicit feedback has positive signals only. We see interest when it happens. We never see a clear “no”. A view without an order is weaker, but it is still a sign of interest. An empty cell is unknown, not negative.
Measured: what each kind of pair says about the next two weeks
Section titled “Measured: what each kind of pair says about the next two weeks”We can test this on Larkfield’s data. Take the 2,118 customers who ordered in the test period (22 June to 5 July 2026). For each customer and each product, look at what happened before the test period. Then count how often the customer ordered that product in the test period.
Text description of the chart
Four bars, for the 2,118 test customers. Pairs the customer had already ordered: 3.27% were ordered again in the test period (265 of 8,106). Pairs added to the basket but not ordered: 1.66% (83 of 5,014). Pairs only viewed: 0.62% (202 of 32,465). Pairs with no event, among the products on sale: 0.05% (3,193 of 6,295,707).
Three things stand out:
- A view is evidence. Pairs that were only viewed were ordered 0.62% of the time. Pairs with no event: 0.05%. A view makes an order about 12 times more likely.
- “Not bought” is often “not yet”. 285 pairs that were viewed or added to the basket, but never ordered, were ordered in the next two weeks. This is the number you count in the notebook.
- Most orders are new products for the customer. Of the 3,743 products ordered in the test period, 3,193 (85%) had no event at all from that customer before. A recommender cannot only repeat what a customer touched. It must find new products, and for that it uses what other customers did.
Common mistake
Treating every empty cell as a dislike
Tomás builds his first model with a simple rule: a filled cell is “liked” (1), an empty cell is “disliked” (0). Then he trains a classifier to separate the two. The model learns to push down every product that a customer has not touched yet. But 85% of the test orders were exactly such products. The model is trained to be wrong about the orders we care about most. The fix is not a better classifier. Treat empty cells as unknown. Methods for implicit feedback give them a low weight, or sample a few of them as weak negatives. You will see both in ALS and BPR with implicit.
Larkfield’s choice
Section titled “Larkfield’s choice”Priya writes down the decision for the team:
- Larkfield uses implicit feedback: views, basket adds and orders.
- Each event has a weight: view 1, basket 3, order 5. The weights are a setting that the team can test.
- The interaction of a customer with a product is the sum of the weights of their events before the test period.
- An empty cell means “unknown”. No method may treat it as a dislike.
In the project, the weights are one line of larkfield_recs/data.py:
EVENT_WEIGHTS = {"view": 1.0, "basket": 3.0, "order": 5.0}The next lesson puts these sums into one big table, the user-item matrix, and the command recs matrix builds it with any weights you give.
When to use implicit feedback, and what it costs
Section titled “When to use implicit feedback, and what it costs”| Use it when | Be careful when |
|---|---|
| You have logs of what people do (almost every shop, app or media service). | A product was shown more often than others: it gets more views because of where you put it, not because people like it more. |
| Ratings are rare, or come from a few unusual people. | An event is cheap and noisy, such as a scroll or a hover. Give it a very small weight, or leave it out. |
| You want a signal for new customers who have not bought yet. | The product is bought once in a lifetime (a sofa). An order then means “do not show me more sofas”, not “I want more”. |
Explicit ratings are still useful where they exist: the case study Books for readers uses a data set with star ratings. And the costs of implicit feedback are real. The data is noisy, positive-only, and shaped by what the shop already shows. Feedback loops and popularity bias comes back to that last point: a recommender that learns only from its own suggestions can get stuck.
Guided practice: weights in the notebook
Section titled “Guided practice: weights in the notebook”- Open the notebook of this lesson in Colab or Kaggle (the buttons at the top of the lesson), or in Jupyter on your computer.
- Run the setup cells. Wait for
Ready: 3 files in larkfield-recs. - Run section 1. It prints the number of customer-product pairs before the test period (467,160). It shows a table with the sum of the weights for each pair and the set of event kinds.
- Run the next cell. It shows one customer’s products, strongest first.
- Run section 2. It counts the pairs that were viewed or added to the basket, but never ordered.
Show the expected result
Section 1 prints 467,160 customer-product pairs before the test period. The second cell shows the products of one customer with their summed weights. Section 2 prints 393,567 pairs were viewed (or added to the basket) but not ordered: 341,546 pairs with views only and 52,021 with a basket add but no order.
Your turn: were they bought later?
Section titled “Your turn: were they bought later?”Predict first: of the 393,567 pairs that were viewed but never ordered, how many were ordered in the next two weeks? Then write the cell of section 3 and run the check.
Show the answer
285 pairs. The check prints Check passed: later is 285. The solution is one line:
later = sum(1 for (c, p) in viewed_only.index if p in truth.get(c, set()))285 looks small next to 393,567. But only 2,118 customers ordered in the test period. Most of these pairs belong to customers who did not come back in those two weeks. For the test customers, a viewed pair was ordered 12 times as often as a pair with no event (the chart above).
Knowledge check
Section titled “Knowledge check”Check your understanding
Larkfield has views, basket adds and orders, but no ratings. Explain implicit feedback in your own words, and give one reason why we keep views instead of using orders only.
Show the answer
Implicit feedback is what people do, not what they say: views, basket adds and orders are signs of interest of different strength. We keep views because they carry real information. Before the test period, 15,520 customers had events but no order, and 952 of the 2,118 test customers had never ordered. For them, views and baskets are the only clues. And a viewed pair was ordered about 12 times as often as a pair with no event.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
A customer viewed a garden table, added it to the basket, viewed it again and ordered it. What is the interaction with the weights view 1, basket 3, order 5? And what is the interaction with a garden chair that the customer never viewed?
Show the answer
1 + 3 + 1 + 5 = 10 for the table. The chair has no event, so its cell is empty. It is not a 0 rating and not a dislike. The customer may not have seen it, may not need it yet, or may have bought one elsewhere.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás says: "An empty cell means the customer did not want the product, so I will train the model to predict 0 there." Use one measured number from this lesson to explain why this is a problem.
Show the answer
85% of the products ordered in the test period (3,193 of 3,743) had no event from that customer before. A model trained to predict 0 for empty cells learns to push down exactly these products. Empty cells are unknown, not negative. Methods for implicit feedback give them a low weight or sample a few as weak negatives.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Larkfield has no ratings. It has implicit feedback: 572,465 views, 149,577 basket adds and 83,274 order lines in the synthetic log.
- We give each event a weight (view 1, basket 3, order 5) and add them for each customer and product: the interaction. The weights are an assumption, a setting that can be tested.
- Implicit feedback has positive signals only. An empty cell is unknown, not a dislike: never saw it, not now, bought elsewhere, or a real no.
- Measured: a viewed pair was ordered 12 times as often as a pair with no event (0.62% against 0.05%). 285 viewed-but-not-ordered pairs were ordered in the next two weeks.
- 85% of the test orders were products the customer had never touched. A recommender must find new products, using what other customers did.
Next, we put every interaction into one table, the user-item matrix, and see how empty it is.
Next: The user-item matrix