ALS and BPR with implicit
- Lesson 15
- 40 min
- Module 5: Matrix factorisation for implicit feedback
Where we are
We factorised a tiny matrix by hand. Real data needs a fast library and the right settings. In this lesson, we train ALS and BPR with the implicit library, and we measure their time, memory and scores.
Lesson 15 of 29 · Recommendation and Personalisation: Show Each Person What MattersALS and BPR with implicit
1. The recommendation problem
2. Data, implicit feedback and an honest split
3. Measure recommendations offline
4. Neighbours and collaborative filtering
5. Matrix factorisation for implicit feedback
6. Content, hybrids and cold start
7. Two stages: candidates and ranking
8. From offline scores to the real shop
9. Run it in production
What you will be able to do
Train ALS and BPR on Larkfield’s whole shop log with the implicit library. Learn how ALS uses confidence weights for views, basket adds and orders, which settings matter, and what the two methods cost in time and memory. Then place them in the course’s comparison table.
NotebookDo this lesson in a notebook
The notebook computes the confidence of one cell, builds the shop log’s sparse matrix, and trains ALS with implicit. You train BPR yourself and compare the two by segment. Its setup cells download about 8 MB of practice data and install implicit. About 25 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- Customer and product vectors, the dot product and ALS, from Hidden tastes.
- Event weights (view 1, basket add 3, order 5), from Interactions, not ratings.
recs evaluate --by segmentandrecs compare, from Evaluate by segment.
What you need on your computer (only for the project steps)
- Your
larkfield-recsproject at the end of Module 4.python -m pytest -qprints17 passed. - The practice files in
../nextia-learning-labs, beside the project folder. - An internet connection once, to install implicit (about 1 MB).
Tested on macOS with Python 3.12. The Windows (PowerShell) commands were not run.
How sure is the shop assistant?
Section titled “How sure is the shop assistant?”Go back to the shop assistant of the last lesson. A customer tells her three things:
- “I looked at that parasol once.”
- “I bought that garden bench, and I bought a second one for my mother.”
- Nothing at all about dog beds.
She is a little sure that the customer likes the parasol. She is very sure about the bench. And the silence about dog beds? Perhaps the customer has no dog. Or perhaps she never saw the dog beds. Silence is weak evidence. It is not a “no”.
A good assistant weighs each signal by how sure she can be. This lesson gives ALS the same skill.
The problem: the hand method treats every cell the same
Section titled “The problem: the hand method treats every cell the same”In the last lesson, the tiny matrix had 1 for “bought” and 0 for “no event”. ALS tried as hard to fit every 0 as every 1. For Larkfield’s shop log, that fails in two ways.
It says the wrong thing. Before the test period, the matrix has 40,000 × 2,994 cells, and only 467,160 of them are filled: 0.39%. So 99.6% of the cells are empty. If every empty cell counts as a firm “does not like”, the method learns mostly that nobody likes anything. An order and a single view also count the same, although an order says much more.
It does not fit on a laptop. The hand method multiplied the full matrix. The project’s recs matrix measured it: the sparse matrix takes 3.9 MB, but the same matrix as a full table of numbers takes 479 MB. The hand method also makes several copies of that size. That is the wrong way to use the memory of a laptop.
The idea: confidence weights
Section titled “The idea: confidence weights”A method for this case was published in 2008 by Hu, Koren and Volinsky, under the name collaborative filtering for implicit feedback. It splits each cell into two numbers:
- The preference: 1 if the customer touched the product (any event), 0 if not. This is what the score tries to reach.
- The confidence: how much the method cares about getting this cell right. A filled cell gets alpha × its weight. An empty cell gets 1.
Alpha is a setting: how much more a filled cell counts than an empty one. The weight is the sum of the event weights in the cell: view 1, basket add 3, order 5.
A worked example
Section titled “A worked example”Ana’s three cells, with alpha = 2:
| Product | Ana’s events | Weight | Preference (target) | Confidence |
|---|---|---|---|---|
| Garden bench | view, basket add, order | 1 + 3 + 5 = 9 | 1 | 2 × 9 = 18 |
| Parasol | one view | 1 | 1 | 2 × 1 = 2 |
| Dog bed | none | 0 | 0 | 1 |
ALS still tries to give the bench a score near 1 and the dog bed a score near 0. But a miss on the bench costs 18 times more than a miss on the dog bed. So the empty cells pull the scores down only a little. The bought products pull them up strongly.
The implicit library also never builds the full table. It works with the 467,160 filled cells and one small table of 64 × 64 numbers that stands for all the empty cells together. That is why it needs megabytes, not gigabytes.
ALS, round by round
Section titled “ALS, round by round”The rounds are the same as in the last lesson. Only the “best fit” in each step now uses the confidences.
Text description of the diagram
Five boxes. Step 1: start with small random numbers in every vector. Step 2: keep the product vectors fixed; for each customer, find the vector that best fits their row. Step 3: keep the customer vectors fixed; for each product, find the vector that best fits its column. A decision: 15 rounds done? “No” goes back to step 2. “Yes” leads to the last box, with a thick border: the customer and product vectors are ready, and the score is the dot product.
Each step is easy because one side is fixed. With the products fixed, every customer is a separate small problem with an exact answer. The library solves the 40,000 customer problems one after the other, then the 2,994 product problems.
A second idea: learn the order, not the cells
Section titled “A second idea: learn the order, not the cells”A shop assistant does not need to know that a customer likes the bench “0.9”. She needs to know that the bench comes before the dog bed. A list of suggestions is an order.
BPR (Bayesian personalised ranking, published by Rendle and others in 2009) learns exactly that. It looks at pairs:
Text description of the diagram
A customer, Ana, leads to two products: one she touched, a parasol, and a random one she did not touch, a dog bed. Both lead to a question: is the score of the parasol above the score of the dog bed? “Yes, by a lot” leads to “change almost nothing”. “No, or only a little” leads to a box with a thick border: move the vectors so that the parasol goes up and the dog bed goes down.
BPR makes millions of these small steps. The size of each step is the learning rate. BPR has no alpha: every touched product counts the same, a view as much as an order.
The settings that matter
Section titled “The settings that matter”Both methods have a few settings. These are the project’s defaults (in larkfield_recs/methods/mf.py):
| Setting | ALS | BPR | What it does |
|---|---|---|---|
factors |
64 | 64 | How many numbers per vector. More numbers can hold more tastes, but cost time and can fit noise. |
regularization |
0.05 | 0.01 | The penalty for large numbers. Higher keeps the vectors simpler. |
alpha |
2.0 | (none) | How much a filled cell counts against an empty one. |
iterations |
15 | 100 | Rounds for ALS; passes over the data for BPR. |
learning_rate |
(none) | 0.05 | The size of each BPR step. |
seed |
37 | 37 | The random start, so that two runs give the same lists. |
The project also uses one thread (set LARKFIELD_THREADS to change it). With one thread, results repeat exactly, and the numeric libraries do not compete for the processor.
Guided practice: ALS and BPR in your project
Section titled “Guided practice: ALS and BPR in your project”The new method lives in one file, larkfield_recs/methods/mf.py. Its heart is short:
self.model = AlternatingLeastSquares(factors=s.get("factors", 64), regularization=s.get("regularization", 0.05), iterations=s.get("iterations", 15), alpha=s.get("alpha", 2.0), random_state=s.get("seed", 37), num_threads=_threads())self.model.fit(m, show_progress=False)m is the weighted matrix of the period, the same one that recs matrix describes. The score of a product is the dot product, as in the last lesson. A customer with no event before the period has no vector, so the method gives them the trending list.
-
In your
larkfield-recsfolder, with the virtual environment active, copy the method file:Terminal window S=../nextia-learning-labs/C37/snapshots/end-of-m05cp $S/larkfield_recs/methods/mf.py larkfield_recs/methods/Terminal window $S = "..\nextia-learning-labs\C37\snapshots\end-of-m05"Copy-Item "$S\larkfield_recs\methods\mf.py" larkfield_recs\methods\ -
List the methods. Two new lines appear:
Terminal window recs evaluate --listals matrix factorisation, alternating least squares (implicit)bpr matrix factorisation, Bayesian personalised ranking (implicit) -
Try
recs evaluate als. It fails, because implicit is not installed yet. The last line of the error is:ModuleNotFoundError: No module named 'implicit' -
Copy the new
requirements.txt. It adds one line,implicit==0.7.3. Then install:Terminal window cp $S/requirements.txt .python -m pip install -r requirements.txtTerminal window Copy-Item "$S\requirements.txt" .python -m pip install -r requirements.txtThe last line is:
Successfully installed implicit-0.7.3 threadpoolctl-3.8.0 tqdm-4.70.1 -
Train and score ALS by segment (about 5 seconds):
Terminal window recs evaluate als --by segmentMethod als (matrix factorisation, alternating least squares (implicit)), test periodCustomers scored: 2,118; trained on events before 2026-06-22recall@10 0.142 (95% interval 0.129 to 0.156), NDCG@10 0.090, MAP@10 0.066, precision@10 0.022, hit rate 18.5%coverage 33.0% of the products on sale, novelty 5.98 bits, diversity 0.66, new products in the lists 10.4%time: fit 5.6 s, suggestions 0.1 ssegment customers recall@10 NDCG@10 95% interval (recall)new 952 0.150 0.100 0.129 to 0.172occasional 556 0.120 0.075 0.097 to 0.147loyal 610 0.150 0.088 0.125 to 0.176new products: 647 orders of new products in the period; recall@10 on them 0.136Your fit time will differ. On a quiet run, our laptop took 3.8 seconds; this run shared the computer with other work.
-
Do the same for BPR:
Terminal window recs evaluate bpr --by segmentMethod bpr (matrix factorisation, Bayesian personalised ranking (implicit)), test periodCustomers scored: 2,118; trained on events before 2026-06-22recall@10 0.109 (95% interval 0.097 to 0.122), NDCG@10 0.069, MAP@10 0.050, precision@10 0.016, hit rate 14.1%coverage 76.3% of the products on sale, novelty 6.74 bits, diversity 0.49, new products in the lists 11.6%time: fit 4.6 s, suggestions 0.1 ssegment customers recall@10 NDCG@10 95% interval (recall)new 952 0.144 0.094 0.123 to 0.165occasional 556 0.092 0.059 0.070 to 0.115loyal 610 0.070 0.039 0.053 to 0.089new products: 647 orders of new products in the period; recall@10 on them 0.131
The notebook gets the same two numbers, 0.142 and 0.109, with its own 40 lines of code. Its check prints Check passed: bpr_recall is 0.109.
Read the results: the comparison table so far
Section titled “Read the results: the comparison table so far”Here are all the methods of the course so far, on the same test customers, with their 95% intervals.
Text description of the chart
One row per method, recall@10 with its 95% interval, and a vertical line at trending’s value. popular 0.106 (0.095 to 0.118). trending 0.157 (0.142 to 0.172). popular-category 0.143 (0.129 to 0.157). cooc 0.122 (0.109 to 0.136). item-cf 0.088 (0.077 to 0.101). user-user 0.145 (0.132 to 0.159). als 0.142 (0.129 to 0.156). bpr 0.109 (0.097 to 0.122).
Three things to see:
- ALS clearly beats item-cf. On the same customers,
recs compare item-cf als --by segmentprints a difference of +0.054 (95% interval +0.044 to +0.064). The gain is largest for occasional customers (+0.076) and loyal customers (+0.083). These are the customers whose few old orders gave item-cf too little to follow. - ALS does not beat trending. On Larkfield’s test period, ALS has 0.142 and trending 0.156. The paired difference is −0.014 (−0.030 to +0.002): not clear on all customers. But for new customers (−0.019, −0.036 to −0.002) and occasional customers (−0.041, −0.079 to −0.002), ALS is clearly worse. In late June, many customers buy what is selling in late June, and the 28-day trending list follows that. ALS learns tastes from 17 months, including winter.
- BPR finds fewer right products, but shows many more. Look at the second view of the same runs:
Text description of the chart
A scatter plot with coverage from 0 to 100% across and recall@10 from 0 to 0.2 up. popular at 0.3%, 0.106. trending at 0.3%, 0.157, top left. popular-category at 6.7%, 0.143. cooc at 36.2%, 0.122. als at 33.0%, 0.142. item-cf at 53.2%, 0.088. user-user at 54.2%, 0.145. bpr at 76.3%, 0.109, on the right.
BPR treats a view like an order, and it pushes every touched product above random ones. So many products that were only viewed come up, and the lists spread over 76.3% of the shop. That is good for variety and bad for recall: BPR’s recall for loyal customers is only 0.070. ALS, with its confidence weights, trusts orders more and stays closer to what people buy.
Time and memory on a laptop
Section titled “Time and memory on a laptop”The course team’s measurements, one thread, Apple M4 (training seconds and the process’s peak memory, with the data loaded):
| Method | Training | Peak memory |
|---|---|---|
popular |
0.0 s | 299 MB |
item-cf |
0.4 s | 609 MB |
user-user |
0.1 s (and 0.5 s to score) | 866 MB |
als |
3.8 s | 496 MB |
bpr |
3.9 s | 494 MB |
About 300 MB of each number is the shop log itself. ALS and BPR add about 200 MB, less than the neighbour methods. They keep about 2.8 million numbers (64 per customer and per product), not tables of neighbours.
Training takes seconds. The course assumes US$0.17 an hour for a server with 4 CPUs. Under that assumption, one ALS training costs a small fraction of a cent. Check the provider’s current page for real prices. Time and memory are not the reason to avoid matrix factorisation in a shop of this size. They matter more for a shop with millions of products, and for retraining many times, as in the next lesson.
Your turn: trust orders even more
Section titled “Your turn: trust orders even more”Change one setting: alpha. Before you run it, predict: if a filled cell counts 20 times its weight instead of 2 times, will recall go up or down?
Use the validation period, not the test period. The next lesson explains why.
recs evaluate als --period validationrecs evaluate als --set alpha=20 --period validationShow the answer
Method als (matrix factorisation, alternating least squares (implicit)), validation periodCustomers scored: 2,152; trained on events before 2026-06-08recall@10 0.155 (95% interval 0.141 to 0.169), NDCG@10 0.096, MAP@10 0.071, precision@10 0.022, hit rate 19.7%coverage 32.8% of the products on sale, novelty 5.98 bits, diversity 0.65, new products in the lists 6.6%time: fit 3.2 s, suggestions 0.1 sMethod als (matrix factorisation, alternating least squares (implicit)), validation period, settings {'alpha': 20}Customers scored: 2,152; trained on events before 2026-06-08recall@10 0.136 (95% interval 0.122 to 0.150), NDCG@10 0.081, MAP@10 0.058, precision@10 0.019, hit rate 16.8%coverage 40.0% of the products on sale, novelty 6.13 bits, diversity 0.61, new products in the lists 6.8%time: fit 3.5 s, suggestions 0.1 sRecall falls from 0.155 to 0.136. With alpha 20, the empty cells almost stop counting. The vectors then fit each customer’s few products too closely, and they say little about the products the customer has not touched yet.
We tried two more settings on the validation period. Only 3 rounds instead of 15 gave 0.156 in 0.8 seconds: here, more rounds do not help. Only 8 numbers per vector gave 0.147, and the coverage fell to 6.3%. With so few numbers, the model can hold little more than “popular in my categories”.
When to use ALS or BPR, and what they cost
Section titled “When to use ALS or BPR, and what they cost”| ALS | BPR | |
|---|---|---|
| Use it when | you have event weights you trust (orders above views) | only the order of a list matters, and you want variety |
| On Larkfield’s test period | 0.142, clearly above item-cf, below trending | 0.109, coverage 76.3% |
| Watch out | alpha and regularisation change the result; new customers get trending | a view counts like an order; weak for loyal customers here |
| Cost | 3.8 s, 496 MB | 3.9 s, 494 MB |
Neither method is better than trending on this test period. That is a real result, not a mistake. A personal method must beat a strong non-personal one to be worth its complexity. The next lesson tunes ALS, honestly, and checks again by segment.
Common mistakes
Section titled “Common mistakes”Common mistake
'No module named implicit' after you copied mf.py
Symptom: recs evaluate als stops with ModuleNotFoundError: No module named 'implicit'.
Cause: mf.py imports implicit, and the library is not installed in the active virtual environment.
Fix: copy the new requirements.txt and run python -m pip install -r requirements.txt. Check that the virtual environment is active: the prompt starts with (.venv).
Common mistake
Different numbers every run
Symptom: you changed the code to use all processor cores, and recall changes a little from one run to the next.
Cause: with several threads, the order of the small calculations changes, so the results differ in the last digits, and some lists change.
Fix: keep one thread (the default, LARKFIELD_THREADS=1) when you compare methods. Use more threads only to train faster when the exact numbers do not matter.
Knowledge check
Section titled “Knowledge check”Check your understanding
Ben viewed a lamp twice and ordered it once. With view 1, basket add 3, order 5 and alpha 2, what are the preference and the confidence of his lamp cell? And of a cell with no event?
Show the answer
The lamp cell: weight 1 + 1 + 5 = 7, preference 1, confidence 2 × 7 = 14. An empty cell: preference 0, confidence 1. ALS cares 14 times more about giving the lamp a high score than about giving an untouched product a low one.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
BPR has recall@10 0.109 and coverage 76.3%; ALS has 0.142 and 33.0%. Explain the difference from how each method learns.
Show the answer
BPR learns only an order: every touched product, viewed or ordered, should score above a random untouched one. So products that were only viewed come up too, and the lists spread over three quarters of the shop. Many of those products are not ordered, so recall falls. ALS weighs each cell by its confidence: an order counts more than a view. Its lists stay closer to what people buy, with higher recall and less variety.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás says: 'ALS is a real machine learning model, so it must beat trending.' What does the project show, and what would you tell him?
Show the answer
On Larkfield’s test period, ALS has recall@10 0.142 and trending 0.156. recs compare trending als --by segment gives −0.014 (−0.030 to +0.002). That is not clear on all customers, but ALS is clearly worse for new and occasional customers. A model is better only if the measurement shows it. In late June, the 28-day trending list follows the season, and ALS learns from 17 months of history. ALS does clearly beat item-cf (+0.054). The next step is to tune it honestly, not to assume it wins.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Implicit feedback: an empty cell is weak evidence, not a “no”. ALS gives each cell a preference (touched or not) and a confidence: alpha × its weight for a filled cell, 1 for an empty one.
- The implicit library works on the 467,160 filled cells only; the full table would be 479 MB.
- BPR learns an order: a touched product above a random untouched one. It has no alpha, so a view counts like an order.
- The settings: factors, regularisation, alpha and iterations for ALS; factors, learning rate and iterations for BPR. One thread, so the results repeat.
- On Larkfield’s test period: ALS 0.142, clearly above item-cf (+0.054), but below trending (0.156). BPR 0.109 with coverage 76.3%.
- Training takes about 4 seconds and about 500 MB of memory on a laptop.
- Your project now has
alsandbpr.
ALS has settings, and alpha 20 already showed that they change the result. Next, you tune them on the validation period and test only once.