Hybrids
- Lesson 19
- 35 min
- Module 6: Content, hybrids and cold start
Where we are
Each method serves some customers and products well. We can combine them. In this lesson, we blend scores, switch by segment, and use content inside the model, and we measure what each choice costs.
Lesson 19 of 29 · Recommendation and Personalisation: Show Each Person What MattersHybrids
1. The recommendation problem
2. Data, implicit feedback and an honest split
3. Measure recommendations offline
4. Neighbours and collaborative filtering
5. Matrix factorisation for implicit feedback
6. Content, hybrids and cold start
7. Two stages: candidates and ranking
8. From offline scores to the real shop
9. Run it in production
What you will be able to do
Combine methods that are good for different customers. Blend scores after scaling them, switch by segment, or give content to a learned model. Measure each choice on the same customers, choose the weights on the validation period, and see what the winner costs.
NotebookDo this lesson in a notebook
The notebook scales and adds two scores on a tiny example, then blends content and trending for every Larkfield customer. You choose the weight of trending on the validation period and score the test period once. Its setup cells download about 9 MB of practice data. About 20 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- Content-based suggestions, from Content-based recommendations, and the cold-start grid, from Cold start for new products and new customers.
recs tuneand the rule “choose on validation, test once”, from Tune without fooling yourself.- The paired comparison with
recs compare, from Evaluate by segment.
What you need on your computer (only for the project steps)
- Your
larkfield-recsproject withmethods/content.pyandmethods/hybrid.pyfrom the last two lessons. - The practice files in
../nextia-learning-labs, beside the project folder.
Tested on macOS with Python 3.12. The Windows (PowerShell) commands were not run.
Three advisers at one table
Section titled “Three advisers at one table”A good shop assistant listens to three voices before she suggests something to a regular customer. Her memory: “she bought pots and plants before”. The labels: “this new planter looks like the pots she likes”. The till: “everybody buys garden gloves this week”.
She does not pick one voice and ignore the others. For a stranger, she mostly trusts the till. For a regular, she trusts her memory more, but she still checks what sells this week. That is a hybrid: one suggestion made from several methods.
The problem: every method fails somebody
Section titled “The problem: every method fails somebody”Look at the methods of this course so far, by segment, on the test period.
| Method | All (2,118) | New (952) | Occasional (556) | Loyal (610) |
|---|---|---|---|---|
trending |
0.156 | 0.169 | 0.162 | 0.132 |
popular-category |
0.143 | 0.118 | 0.161 | 0.164 |
als (tuned) |
0.149 | 0.154 | 0.140 | 0.150 |
content |
0.119 | 0.151 | 0.085 | 0.102 |
No single method of Modules 1 to 5 is above trending on all customers. Trending wins for new and occasional customers. For loyal customers, popular-category and ALS are better. And for new products, the last lesson showed that only content and kept places help.
The hard switch of the last lesson, hybrid-switch, gives 0.142: no better than ALS. It gives each customer one method, and throws away trending as soon as a customer has a few events. We need a better way to combine.
The idea: three ways to combine methods
Section titled “The idea: three ways to combine methods”Text description of the diagram
Three rows, each with two boxes and an arrow. Row 1, blend: every method scores every product, and the list uses a weighted sum of the scores. Row 2, switch: count the customer’s events; with 0 events use trending, with 1 to 4 content, with 5 or more ALS. Row 3, content inside the model: the content score and product details become inputs, called features, of a learned model, which Module 7 builds.
- A blend gives every product a score from each method, multiplies each score by a weight, and adds them. Every customer gets a bit of every method.
- A switch gives each customer one method, chosen by a rule such as the number of events. You measured it in the last lesson.
- Content inside the model gives a learned model the content score and the product details as inputs, beside the history. The model learns how much to trust each input. Some libraries do this inside matrix factorisation; LightFM is a known example. Its last release, from 2023, comes only as source code that you must compile, so this course does not use it. Module 7 does the same idea with a ranker. The content method is one of its candidate sources. And how well a product’s category and style match the customer’s history are among its inputs.
A blend needs scaled scores
Section titled “A blend needs scaled scores”Different methods give scores on different scales. A cosine is between −1 and 1. Larkfield’s cosines are close together, from about 0.8 to 0.95. Trending counts buyers: 0, 40, 120. If you add them as they are, the counts decide everything.
So the project scales the scores of each method for each customer. The customer’s lowest score becomes 0, and the highest becomes 1. It takes the logarithm of the trending counts first, so that one huge bestseller does not squash the rest.
A tiny example we made: one customer, four products, content and trending scores.
| Product | Content (cosine) | Trending (buyers) | Content, 0..1 | Trending, 0..1 | Blend = content + 0.5 × trending |
|---|---|---|---|---|---|
| A | 0.91 | 3 | 1.00 | 0.29 | 1.14 |
| B | 0.88 | 40 | 0.73 | 0.77 | 1.12 |
| D | 0.86 | 0 | 0.55 | 0.00 | 0.55 |
| C | 0.80 | 120 | 0.00 | 1.00 | 0.50 |
Without scaling, product C would win by far: 0.80 + 0.5 × 120 = 60.8. After scaling, A wins because the customer’s taste points to it. B is a close second, because it is both close and popular. The order now uses both voices.
Larkfield’s blend
Section titled “Larkfield’s blend”Text description of the diagram
Three scores on the left: the ALS score from history, the content score from text and the trending score from the last 28 days. Each is scaled to 0..1 for this customer. The arrows carry weights: 1 for ALS, 0.5 for content, 0.5 for trending. They meet in a box with a thick border, “Add”. The sum gives the list: the 10 products with the highest sum. A dashed arrow from a note joins the sum: for a customer with no history, ALS and content give 0, so only trending decides.
The project’s hybrid-blend adds three parts: ALS × 1, content × 0.5 and trending × 0.5. A customer with no history gets 0 from ALS and content, so trending alone orders their list. A loyal customer gets mostly ALS and content. But a product that sells well this week still gets a push.
Measure each choice
Section titled “Measure each choice”On the test period, the blend has recall@10 0.167 (0.154 to 0.182). It is the first method in the course’s comparison table with a higher number than trending (0.156). Is it really better? Compare on the same customers:
recall@10: trending 0.156, hybrid-blend 0.167difference +0.011 (95% interval -0.004 to +0.027); better for 235 customers, worse for 211, the same for 1,672The difference is not clear: the interval includes 0.coverage 0.3% -> 32.4%; novelty 5.10 -> 5.85 new 952 customers -0.010 (-0.028 to +0.006) occasional 556 customers -0.008 (-0.043 to +0.030) loyal 610 customers +0.061 (+0.029 to +0.095)Not clear on all customers. Clearly better for loyal customers, +0.061. For new and occasional customers, about the same as trending. And it shows 32.4% of the products on sale, not 0.3%. Against ALS alone, the blend is clearly better: +0.025 (+0.018 to +0.033).
Which part does the work?
Section titled “Which part does the work?”To understand the blend, remove one part at a time. These runs explain the result; they do not choose anything.
| Blend | Recall@10, test | Coverage |
|---|---|---|
| ALS × 1 + content × 0.5 + trending × 0.5 (default) | 0.167 | 32.4% |
| without content | 0.162 | 24.6% |
| without trending | 0.101 | 40.6% |
Content adds a little recall and a lot of coverage. Trending is the part that carries the blend. Without it, recall falls to 0.101, the worst result on this page. The section Common mistakes below shows why.
Choose the weights on the validation period
Section titled “Choose the weights on the validation period”The weights 1, 0.5 and 0.5 were a first guess. Weights are settings, so the rule of Module 5 applies: try them on the validation period, keep the best, and score the test period once. recs tune does this:
{"w_content": 0, "w_trending": 0.5} recall@10 0.171 NDCG@10 0.104 fit 3.1 s{"w_content": 0, "w_trending": 1} recall@10 0.189 NDCG@10 0.111 fit 2.9 s{"w_content": 0, "w_trending": 2} recall@10 0.190 NDCG@10 0.106 fit 3.0 s{"w_content": 0.5, "w_trending": 0.5} recall@10 0.172 NDCG@10 0.110 fit 3.2 s{"w_content": 0.5, "w_trending": 1} recall@10 0.188 NDCG@10 0.116 fit 3.5 s{"w_content": 0.5, "w_trending": 2} recall@10 0.196 NDCG@10 0.117 fit 3.5 s{"w_content": 1, "w_trending": 0.5} recall@10 0.173 NDCG@10 0.113 fit 3.6 s{"w_content": 1, "w_trending": 1} recall@10 0.186 NDCG@10 0.118 fit 3.5 s{"w_content": 1, "w_trending": 2} recall@10 0.199 NDCG@10 0.122 fit 3.5 sBest on validation: {"w_content": 1, "w_trending": 2} (recall@10 0.199); saved to outputs/best-hybrid-blend.jsonNow test it once: recs evaluate hybrid-blend --set w_content=1 --set w_trending=2More trending helps in every row. The best weights are ALS × 1, content × 1 and trending × 2. Scored once on the test period, this blend has recall@10 0.193 (0.178 to 0.208). Against trending, on the same customers, the difference is +0.037 (+0.025 to +0.048): clear. It is clear in every segment too: new +0.016 (+0.004 to +0.029), occasional +0.042 (+0.016 to +0.070), loyal +0.063 (+0.038 to +0.089).
Text description of the chart
Four panels with recall@10 from 0 to 0.25. All customers: trending 0.156 (0.143 to 0.172); als (tuned) 0.149 (0.135 to 0.164); content 0.119 (0.106 to 0.132); hybrid-switch 0.142 (0.129 to 0.157); hybrid-blend 0.167 (0.154 to 0.182); blend, tuned 0.193 (0.178 to 0.208). New customers: trending 0.169, als (tuned) 0.154, content 0.151, hybrid-switch 0.150, hybrid-blend 0.159, blend, tuned 0.185. Occasional customers: trending 0.162, als (tuned) 0.140, content 0.085, hybrid-switch 0.120, hybrid-blend 0.154, blend, tuned 0.204. Loyal customers: trending 0.132, als (tuned) 0.150, content 0.102, hybrid-switch 0.150, hybrid-blend 0.194, blend, tuned 0.195. In every panel, the tuned blend has the highest point, and content the lowest or close to it.
What the winner costs
Section titled “What the winner costs”A higher recall is not free. The tuned blend leans on trending, so it shows fewer different products. Coverage is 14.4% instead of 32.4%, and novelty falls from 5.85 to 5.26 bits. This is the trade-off of Beyond accuracy: more accuracy, less of the shop. It still shows far more of the shop than trending (0.3%).
And the launch? On the launch period, the default blend shows 0 of the 240 launched products (0.1% new products in the lists). A blend of history scores does not fix cold start for products. A product with no history gets no ALS score and no trending score. Its content score alone is rarely high enough. If Hana’s new range must be seen, keep the kept place of the last lesson on top of whatever blend you use.
| Choice | Recall@10, test | Better than trending? | Coverage | Launched products shown | Effort |
|---|---|---|---|---|---|
| Switch by events | 0.142 | no | 54.5% | (not measured) | one rule |
| Blend, first-guess weights | 0.167 | not clear; loyal only | 32.4% | 0 | three models |
| Blend, weights chosen on validation | 0.193 | yes, every segment | 14.4% | (not measured) | three models + tuning |
| ALS + one kept place | 0.133 on launch | no | 37.4% on launch | 186 | one rule on top |
Guided practice: blends in your project
Section titled “Guided practice: blends in your project”-
Copy the tests of this module. You copied
hybrid.pyin the last lesson:Terminal window S=../nextia-learning-labs/C37/snapshots/end-of-m06cp $S/tests/test_m06.py tests/Terminal window $S = "..\nextia-learning-labs\C37\snapshots\end-of-m06"Copy-Item "$S\tests\test_m06.py" tests\ -
Score the blend by segment:
Terminal window recs evaluate hybrid-blend --by segmentMethod hybrid-blend (ALS + 0.5 content + 0.5 trending, each scaled 0..1 per customer), test periodCustomers scored: 2,118; trained on events before 2026-06-22recall@10 0.167 (95% interval 0.154 to 0.182), NDCG@10 0.107, MAP@10 0.080, precision@10 0.026, hit rate 21.8%coverage 32.4% of the products on sale, novelty 5.85 bits, diversity 0.71, new products in the lists 12.9%time: fit 3.2 s, suggestions 0.2 ssegment customers recall@10 NDCG@10 95% interval (recall)new 952 0.159 0.107 0.137 to 0.181occasional 556 0.154 0.094 0.128 to 0.183loyal 610 0.194 0.120 0.166 to 0.224new products: 647 orders of new products in the period; recall@10 on them 0.189 -
Compare it with trending:
recs compare trending hybrid-blend --by segment. Runrecs evaluate trendingfirst if you have no saved run. You see the comparison above: +0.011, not clear. -
Tune the weights on the validation period. It takes about 40 seconds:
Terminal window recs tune hybrid-blend --grid w_content=0,0.5,1 --grid w_trending=0.5,1,2You see the nine rows above, and the best weights
{"w_content": 1, "w_trending": 2}. -
Score the chosen weights on the test period, once:
Terminal window recs evaluate hybrid-blend --set w_content=1 --set w_trending=2 --by segmentMethod hybrid-blend (ALS + 0.5 content + 0.5 trending, each scaled 0..1 per customer), test period, settings {'w_content': 1, 'w_trending': 2}Customers scored: 2,118; trained on events before 2026-06-22recall@10 0.193 (95% interval 0.178 to 0.208), NDCG@10 0.119, MAP@10 0.086, precision@10 0.029, hit rate 25.5%coverage 14.4% of the products on sale, novelty 5.26 bits, diversity 0.92, new products in the lists 20.9%time: fit 3.5 s, suggestions 0.2 ssegment customers recall@10 NDCG@10 95% interval (recall)new 952 0.185 0.118 0.161 to 0.208occasional 556 0.204 0.120 0.172 to 0.234loyal 610 0.195 0.119 0.168 to 0.224new products: 647 orders of new products in the period; recall@10 on them 0.303 -
Compare it with trending. The run’s name has the settings in square brackets, so put it in quotes:
Terminal window recs compare trending "hybrid-blend[w_content=1,w_trending=2]" --by segmentrecall@10: trending 0.156, hybrid-blend[w_content=1,w_trending=2] 0.193difference +0.037 (95% interval +0.025 to +0.048); better for 185 customers, worse for 87, the same for 1,846The difference is clear: the interval does not include 0.coverage 0.3% -> 14.4%; novelty 5.10 -> 5.26new 952 customers +0.016 (+0.004 to +0.029)occasional 556 customers +0.042 (+0.016 to +0.070)loyal 610 customers +0.063 (+0.038 to +0.089) -
Run the tests with
python -m pytest -q. You see23 passed. -
Your project now matches the snapshot
end-of-m06. Only itsREADME.mddiffers.
Your turn: past the edge of the grid
Section titled “Your turn: past the edge of the grid”The best weight of trending, 2, is the largest value in the grid. When the best value is at the edge, a larger value might be better still. Change one thing: extend the grid. Predict first: will w_trending=4 be even better on the validation period?
recs tune hybrid-blend --grid w_content=0.5,1,2 --grid w_trending=2,4Show the answer
{"w_content": 0.5, "w_trending": 2} recall@10 0.196 NDCG@10 0.117 fit 3.4 s{"w_content": 0.5, "w_trending": 4} recall@10 0.181 NDCG@10 0.099 fit 3.1 s{"w_content": 1, "w_trending": 2} recall@10 0.199 NDCG@10 0.122 fit 2.9 s{"w_content": 1, "w_trending": 4} recall@10 0.188 NDCG@10 0.105 fit 3.0 s{"w_content": 2, "w_trending": 2} recall@10 0.200 NDCG@10 0.124 fit 3.0 s{"w_content": 2, "w_trending": 4} recall@10 0.197 NDCG@10 0.115 fit 2.9 sBest on validation: {"w_content": 2, "w_trending": 2} (recall@10 0.200); saved to outputs/best-hybrid-blend.jsonNow test it once: recs evaluate hybrid-blend --set w_content=2 --set w_trending=2No. With w_trending=4, every row is worse: too much trending makes the list the same for everybody again. So 2 is near the top. w_content=2 gives 0.200 against 0.199, a difference far smaller than the interval of a recall on about 2,000 customers. Keep the simpler choice you already tested, and do not run the test period again for a difference of 0.001.
When to use a hybrid, and what it costs
Section titled “When to use a hybrid, and what it costs”Use a blend when different methods are good for different customers, and you can score every product with each one. It is the default design for a home page at this stage of the course. Choose the weights on validation, check by segment and on new products, and look at coverage before you celebrate.
Use a switch when a rule is clear and cheap, or when one method cannot run for some customers at all. The ALS fallback for customers it does not know is a small switch. At Larkfield, a switch by the number of events was not better than ALS.
Put content inside the model when you have a learned ranker that can take many inputs. It learns weights per situation that a fixed blend cannot. Module 7 builds it, and it reaches recall@10 0.234.
What it costs: a blend runs three methods, so it needs three fits (about 3 seconds here). It needs more memory: a peak of 796 MB against 489 MB for tuned ALS. It has more settings to tune, and every tuning run must stay on validation. And it is harder to explain. “You bought pots, it looks like your planter, and it sells well” is three reasons in one.
Common mistakes
Section titled “Common mistakes”Common mistake
A blend without its fallback
Symptom: you set the weight of trending to 0, to get a “purely personal” blend. Recall falls far below ALS alone:
recs evaluate hybrid-blend --set w_trending=0 --by segmentMethod hybrid-blend (ALS + 0.5 content + 0.5 trending, each scaled 0..1 per customer), test period, settings {'w_trending': 0}Customers scored: 2,118; trained on events before 2026-06-22recall@10 0.101 (95% interval 0.090 to 0.113), NDCG@10 0.066, MAP@10 0.050, precision@10 0.016, hit rate 13.2%coverage 40.6% of the products on sale, novelty 7.26 bits, diversity 0.62, new products in the lists 4.9%time: fit 3.3 s, suggestions 0.2 ssegment customers recall@10 NDCG@10 95% interval (recall)new 952 0.043 0.030 0.030 to 0.056occasional 556 0.118 0.082 0.095 to 0.144loyal 610 0.178 0.109 0.151 to 0.206new products: 647 orders of new products in the period; recall@10 on them 0.036Cause: for the 633 customers with no events, ALS and content give 0 for every product. With no trending part, all products tie at 0, and the list is an arbitrary set of products. ALS alone has a fallback for these customers; this blend does not.
Fix: keep a part in every blend that scores every customer, such as trending, or fill an empty row with the fallback list. Then check new customers by segment after every change.
Common mistake
Weights chosen on the test period
Symptom: you try weights with recs evaluate hybrid-blend --set … on the test period until one looks best, then report it.
Cause: the test period chose the weights, so it no longer measures how the blend does on new data. Its number is too optimistic.
Fix: use recs tune, which reads only the validation period, then run the test period once with the chosen weights, as in steps 4 and 5.
Knowledge check
Section titled “Knowledge check”Check your understanding
Why must a blend scale the scores of its methods before it adds them? Use Larkfield's content and trending scores in your answer.
Show the answer
The methods give scores on different scales. Larkfield’s content scores are cosines close together, about 0.8 to 0.95. Trending scores are buyer counts from 0 to more than 100. Added as they are, the counts decide the whole order, and content does nothing. Scaling each method to 0..1 for each customer, after a logarithm for the counts, puts the two on the same scale. Then the weights decide how much each method counts.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
The first-guess blend has recall@10 0.167 against 0.156 for trending, but recs compare says the difference is not clear. The tuned blend has 0.193 and a clear difference. What changed, and how do you know the tuned result is fair?
Show the answer
The weights changed: trending’s weight went from 0.5 to 2 and content’s from 0.5 to 1. That helps new and occasional customers, where trending is strong. So the blend now wins in every segment: +0.037 (+0.025 to +0.048) overall. It is fair because recs tune chose the weights on the validation period only. Then we scored the test period once, with the chosen weights. Its cost is coverage: 14.4% instead of 32.4%.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás says: 'The tuned blend beats trending for every segment, so it also solves the new range problem.' Is he right?
Show the answer
No. A blend of history scores gives a product with no history no ALS score and no trending score. On the launch period, the default blend showed 0 of the 240 launched products. Its good recall comes from known products. To show a new range, Larkfield still needs content in a kept place (one kept place showed 186 launched products), on top of the blend. Each fix serves a different cell of the cold-start grid.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A hybrid combines methods that are good for different customers. A blend adds weighted scores. A switch picks one method per customer. Content inside the model gives content to a learned model (Module 7).
- Scale each method’s scores to 0..1 for each customer before you add them; otherwise the method with the largest numbers decides alone.
- The first-guess blend (ALS + 0.5 content + 0.5 trending) has recall@10 0.167, a higher number than trending. But it is not clearly better: the paired difference is +0.011 (−0.004 to +0.027). It is clearly better only for loyal customers.
- Weights chosen on validation (1, 1, 2) give 0.193 on the test period, clearly above trending in every segment. The cost is coverage: 14.4%.
- A blend needs a part that scores customers with no history. It does not show new products: keep the kept place for that.
- Your project now matches the end of Module 6.
Every method so far scores all 2,994 products for every customer, and a blend runs three of them. Next, you split the work in two: cheap sources find a few hundred candidates, and a learned model ranks them.
Next: Why two stages