Skip to content
Français

Hybrids

  • Lesson 19
  • 35 min
  • Module 6: Content, hybrids and cold start

Where we are

Each method serves some customers and products well. We can combine them. In this lesson, we blend scores, switch by segment, and use content inside the model, and we measure what each choice costs.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Combine methods that are good for different customers. Blend scores after scaling them, switch by segment, or give content to a learned model. Measure each choice on the same customers, choose the weights on the validation period, and see what the winner costs.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook scales and adds two scores on a tiny example, then blends content and trending for every Larkfield customer. You choose the weight of trending on the validation period and score the test period once. Its setup cells download about 9 MB of practice data. About 20 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 35 minutes

What you need to know

What you need on your computer (only for the project steps)

  • Your larkfield-recs project with methods/content.py and methods/hybrid.py from the last two lessons.
  • The practice files in ../nextia-learning-labs, beside the project folder.

Tested on macOS with Python 3.12. The Windows (PowerShell) commands were not run.

A good shop assistant listens to three voices before she suggests something to a regular customer. Her memory: “she bought pots and plants before”. The labels: “this new planter looks like the pots she likes”. The till: “everybody buys garden gloves this week”.

She does not pick one voice and ignore the others. For a stranger, she mostly trusts the till. For a regular, she trusts her memory more, but she still checks what sells this week. That is a hybrid: one suggestion made from several methods.

Look at the methods of this course so far, by segment, on the test period.

Method All (2,118) New (952) Occasional (556) Loyal (610)
trending 0.156 0.169 0.162 0.132
popular-category 0.143 0.118 0.161 0.164
als (tuned) 0.149 0.154 0.140 0.150
content 0.119 0.151 0.085 0.102

No single method of Modules 1 to 5 is above trending on all customers. Trending wins for new and occasional customers. For loyal customers, popular-category and ALS are better. And for new products, the last lesson showed that only content and kept places help.

The hard switch of the last lesson, hybrid-switch, gives 0.142: no better than ALS. It gives each customer one method, and throws away trending as soon as a customer has a few events. We need a better way to combine.

Three ways to combine methods. 1. Blend: every method scores every product, and the list uses a weighted sum of the scores. 2. Switch: count the customer's events; 0 events gives trending, 1 to 4 content, 5 or more ALS. 3. Content inside the model: the content score and product details become inputs (features) of a learned model, in Module 7.
Three ways to build a hybrid. This lesson measures the blend and the switch; Module 7 builds the third.
Text description of the diagram

Three rows, each with two boxes and an arrow. Row 1, blend: every method scores every product, and the list uses a weighted sum of the scores. Row 2, switch: count the customer’s events; with 0 events use trending, with 1 to 4 content, with 5 or more ALS. Row 3, content inside the model: the content score and product details become inputs, called features, of a learned model, which Module 7 builds.

  • A blend gives every product a score from each method, multiplies each score by a weight, and adds them. Every customer gets a bit of every method.
  • A switch gives each customer one method, chosen by a rule such as the number of events. You measured it in the last lesson.
  • Content inside the model gives a learned model the content score and the product details as inputs, beside the history. The model learns how much to trust each input. Some libraries do this inside matrix factorisation; LightFM is a known example. Its last release, from 2023, comes only as source code that you must compile, so this course does not use it. Module 7 does the same idea with a ranker. The content method is one of its candidate sources. And how well a product’s category and style match the customer’s history are among its inputs.

Different methods give scores on different scales. A cosine is between −1 and 1. Larkfield’s cosines are close together, from about 0.8 to 0.95. Trending counts buyers: 0, 40, 120. If you add them as they are, the counts decide everything.

So the project scales the scores of each method for each customer. The customer’s lowest score becomes 0, and the highest becomes 1. It takes the logarithm of the trending counts first, so that one huge bestseller does not squash the rest.

A tiny example we made: one customer, four products, content and trending scores.

Product Content (cosine) Trending (buyers) Content, 0..1 Trending, 0..1 Blend = content + 0.5 × trending
A 0.91 3 1.00 0.29 1.14
B 0.88 40 0.73 0.77 1.12
D 0.86 0 0.55 0.00 0.55
C 0.80 120 0.00 1.00 0.50

Without scaling, product C would win by far: 0.80 + 0.5 × 120 = 60.8. After scaling, A wins because the customer’s taste points to it. B is a close second, because it is both close and popular. The order now uses both voices.

The hybrid-blend method. The ALS score, the content score and the trending score are each scaled to 0..1 for the customer. They are multiplied by 1, 0.5 and 0.5 and added. The list is the 10 products with the highest sum. For a customer with no history, ALS and content give 0, so only trending decides.
The project's hybrid-blend: three scaled scores, three weights, one sum.
Text description of the diagram

Three scores on the left: the ALS score from history, the content score from text and the trending score from the last 28 days. Each is scaled to 0..1 for this customer. The arrows carry weights: 1 for ALS, 0.5 for content, 0.5 for trending. They meet in a box with a thick border, “Add”. The sum gives the list: the 10 products with the highest sum. A dashed arrow from a note joins the sum: for a customer with no history, ALS and content give 0, so only trending decides.

The project’s hybrid-blend adds three parts: ALS × 1, content × 0.5 and trending × 0.5. A customer with no history gets 0 from ALS and content, so trending alone orders their list. A loyal customer gets mostly ALS and content. But a product that sells well this week still gets a push.

On the test period, the blend has recall@10 0.167 (0.154 to 0.182). It is the first method in the course’s comparison table with a higher number than trending (0.156). Is it really better? Compare on the same customers:

recall@10: trending 0.156, hybrid-blend 0.167
difference +0.011 (95% interval -0.004 to +0.027); better for 235 customers, worse for 211, the same for 1,672
The difference is not clear: the interval includes 0.
coverage 0.3% -> 32.4%; novelty 5.10 -> 5.85
new 952 customers -0.010 (-0.028 to +0.006)
occasional 556 customers -0.008 (-0.043 to +0.030)
loyal 610 customers +0.061 (+0.029 to +0.095)

Not clear on all customers. Clearly better for loyal customers, +0.061. For new and occasional customers, about the same as trending. And it shows 32.4% of the products on sale, not 0.3%. Against ALS alone, the blend is clearly better: +0.025 (+0.018 to +0.033).

To understand the blend, remove one part at a time. These runs explain the result; they do not choose anything.

Blend Recall@10, test Coverage
ALS × 1 + content × 0.5 + trending × 0.5 (default) 0.167 32.4%
without content 0.162 24.6%
without trending 0.101 40.6%

Content adds a little recall and a lot of coverage. Trending is the part that carries the blend. Without it, recall falls to 0.101, the worst result on this page. The section Common mistakes below shows why.

Choose the weights on the validation period

Section titled “Choose the weights on the validation period”

The weights 1, 0.5 and 0.5 were a first guess. Weights are settings, so the rule of Module 5 applies: try them on the validation period, keep the best, and score the test period once. recs tune does this:

{"w_content": 0, "w_trending": 0.5} recall@10 0.171 NDCG@10 0.104 fit 3.1 s
{"w_content": 0, "w_trending": 1} recall@10 0.189 NDCG@10 0.111 fit 2.9 s
{"w_content": 0, "w_trending": 2} recall@10 0.190 NDCG@10 0.106 fit 3.0 s
{"w_content": 0.5, "w_trending": 0.5} recall@10 0.172 NDCG@10 0.110 fit 3.2 s
{"w_content": 0.5, "w_trending": 1} recall@10 0.188 NDCG@10 0.116 fit 3.5 s
{"w_content": 0.5, "w_trending": 2} recall@10 0.196 NDCG@10 0.117 fit 3.5 s
{"w_content": 1, "w_trending": 0.5} recall@10 0.173 NDCG@10 0.113 fit 3.6 s
{"w_content": 1, "w_trending": 1} recall@10 0.186 NDCG@10 0.118 fit 3.5 s
{"w_content": 1, "w_trending": 2} recall@10 0.199 NDCG@10 0.122 fit 3.5 s
Best on validation: {"w_content": 1, "w_trending": 2} (recall@10 0.199); saved to outputs/best-hybrid-blend.json
Now test it once: recs evaluate hybrid-blend --set w_content=1 --set w_trending=2

More trending helps in every row. The best weights are ALS × 1, content × 1 and trending × 2. Scored once on the test period, this blend has recall@10 0.193 (0.178 to 0.208). Against trending, on the same customers, the difference is +0.037 (+0.025 to +0.048): clear. It is clear in every segment too: new +0.016 (+0.004 to +0.029), occasional +0.042 (+0.016 to +0.070), loyal +0.063 (+0.038 to +0.089).

Recall@10 on the test period with 95% intervals, for all customers and by segment. The tuned blend is the only one above trending in every segment.
Text description of the chart

Four panels with recall@10 from 0 to 0.25. All customers: trending 0.156 (0.143 to 0.172); als (tuned) 0.149 (0.135 to 0.164); content 0.119 (0.106 to 0.132); hybrid-switch 0.142 (0.129 to 0.157); hybrid-blend 0.167 (0.154 to 0.182); blend, tuned 0.193 (0.178 to 0.208). New customers: trending 0.169, als (tuned) 0.154, content 0.151, hybrid-switch 0.150, hybrid-blend 0.159, blend, tuned 0.185. Occasional customers: trending 0.162, als (tuned) 0.140, content 0.085, hybrid-switch 0.120, hybrid-blend 0.154, blend, tuned 0.204. Loyal customers: trending 0.132, als (tuned) 0.150, content 0.102, hybrid-switch 0.150, hybrid-blend 0.194, blend, tuned 0.195. In every panel, the tuned blend has the highest point, and content the lowest or close to it.

A higher recall is not free. The tuned blend leans on trending, so it shows fewer different products. Coverage is 14.4% instead of 32.4%, and novelty falls from 5.85 to 5.26 bits. This is the trade-off of Beyond accuracy: more accuracy, less of the shop. It still shows far more of the shop than trending (0.3%).

And the launch? On the launch period, the default blend shows 0 of the 240 launched products (0.1% new products in the lists). A blend of history scores does not fix cold start for products. A product with no history gets no ALS score and no trending score. Its content score alone is rarely high enough. If Hana’s new range must be seen, keep the kept place of the last lesson on top of whatever blend you use.

Choice Recall@10, test Better than trending? Coverage Launched products shown Effort
Switch by events 0.142 no 54.5% (not measured) one rule
Blend, first-guess weights 0.167 not clear; loyal only 32.4% 0 three models
Blend, weights chosen on validation 0.193 yes, every segment 14.4% (not measured) three models + tuning
ALS + one kept place 0.133 on launch no 37.4% on launch 186 one rule on top
  1. Copy the tests of this module. You copied hybrid.py in the last lesson:

    Terminal window
    S=../nextia-learning-labs/C37/snapshots/end-of-m06
    cp $S/tests/test_m06.py tests/
  2. Score the blend by segment:

    Terminal window
    recs evaluate hybrid-blend --by segment
    Method hybrid-blend (ALS + 0.5 content + 0.5 trending, each scaled 0..1 per customer), test period
    Customers scored: 2,118; trained on events before 2026-06-22
    recall@10 0.167 (95% interval 0.154 to 0.182), NDCG@10 0.107, MAP@10 0.080, precision@10 0.026, hit rate 21.8%
    coverage 32.4% of the products on sale, novelty 5.85 bits, diversity 0.71, new products in the lists 12.9%
    time: fit 3.2 s, suggestions 0.2 s
    segment customers recall@10 NDCG@10 95% interval (recall)
    new 952 0.159 0.107 0.137 to 0.181
    occasional 556 0.154 0.094 0.128 to 0.183
    loyal 610 0.194 0.120 0.166 to 0.224
    new products: 647 orders of new products in the period; recall@10 on them 0.189
  3. Compare it with trending: recs compare trending hybrid-blend --by segment. Run recs evaluate trending first if you have no saved run. You see the comparison above: +0.011, not clear.

  4. Tune the weights on the validation period. It takes about 40 seconds:

    Terminal window
    recs tune hybrid-blend --grid w_content=0,0.5,1 --grid w_trending=0.5,1,2

    You see the nine rows above, and the best weights {"w_content": 1, "w_trending": 2}.

  5. Score the chosen weights on the test period, once:

    Terminal window
    recs evaluate hybrid-blend --set w_content=1 --set w_trending=2 --by segment
    Method hybrid-blend (ALS + 0.5 content + 0.5 trending, each scaled 0..1 per customer), test period, settings {'w_content': 1, 'w_trending': 2}
    Customers scored: 2,118; trained on events before 2026-06-22
    recall@10 0.193 (95% interval 0.178 to 0.208), NDCG@10 0.119, MAP@10 0.086, precision@10 0.029, hit rate 25.5%
    coverage 14.4% of the products on sale, novelty 5.26 bits, diversity 0.92, new products in the lists 20.9%
    time: fit 3.5 s, suggestions 0.2 s
    segment customers recall@10 NDCG@10 95% interval (recall)
    new 952 0.185 0.118 0.161 to 0.208
    occasional 556 0.204 0.120 0.172 to 0.234
    loyal 610 0.195 0.119 0.168 to 0.224
    new products: 647 orders of new products in the period; recall@10 on them 0.303
  6. Compare it with trending. The run’s name has the settings in square brackets, so put it in quotes:

    Terminal window
    recs compare trending "hybrid-blend[w_content=1,w_trending=2]" --by segment
    recall@10: trending 0.156, hybrid-blend[w_content=1,w_trending=2] 0.193
    difference +0.037 (95% interval +0.025 to +0.048); better for 185 customers, worse for 87, the same for 1,846
    The difference is clear: the interval does not include 0.
    coverage 0.3% -> 14.4%; novelty 5.10 -> 5.26
    new 952 customers +0.016 (+0.004 to +0.029)
    occasional 556 customers +0.042 (+0.016 to +0.070)
    loyal 610 customers +0.063 (+0.038 to +0.089)
  7. Run the tests with python -m pytest -q. You see 23 passed.

  8. Your project now matches the snapshot end-of-m06. Only its README.md differs.

The best weight of trending, 2, is the largest value in the grid. When the best value is at the edge, a larger value might be better still. Change one thing: extend the grid. Predict first: will w_trending=4 be even better on the validation period?

Terminal window
recs tune hybrid-blend --grid w_content=0.5,1,2 --grid w_trending=2,4
Show the answer
{"w_content": 0.5, "w_trending": 2} recall@10 0.196 NDCG@10 0.117 fit 3.4 s
{"w_content": 0.5, "w_trending": 4} recall@10 0.181 NDCG@10 0.099 fit 3.1 s
{"w_content": 1, "w_trending": 2} recall@10 0.199 NDCG@10 0.122 fit 2.9 s
{"w_content": 1, "w_trending": 4} recall@10 0.188 NDCG@10 0.105 fit 3.0 s
{"w_content": 2, "w_trending": 2} recall@10 0.200 NDCG@10 0.124 fit 3.0 s
{"w_content": 2, "w_trending": 4} recall@10 0.197 NDCG@10 0.115 fit 2.9 s
Best on validation: {"w_content": 2, "w_trending": 2} (recall@10 0.200); saved to outputs/best-hybrid-blend.json
Now test it once: recs evaluate hybrid-blend --set w_content=2 --set w_trending=2

No. With w_trending=4, every row is worse: too much trending makes the list the same for everybody again. So 2 is near the top. w_content=2 gives 0.200 against 0.199, a difference far smaller than the interval of a recall on about 2,000 customers. Keep the simpler choice you already tested, and do not run the test period again for a difference of 0.001.

Use a blend when different methods are good for different customers, and you can score every product with each one. It is the default design for a home page at this stage of the course. Choose the weights on validation, check by segment and on new products, and look at coverage before you celebrate.

Use a switch when a rule is clear and cheap, or when one method cannot run for some customers at all. The ALS fallback for customers it does not know is a small switch. At Larkfield, a switch by the number of events was not better than ALS.

Put content inside the model when you have a learned ranker that can take many inputs. It learns weights per situation that a fixed blend cannot. Module 7 builds it, and it reaches recall@10 0.234.

What it costs: a blend runs three methods, so it needs three fits (about 3 seconds here). It needs more memory: a peak of 796 MB against 489 MB for tuned ALS. It has more settings to tune, and every tuning run must stay on validation. And it is harder to explain. “You bought pots, it looks like your planter, and it sells well” is three reasons in one.

Common mistake

A blend without its fallback

Symptom: you set the weight of trending to 0, to get a “purely personal” blend. Recall falls far below ALS alone:

Terminal window
recs evaluate hybrid-blend --set w_trending=0 --by segment
Method hybrid-blend (ALS + 0.5 content + 0.5 trending, each scaled 0..1 per customer), test period, settings {'w_trending': 0}
Customers scored: 2,118; trained on events before 2026-06-22
recall@10 0.101 (95% interval 0.090 to 0.113), NDCG@10 0.066, MAP@10 0.050, precision@10 0.016, hit rate 13.2%
coverage 40.6% of the products on sale, novelty 7.26 bits, diversity 0.62, new products in the lists 4.9%
time: fit 3.3 s, suggestions 0.2 s
segment customers recall@10 NDCG@10 95% interval (recall)
new 952 0.043 0.030 0.030 to 0.056
occasional 556 0.118 0.082 0.095 to 0.144
loyal 610 0.178 0.109 0.151 to 0.206
new products: 647 orders of new products in the period; recall@10 on them 0.036

Cause: for the 633 customers with no events, ALS and content give 0 for every product. With no trending part, all products tie at 0, and the list is an arbitrary set of products. ALS alone has a fallback for these customers; this blend does not.

Fix: keep a part in every blend that scores every customer, such as trending, or fill an empty row with the fallback list. Then check new customers by segment after every change.

Common mistake

Weights chosen on the test period

Symptom: you try weights with recs evaluate hybrid-blend --set … on the test period until one looks best, then report it.

Cause: the test period chose the weights, so it no longer measures how the blend does on new data. Its number is too optimistic.

Fix: use recs tune, which reads only the validation period, then run the test period once with the chosen weights, as in steps 4 and 5.

Check your understanding

Why must a blend scale the scores of its methods before it adds them? Use Larkfield's content and trending scores in your answer.

Show the answer

The methods give scores on different scales. Larkfield’s content scores are cosines close together, about 0.8 to 0.95. Trending scores are buyer counts from 0 to more than 100. Added as they are, the counts decide the whole order, and content does nothing. Scaling each method to 0..1 for each customer, after a logarithm for the counts, puts the two on the same scale. Then the weights decide how much each method counts.

Check your understanding

The first-guess blend has recall@10 0.167 against 0.156 for trending, but recs compare says the difference is not clear. The tuned blend has 0.193 and a clear difference. What changed, and how do you know the tuned result is fair?

Show the answer

The weights changed: trending’s weight went from 0.5 to 2 and content’s from 0.5 to 1. That helps new and occasional customers, where trending is strong. So the blend now wins in every segment: +0.037 (+0.025 to +0.048) overall. It is fair because recs tune chose the weights on the validation period only. Then we scored the test period once, with the chosen weights. Its cost is coverage: 14.4% instead of 32.4%.

Check your understanding

Tomás says: 'The tuned blend beats trending for every segment, so it also solves the new range problem.' Is he right?

Show the answer

No. A blend of history scores gives a product with no history no ALS score and no trending score. On the launch period, the default blend showed 0 of the 240 launched products. Its good recall comes from known products. To show a new range, Larkfield still needs content in a kept place (one kept place showed 186 launched products), on top of the blend. Each fix serves a different cell of the cold-start grid.

Recap

  • A hybrid combines methods that are good for different customers. A blend adds weighted scores. A switch picks one method per customer. Content inside the model gives content to a learned model (Module 7).
  • Scale each method’s scores to 0..1 for each customer before you add them; otherwise the method with the largest numbers decides alone.
  • The first-guess blend (ALS + 0.5 content + 0.5 trending) has recall@10 0.167, a higher number than trending. But it is not clearly better: the paired difference is +0.011 (−0.004 to +0.027). It is clearly better only for loyal customers.
  • Weights chosen on validation (1, 1, 2) give 0.193 on the test period, clearly above trending in every segment. The cost is coverage: 14.4%.
  • A blend needs a part that scores customers with no history. It does not show new products: keep the kept place for that.
  • Your project now matches the end of Module 6.

Every method so far scores all 2,994 products for every customer, and a blend runs three of them. Next, you split the work in two: cheap sources find a few hundred candidates, and a learned model ranks them.

Next: Why two stages