Cold start for new products and new customers
- Lesson 18
- 40 min
- Module 6: Content, hybrids and cold start
Where we are
Content gives every product a vector, even a new one. New customers are a different problem. In this lesson, we serve new products with content and new customers with popularity and context, and decide when to switch.
Lesson 18 of 29 · Recommendation and Personalisation: Show Each Person What MattersCold start for new products and new customers
1. The recommendation problem
2. Data, implicit feedback and an honest split
3. Measure recommendations offline
4. Neighbours and collaborative filtering
5. Matrix factorisation for implicit feedback
6. Content, hybrids and cold start
7. Two stages: candidates and ranking
8. From offline scores to the real shop
9. Run it in production
What you will be able to do
Serve the two kinds of newcomers: new products and new customers. Keep a place in the list for new products and measure what it costs known customers. Start new customers with popularity, test whether context helps, and decide when their own history is worth using.
NotebookDo this lesson in a notebook
The notebook replays the launch of 1 May 2026. It checks that the launched products have no history. It tests a regional trending list for new customers, and it picks each customer’s closest launched product by text. You measure what one kept place in the trending list costs known customers. Its setup cells download about 9 MB of practice data. About 20 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- Content-based suggestions and customer profiles, from Content-based recommendations.
- Segments and the paired comparison with
recs compare, from Evaluate by segment. - ALS, from ALS and BPR with implicit, and tuning on the validation period, from Tune without fooling yourself.
What you need on your computer (only for the project steps)
- Your
larkfield-recsproject withmethods/content.pyfrom the last lesson. - The practice files in
../nextia-learning-labs, beside the project folder.
Tested on macOS with Python 3.12. The Windows (PowerShell) commands were not run.
Two kinds of stranger
Section titled “Two kinds of stranger”Think of the shop assistant again. Two things are new this morning.
A stranger walks in. She knows nothing about him. She does not ask for his life story. She points to what sells well this week. Then she watches what he picks up, and after a few minutes she suggests things like it.
A new shelf of summer products arrives. Her regular customers do not know it exists. So she puts one new product on the table by the door, next to the things each regular usually buys.
These are the two kinds of cold start: a new customer and a new product. The methods to fix them are different.
The problem: four cells, one of them empty
Section titled “The problem: four cells, one of them empty”Combine the two questions: is the customer new or known, and is the product new or known? That gives four cells. Larkfield’s launch period (1 to 14 May 2026) has all four. We count, in each cell, the orders that the lists found.
Text description of the diagram
A grid of two rows and two columns. The rows are the customers: 1,177 known customers, with 1 or more orders before 1 May, and 936 new customers, with no order before. The columns are the products: known products, on sale before 1 May, and new products, launched on 1 May.
- Known customer, known product (green): history works. ALS found 12.2% of 1,876 orders. Trending found 14.1%.
- Known customer, new product (orange): content, or a kept place. ALS and trending found 0 of 361 orders. Content found 9; ALS with one kept place found 10.
- New customer, known product (green): popularity and context. Trending found 16.4% of 1,294 orders; ALS, which falls back to trending for customers it does not know, 15.1%.
- New customer, new product (red, with a thick border): the hardest cell. The four methods we measured (trending, ALS, content and ALS with one kept place) found 0 of 259 orders.
The top-left cell is where Modules 4 and 5 live. The other three cells are this lesson. The bottom-right cell is the honest bad news. A stranger buys a product that is one day old. None of the four methods we measured found a single such order.
New products: keep a place for them
Section titled “New products: keep a place for them”The last lesson showed that content can put launched products in the lists. But a content list alone loses too much for known customers. So we try a smaller change. We keep ALS for everybody. But we keep one place in each list for a new product: the new product closest to the customer’s profile, by text. The project calls this method als+new, and the number of kept places slots.
A real list, before and after
Section titled “A real list, before and after”Here is customer C100050, a loyal customer with 7 orders before 1 May. ALS knows them well: indoor plants, pots and bedside tables.
Text description of the diagram
Two columns of ten products for customer C100050 on 1 May 2026. The left column is the ALS list: 1. Black Pot Dracaena; 2. Forge Natural Concrete Succulent Pot; 3. Black Rustic Wooden Planter Box; 4. Oak Pot Chinese Money Plant; 5. Cove White Rope Hanging Planter; 6. Kensington Walnut Classic Bedside Table; 7. Navy Pot Fiddle-Leaf Fig; 8. Hollow Farm Black Barn-Door Bedside Cabinet; 9. Charcoal Pot Bird of Paradise; 10. Cyclamen Corms. The right column is the list with one kept place. Places 1 to 3 are the same. Place 4 is new and highlighted: Rustic Grey Reclaimed Wood Planter Box, a product launched that day, in a subcategory the customer bought before. Arrows show the old places 4 to 9 moving down one place each. The old place 10, Cyclamen Corms, is marked in red: it drops out.
The new planter box is a sensible suggestion: this customer buys pots and planters. The price is the cyclamen corms at place 10. If the customer would have bought them, the list now misses an order. (In this case, the customer ordered none of the eleven products.)
Omar’s limit
Section titled “Omar’s limit”Omar, the product manager, wants the new range seen. But he does not want to hurt the customers who already buy. His rule: the recall@10 of known customers may fall by at most 2% of its value. This limit is an assumption the course chose for Larkfield; your shop sets its own.
We measured the launch period with 0, 1, 2 and 3 kept places:
| Kept places | Launched products shown (of 240) | New products in the lists | Recall@10, all | Recall@10, known customers | Fall for known customers |
|---|---|---|---|---|---|
| 0 (ALS) | 0 | 0.0% | 0.137 | 0.123 | – |
| 1 | 186 | 10.0% | 0.133 | 0.121 | 1.3% |
| 2 | 221 | 20.0% | 0.125 | 0.112 | 8.6% |
| 3 | 233 | 30.0% | 0.118 | 0.107 | 13.2% |
Text description of the chart
A line chart. Across: the number of kept places, 0 to 3. Up: the fall in recall@10 of known customers, in percent, 0 to 15. The line starts at 0 with no kept place, rises to 1.3% with one, 8.6% with two and 13.2% with three. A dashed horizontal line at 2% is Omar’s limit. Only the point for one kept place is under it.
One kept place stays inside the limit: known customers lose 1.3% of their recall, and 186 of the 240 launched products get shown. Two places cost 8.6%. The cost does not grow evenly. One reason: the first kept place pushes out place 10, which rarely holds an order. The second one also pushes out place 9.
recs compare gives the same story with an interval, on the same customers: one kept place changes recall@10 by −0.004 (−0.007 to −0.001). It is a small loss, but a real one.
Where you put the kept place does not change recall@10: a list of 10 contains the product or not. It changes NDCG@10 a little: 0.0861 at place 4 and 0.0872 at place 10.
Shown is not the same as bought
Section titled “Shown is not the same as bought”Now the uncomfortable number. With one kept place, the lists found 1.7% of the 665 orders of new products. Content alone found 1.5%. Each customer gets the new product closest to their past, and people rarely buy exactly that one.
So a kept place buys exposure: customers see the new range, and the shop starts to collect views and clicks for it. After a week or two, the new products have a history, and ALS can learn them. The kept place is a bridge, not a model that guesses well.
New customers: popularity, context, first clicks
Section titled “New customers: popularity, context, first clicks”Now the left column of the grid: customers with no orders.
Start with popularity
Section titled “Start with popularity”When you know nothing about a person, what sells now is the best guess. On the test period, trending has recall@10 0.169 for new customers, better than every personal method in Modules 4 and 5. ALS gives unknown customers the trending list for this reason.
Context: test it before you trust it
Section titled “Context: test it before you trust it”Context is what you know about the visit, not about the person. Three kinds matter at Larkfield:
- The page. On a product page, the product itself is the context. “Similar products” by text or by shared customers needs no history of the visitor.
- The season. Trending follows the season already: it uses only the last 28 days. That is why it shows no winter coats in June.
- The region. Each customer has a region in
customers.csv. Maybe people in British Columbia buy different garden products than people in Quebec?
We tested the region: trending computed inside each of the six regions, against trending for everybody, for new customers.
| Period | Trending for everybody | Trending of the customer’s region | Difference (95% interval) |
|---|---|---|---|
| Validation (946 new customers) | 0.176 | 0.165 | −0.011 (−0.019 to −0.003) |
| Test (952 new customers) | 0.169 | 0.168 | −0.001 (−0.007 to +0.005) |
The regional list is a little worse. The reason is in how we made the shop log: our behaviour model gives regions no effect on taste. A regional list only splits the same orders into six smaller counts, and smaller counts are noisier. A real shop can be different: snow comes earlier in some regions. The lesson is the method, not the result: measure a context before you add it.
First clicks
Section titled “First clicks”A new customer is “new” because they have no orders. Many of them have views. Of the 2,152 customers of the validation period, 617 had no event at all. The others left something to learn from: a few views, a basket.
How fast to switch to personal lists
Section titled “How fast to switch to personal lists”When is a customer’s own history worth more than trending? We split the customers of the validation period by the number of events they had before the period (views, basket adds and orders). Then we measured three methods in each group.
Text description of the chart
A line chart with seven groups across: 0 events (617 customers), 1–2 events (67 customers), 3–4 events (104 customers), 5–9 events (207 customers), 10–19 events (287 customers), 20–49 events (405 customers), 50+ events (465 customers). Up: recall@10 from 0 to 0.25. With 0 events, all three methods give the trending list: 0.175. Trending: 0.175, 0.236, 0.141, 0.132, 0.143, 0.152, 0.123. ALS: 0.175, 0.122, 0.151, 0.126, 0.149, 0.163, 0.142. Content: 0.175, 0.129, 0.138, 0.096, 0.134, 0.117, 0.086. Content stays below trending in every group. ALS is above trending for 3–4 events and for 10 events or more, and the gap is clearest at 20–49 and 50+.
Three things stand out:
- Trending is hard to beat for light customers. With 1 or 2 events, trending has 0.236 and ALS 0.122. One view says little about taste.
- ALS wins with a real history. With 50 or more events, ALS has 0.142 against 0.123 for trending. On the test period, the same group gives 0.148 against 0.126.
- Content is not a good middle step here. It stays below trending in every group.
The groups between are noisy: some have only 67 customers, and the order of the lines changes between the validation and test periods. Do not read a precise switch point from one group.
A switch you can tune
Section titled “A switch you can tune”The project’s hybrid-switch has a simple rule. It uses trending with no history, content with fewer than min_events events, and ALS with more. We tried several values of min_events on the validation period:
min_events |
recall@10, validation |
|---|---|
| 1 | 0.155 |
| 2 | 0.155 |
| 3 | 0.155 |
| 5 (the default) | 0.154 |
| 10 | 0.151 |
| 20 | 0.149 |
| 50 | 0.141 |
Values from 1 to 5 are about the same; a longer content phase is worse. That agrees with the chart: content does not help light customers. The default, 5, is fine, and it gives 0.142 on the test period: no better than ALS alone, and below trending. A hard switch throws away the trending list as soon as a customer has a few events, although trending is still good for them. The next lesson mixes the methods instead of switching.
Guided practice: kept places in your project
Section titled “Guided practice: kept places in your project”-
Copy the hybrid methods. The file has three methods; this lesson uses
als+new:Terminal window S=../nextia-learning-labs/C37/snapshots/end-of-m06cp $S/larkfield_recs/methods/hybrid.py larkfield_recs/methods/Terminal window $S = "..\nextia-learning-labs\C37\snapshots\end-of-m06"Copy-Item "$S\larkfield_recs\methods\hybrid.py" larkfield_recs\methods\ -
Score ALS on the launch period. It takes about 10 seconds:
Terminal window recs evaluate als --period launch --by segmentMethod als (matrix factorisation, alternating least squares (implicit)), launch period (new range)Customers scored: 2,113; trained on events before 2026-05-01recall@10 0.137 (95% interval 0.123 to 0.151), NDCG@10 0.088, MAP@10 0.067, precision@10 0.020, hit rate 17.7%coverage 32.6% of the products on sale, novelty 5.87 bits, diversity 0.64, new products in the lists 0.0%time: fit 5.0 s, suggestions 0.1 ssegment customers recall@10 NDCG@10 95% interval (recall)new 936 0.155 0.104 0.132 to 0.177occasional 605 0.112 0.074 0.087 to 0.136loyal 572 0.134 0.079 0.110 to 0.161new products: 665 orders of new products in the period; recall@10 on them 0.000“New products” here are the products added in the 60 days before 1 May: the 240 launched products and a few earlier ones.
-
Keep one place for a new product:
Terminal window recs evaluate als+new --period launch --set slots=1 --by segmentMethod als+new (ALS with places for the closest new products (2 unless --set slots=N; from place 4; cold start)), launch period (new range), settings {'slots': 1}Customers scored: 2,113; trained on events before 2026-05-01recall@10 0.133 (95% interval 0.120 to 0.147), NDCG@10 0.086, MAP@10 0.065, precision@10 0.019, hit rate 17.2%coverage 37.4% of the products on sale, novelty 6.73 bits, diversity 0.67, new products in the lists 10.0%time: fit 3.3 s, suggestions 0.1 ssegment customers recall@10 NDCG@10 95% interval (recall)new 936 0.148 0.100 0.127 to 0.170occasional 605 0.113 0.074 0.089 to 0.137loyal 572 0.129 0.076 0.105 to 0.156new products: 665 orders of new products in the period; recall@10 on them 0.017The description gives the default, 2 places. Your setting,
{'slots': 1}, is at the end of the same line. -
Compare the two runs on the same customers. Put the run name in quotes, because of the square brackets:
Terminal window recs compare als "als+new[slots=1]" --period launch --by segmentrecall@10: als 0.137, als+new[slots=1] 0.133difference -0.004 (95% interval -0.007 to -0.001); better for 11 customers, worse for 23, the same for 2,079The difference is clear: the interval does not include 0.coverage 32.6% -> 37.4%; novelty 5.87 -> 6.73new 936 customers -0.006 (-0.012 to -0.002)occasional 605 customers +0.001 (-0.003 to +0.006)loyal 572 customers -0.005 (-0.011 to +0.001)Only 34 of the 2,113 customers get a different recall. The loss is mostly for new customers. Their kept product is chosen near the bestsellers, and it replaces a trending product.
Your turn: two kept places
Section titled “Your turn: two kept places”Change one setting: run the default of the method, which keeps two places. Predict first: will known customers lose more or less than 2%?
recs evaluate als+new --period launch --by segmentShow the answer
Method als+new (ALS with places for the closest new products (2 unless --set slots=N; from place 4; cold start)), launch period (new range)Customers scored: 2,113; trained on events before 2026-05-01recall@10 0.125 (95% interval 0.112 to 0.139), NDCG@10 0.083, MAP@10 0.063, precision@10 0.018, hit rate 16.4%coverage 35.6% of the products on sale, novelty 7.58 bits, diversity 0.70, new products in the lists 20.0%time: fit 3.2 s, suggestions 0.1 ssegment customers recall@10 NDCG@10 95% interval (recall)new 936 0.142 0.097 0.122 to 0.163occasional 605 0.106 0.071 0.082 to 0.129loyal 572 0.118 0.071 0.095 to 0.142new products: 665 orders of new products in the period; recall@10 on them 0.027More. Loyal customers drop from 0.134 to 0.118, and occasional customers from 0.112 to 0.106. Together, the known customers lose 8.6% of their recall: more than four times Omar’s limit. The two places show 221 launched products instead of 186, and find 2.7% of the orders of new products instead of 1.7%. For Larkfield, one place is the answer under this limit.
When to use these fixes, and what they cost
Section titled “When to use these fixes, and what they cost”| Fix | Use it for | Do not expect | Cost at Larkfield |
|---|---|---|---|
| One kept place for a new product | A new range that history methods hide | Good guesses: it finds 1.7% of the new-product orders | Known customers lose 1.3% of recall@10 |
| Trending for new customers | Customers with no events | Personal lists | Nothing to train; the same list for everybody |
| Context (page, season, region) | A real signal you have measured | Gains from a context with no effect | A regional list was a little worse here |
| Switch to history | Customers with many events | A clear switch point from a few hundred customers | ALS beats trending in both periods only at 50 events or more |
Two warnings. First, a kept place is a business decision with a measured price, not a free win. Decide the limit with the people who own the measure, as Omar does. Second, every number on this page comes from a synthetic shop: our behaviour model gives new products a small bonus and gives regions no effect. Run the same measurements on your own shop before you copy the decisions.
Common mistakes
Section titled “Common mistakes”Common mistake
zsh: no matches found
Symptom: on macOS, recs compare als als+new[slots=1] --period launch stops before recs runs:
zsh: no matches found: als+new[slots=1]Cause: zsh, the default shell on macOS, reads square brackets as a pattern for file names. No file has that name.
Fix: put the run name in quotes: recs compare als "als+new[slots=1]" --period launch. Quotes work in bash and PowerShell too.
Common mistake
A kept place that looks free on the test period
Symptom: recs evaluate als+new on the test period prints exactly the same numbers as recs evaluate als.
Cause: the method keeps places only for products added in the 30 days before the period. No product was added in the 30 days before 22 June: the summer range is 52 days old by then.
Fix: measure a cold-start method on a period that has new products: --period launch.
Knowledge check
Section titled “Knowledge check”Check your understanding
Name the four cells of the cold-start grid, and say which method serves each one at Larkfield.
Show the answer
Known customer and known product: history methods such as ALS (and trending, which is still strong). Known customer and new product: content, or one kept place for the new product closest to the customer’s taste. New customer and known product: popularity (trending), with context such as the page they are on. New customer and new product: on the launch period, none of the four methods we measured found any of its 259 orders. The only tools are exposure and learning from the first clicks. Exposure means kept places, or exploration: some places given on purpose to products the shop knows little about (Module 8).
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
One kept place shows 186 of the 240 launched products but finds only 1.7% of the orders of new products. Is the kept place a failure? Explain what it buys.
Show the answer
Not a failure, if you judge it by what it is for. It buys exposure: customers see the new range from the first day, and the shop collects views and clicks for products that had none. After a week or two, the new products have a history, and ALS can learn them. It is not a good guess of which new product each person will buy, so do not sell it as one. Its price is 1.3% of the known customers’ recall@10, inside Omar’s 2% limit.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Tomás wants to add the customer's region to the home page for new customers, because 'local lists are more relevant'. What does Larkfield's measurement say, and what should he do?
Show the answer
For new customers, trending computed per region was a little worse than trending for everybody. The difference was −0.011 (−0.019 to −0.003) on the validation period. On the test period, it was −0.001 (−0.007 to +0.005), not clear. In this synthetic shop, regions have no effect on taste, so a regional list only splits the orders into smaller, noisier counts. He should measure every context on the validation period before he adds it, by segment, and keep it only if it helps.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Cold start has two kinds: new products and new customers. The 2 × 2 grid shows which method serves which cell. For new customers buying new products, the four methods we measured found 0 of 259 orders.
- One kept place for the closest new product shows 186 of the 240 launched products. It costs known customers 1.3% of recall@10, inside Omar’s 2% limit. Two places cost 8.6%.
- Shown is not bought: one kept place finds 1.7% of the orders of new products. It buys exposure until the products have a history.
- New customers start with trending. Test context before you trust it: a regional list was a little worse here.
- ALS beats trending clearly only for customers with many events. A hard switch (
hybrid-switch) gives 0.142 on the test period, below trending. - Your project now has
methods/hybrid.py.
Each method is good for some customers and products. Next, you combine them in one list, and measure what each combination costs.
Next: Hybrids