Skip to content
Français

Cold start for new products and new customers

  • Lesson 18
  • 40 min
  • Module 6: Content, hybrids and cold start

Where we are

Content gives every product a vector, even a new one. New customers are a different problem. In this lesson, we serve new products with content and new customers with popularity and context, and decide when to switch.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Serve the two kinds of newcomers: new products and new customers. Keep a place in the list for new products and measure what it costs known customers. Start new customers with popularity, test whether context helps, and decide when their own history is worth using.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook replays the launch of 1 May 2026. It checks that the launched products have no history. It tests a regional trending list for new customers, and it picks each customer’s closest launched product by text. You measure what one kept place in the trending list costs known customers. Its setup cells download about 9 MB of practice data. About 20 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

What you need on your computer (only for the project steps)

  • Your larkfield-recs project with methods/content.py from the last lesson.
  • The practice files in ../nextia-learning-labs, beside the project folder.

Tested on macOS with Python 3.12. The Windows (PowerShell) commands were not run.

Think of the shop assistant again. Two things are new this morning.

A stranger walks in. She knows nothing about him. She does not ask for his life story. She points to what sells well this week. Then she watches what he picks up, and after a few minutes she suggests things like it.

A new shelf of summer products arrives. Her regular customers do not know it exists. So she puts one new product on the table by the door, next to the things each regular usually buys.

These are the two kinds of cold start: a new customer and a new product. The methods to fix them are different.

The problem: four cells, one of them empty

Section titled “The problem: four cells, one of them empty”

Combine the two questions: is the customer new or known, and is the product new or known? That gives four cells. Larkfield’s launch period (1 to 14 May 2026) has all four. We count, in each cell, the orders that the lists found.

A 2 by 2 grid for the launch period, 1 to 14 May 2026. Rows: known customers (1,177, 1 or more orders before) and new customers (936, no order before). Columns: known products and new products launched on 1 May. Known customer, known product: history works; ALS found 12.2% of 1,876 orders, trending 14.1%. Known customer, new product: content or a kept place; ALS and trending found 0 of 361, content 9, one kept place 10. New customer, known product: popularity and context; trending found 16.4% of 1,294, ALS 15.1%. New customer, new product, with a thick border: the hardest cell; the four methods we measured (trending, ALS, content, ALS with a kept place) found 0 of 259.
Which method serves which customer and product, on the launch period. Each cell gives the share of its orders that the lists found.
Text description of the diagram

A grid of two rows and two columns. The rows are the customers: 1,177 known customers, with 1 or more orders before 1 May, and 936 new customers, with no order before. The columns are the products: known products, on sale before 1 May, and new products, launched on 1 May.

  • Known customer, known product (green): history works. ALS found 12.2% of 1,876 orders. Trending found 14.1%.
  • Known customer, new product (orange): content, or a kept place. ALS and trending found 0 of 361 orders. Content found 9; ALS with one kept place found 10.
  • New customer, known product (green): popularity and context. Trending found 16.4% of 1,294 orders; ALS, which falls back to trending for customers it does not know, 15.1%.
  • New customer, new product (red, with a thick border): the hardest cell. The four methods we measured (trending, ALS, content and ALS with one kept place) found 0 of 259 orders.

The top-left cell is where Modules 4 and 5 live. The other three cells are this lesson. The bottom-right cell is the honest bad news. A stranger buys a product that is one day old. None of the four methods we measured found a single such order.

The last lesson showed that content can put launched products in the lists. But a content list alone loses too much for known customers. So we try a smaller change. We keep ALS for everybody. But we keep one place in each list for a new product: the new product closest to the customer’s profile, by text. The project calls this method als+new, and the number of kept places slots.

Here is customer C100050, a loyal customer with 7 orders before 1 May. ALS knows them well: indoor plants, pots and bedside tables.

Two lists of 10 products for customer C100050 on 1 May 2026. Left, the ALS list: 1 Black Pot Dracaena, 2 Forge Natural Concrete Succulent Pot, 3 Black Rustic Wooden Planter Box, 4 Oak Pot Chinese Money Plant, 5 Cove White Rope Hanging Planter, 6 Kensington Walnut Classic Bedside Table, 7 Navy Pot Fiddle-Leaf Fig, 8 Hollow Farm Black Barn-Door Bedside Cabinet, 9 Charcoal Pot Bird of Paradise, 10 Cyclamen Corms. Right, ALS with one kept place: the same first 3 products, then at place 4 the new Rustic Grey Reclaimed Wood Planter Box, then the old places 4 to 9. The old place 10, Cyclamen Corms, drops out.
One kept place: the new product takes place 4, the rest move down, and the last product drops out.
Text description of the diagram

Two columns of ten products for customer C100050 on 1 May 2026. The left column is the ALS list: 1. Black Pot Dracaena; 2. Forge Natural Concrete Succulent Pot; 3. Black Rustic Wooden Planter Box; 4. Oak Pot Chinese Money Plant; 5. Cove White Rope Hanging Planter; 6. Kensington Walnut Classic Bedside Table; 7. Navy Pot Fiddle-Leaf Fig; 8. Hollow Farm Black Barn-Door Bedside Cabinet; 9. Charcoal Pot Bird of Paradise; 10. Cyclamen Corms. The right column is the list with one kept place. Places 1 to 3 are the same. Place 4 is new and highlighted: Rustic Grey Reclaimed Wood Planter Box, a product launched that day, in a subcategory the customer bought before. Arrows show the old places 4 to 9 moving down one place each. The old place 10, Cyclamen Corms, is marked in red: it drops out.

The new planter box is a sensible suggestion: this customer buys pots and planters. The price is the cyclamen corms at place 10. If the customer would have bought them, the list now misses an order. (In this case, the customer ordered none of the eleven products.)

Omar, the product manager, wants the new range seen. But he does not want to hurt the customers who already buy. His rule: the recall@10 of known customers may fall by at most 2% of its value. This limit is an assumption the course chose for Larkfield; your shop sets its own.

We measured the launch period with 0, 1, 2 and 3 kept places:

Kept places Launched products shown (of 240) New products in the lists Recall@10, all Recall@10, known customers Fall for known customers
0 (ALS) 0 0.0% 0.137 0.123 –
1 186 10.0% 0.133 0.121 1.3%
2 221 20.0% 0.125 0.112 8.6%
3 233 30.0% 0.118 0.107 13.2%
What kept places cost known customers on the launch period. One place stays under the 2% limit; two places do not.
Text description of the chart

A line chart. Across: the number of kept places, 0 to 3. Up: the fall in recall@10 of known customers, in percent, 0 to 15. The line starts at 0 with no kept place, rises to 1.3% with one, 8.6% with two and 13.2% with three. A dashed horizontal line at 2% is Omar’s limit. Only the point for one kept place is under it.

One kept place stays inside the limit: known customers lose 1.3% of their recall, and 186 of the 240 launched products get shown. Two places cost 8.6%. The cost does not grow evenly. One reason: the first kept place pushes out place 10, which rarely holds an order. The second one also pushes out place 9.

recs compare gives the same story with an interval, on the same customers: one kept place changes recall@10 by −0.004 (−0.007 to −0.001). It is a small loss, but a real one.

Where you put the kept place does not change recall@10: a list of 10 contains the product or not. It changes NDCG@10 a little: 0.0861 at place 4 and 0.0872 at place 10.

Now the uncomfortable number. With one kept place, the lists found 1.7% of the 665 orders of new products. Content alone found 1.5%. Each customer gets the new product closest to their past, and people rarely buy exactly that one.

So a kept place buys exposure: customers see the new range, and the shop starts to collect views and clicks for it. After a week or two, the new products have a history, and ALS can learn them. The kept place is a bridge, not a model that guesses well.

New customers: popularity, context, first clicks

Section titled “New customers: popularity, context, first clicks”

Now the left column of the grid: customers with no orders.

When you know nothing about a person, what sells now is the best guess. On the test period, trending has recall@10 0.169 for new customers, better than every personal method in Modules 4 and 5. ALS gives unknown customers the trending list for this reason.

Context is what you know about the visit, not about the person. Three kinds matter at Larkfield:

  • The page. On a product page, the product itself is the context. “Similar products” by text or by shared customers needs no history of the visitor.
  • The season. Trending follows the season already: it uses only the last 28 days. That is why it shows no winter coats in June.
  • The region. Each customer has a region in customers.csv. Maybe people in British Columbia buy different garden products than people in Quebec?

We tested the region: trending computed inside each of the six regions, against trending for everybody, for new customers.

Period Trending for everybody Trending of the customer’s region Difference (95% interval)
Validation (946 new customers) 0.176 0.165 −0.011 (−0.019 to −0.003)
Test (952 new customers) 0.169 0.168 −0.001 (−0.007 to +0.005)

The regional list is a little worse. The reason is in how we made the shop log: our behaviour model gives regions no effect on taste. A regional list only splits the same orders into six smaller counts, and smaller counts are noisier. A real shop can be different: snow comes earlier in some regions. The lesson is the method, not the result: measure a context before you add it.

A new customer is “new” because they have no orders. Many of them have views. Of the 2,152 customers of the validation period, 617 had no event at all. The others left something to learn from: a few views, a basket.

When is a customer’s own history worth more than trending? We split the customers of the validation period by the number of events they had before the period (views, basket adds and orders). Then we measured three methods in each group.

Recall@10 on the validation period by the customer's number of events before it. With no event, all three give the trending list. ALS passes trending only with many events.
Text description of the chart

A line chart with seven groups across: 0 events (617 customers), 1–2 events (67 customers), 3–4 events (104 customers), 5–9 events (207 customers), 10–19 events (287 customers), 20–49 events (405 customers), 50+ events (465 customers). Up: recall@10 from 0 to 0.25. With 0 events, all three methods give the trending list: 0.175. Trending: 0.175, 0.236, 0.141, 0.132, 0.143, 0.152, 0.123. ALS: 0.175, 0.122, 0.151, 0.126, 0.149, 0.163, 0.142. Content: 0.175, 0.129, 0.138, 0.096, 0.134, 0.117, 0.086. Content stays below trending in every group. ALS is above trending for 3–4 events and for 10 events or more, and the gap is clearest at 20–49 and 50+.

Three things stand out:

  • Trending is hard to beat for light customers. With 1 or 2 events, trending has 0.236 and ALS 0.122. One view says little about taste.
  • ALS wins with a real history. With 50 or more events, ALS has 0.142 against 0.123 for trending. On the test period, the same group gives 0.148 against 0.126.
  • Content is not a good middle step here. It stays below trending in every group.

The groups between are noisy: some have only 67 customers, and the order of the lines changes between the validation and test periods. Do not read a precise switch point from one group.

The project’s hybrid-switch has a simple rule. It uses trending with no history, content with fewer than min_events events, and ALS with more. We tried several values of min_events on the validation period:

min_events recall@10, validation
1 0.155
2 0.155
3 0.155
5 (the default) 0.154
10 0.151
20 0.149
50 0.141

Values from 1 to 5 are about the same; a longer content phase is worse. That agrees with the chart: content does not help light customers. The default, 5, is fine, and it gives 0.142 on the test period: no better than ALS alone, and below trending. A hard switch throws away the trending list as soon as a customer has a few events, although trending is still good for them. The next lesson mixes the methods instead of switching.

Guided practice: kept places in your project

Section titled “Guided practice: kept places in your project”
  1. Copy the hybrid methods. The file has three methods; this lesson uses als+new:

    Terminal window
    S=../nextia-learning-labs/C37/snapshots/end-of-m06
    cp $S/larkfield_recs/methods/hybrid.py larkfield_recs/methods/
  2. Score ALS on the launch period. It takes about 10 seconds:

    Terminal window
    recs evaluate als --period launch --by segment
    Method als (matrix factorisation, alternating least squares (implicit)), launch period (new range)
    Customers scored: 2,113; trained on events before 2026-05-01
    recall@10 0.137 (95% interval 0.123 to 0.151), NDCG@10 0.088, MAP@10 0.067, precision@10 0.020, hit rate 17.7%
    coverage 32.6% of the products on sale, novelty 5.87 bits, diversity 0.64, new products in the lists 0.0%
    time: fit 5.0 s, suggestions 0.1 s
    segment customers recall@10 NDCG@10 95% interval (recall)
    new 936 0.155 0.104 0.132 to 0.177
    occasional 605 0.112 0.074 0.087 to 0.136
    loyal 572 0.134 0.079 0.110 to 0.161
    new products: 665 orders of new products in the period; recall@10 on them 0.000

    “New products” here are the products added in the 60 days before 1 May: the 240 launched products and a few earlier ones.

  3. Keep one place for a new product:

    Terminal window
    recs evaluate als+new --period launch --set slots=1 --by segment
    Method als+new (ALS with places for the closest new products (2 unless --set slots=N; from place 4; cold start)), launch period (new range), settings {'slots': 1}
    Customers scored: 2,113; trained on events before 2026-05-01
    recall@10 0.133 (95% interval 0.120 to 0.147), NDCG@10 0.086, MAP@10 0.065, precision@10 0.019, hit rate 17.2%
    coverage 37.4% of the products on sale, novelty 6.73 bits, diversity 0.67, new products in the lists 10.0%
    time: fit 3.3 s, suggestions 0.1 s
    segment customers recall@10 NDCG@10 95% interval (recall)
    new 936 0.148 0.100 0.127 to 0.170
    occasional 605 0.113 0.074 0.089 to 0.137
    loyal 572 0.129 0.076 0.105 to 0.156
    new products: 665 orders of new products in the period; recall@10 on them 0.017

    The description gives the default, 2 places. Your setting, {'slots': 1}, is at the end of the same line.

  4. Compare the two runs on the same customers. Put the run name in quotes, because of the square brackets:

    Terminal window
    recs compare als "als+new[slots=1]" --period launch --by segment
    recall@10: als 0.137, als+new[slots=1] 0.133
    difference -0.004 (95% interval -0.007 to -0.001); better for 11 customers, worse for 23, the same for 2,079
    The difference is clear: the interval does not include 0.
    coverage 32.6% -> 37.4%; novelty 5.87 -> 6.73
    new 936 customers -0.006 (-0.012 to -0.002)
    occasional 605 customers +0.001 (-0.003 to +0.006)
    loyal 572 customers -0.005 (-0.011 to +0.001)

    Only 34 of the 2,113 customers get a different recall. The loss is mostly for new customers. Their kept product is chosen near the bestsellers, and it replaces a trending product.

Change one setting: run the default of the method, which keeps two places. Predict first: will known customers lose more or less than 2%?

Terminal window
recs evaluate als+new --period launch --by segment
Show the answer
Method als+new (ALS with places for the closest new products (2 unless --set slots=N; from place 4; cold start)), launch period (new range)
Customers scored: 2,113; trained on events before 2026-05-01
recall@10 0.125 (95% interval 0.112 to 0.139), NDCG@10 0.083, MAP@10 0.063, precision@10 0.018, hit rate 16.4%
coverage 35.6% of the products on sale, novelty 7.58 bits, diversity 0.70, new products in the lists 20.0%
time: fit 3.2 s, suggestions 0.1 s
segment customers recall@10 NDCG@10 95% interval (recall)
new 936 0.142 0.097 0.122 to 0.163
occasional 605 0.106 0.071 0.082 to 0.129
loyal 572 0.118 0.071 0.095 to 0.142
new products: 665 orders of new products in the period; recall@10 on them 0.027

More. Loyal customers drop from 0.134 to 0.118, and occasional customers from 0.112 to 0.106. Together, the known customers lose 8.6% of their recall: more than four times Omar’s limit. The two places show 221 launched products instead of 186, and find 2.7% of the orders of new products instead of 1.7%. For Larkfield, one place is the answer under this limit.

When to use these fixes, and what they cost

Section titled “When to use these fixes, and what they cost”
Fix Use it for Do not expect Cost at Larkfield
One kept place for a new product A new range that history methods hide Good guesses: it finds 1.7% of the new-product orders Known customers lose 1.3% of recall@10
Trending for new customers Customers with no events Personal lists Nothing to train; the same list for everybody
Context (page, season, region) A real signal you have measured Gains from a context with no effect A regional list was a little worse here
Switch to history Customers with many events A clear switch point from a few hundred customers ALS beats trending in both periods only at 50 events or more

Two warnings. First, a kept place is a business decision with a measured price, not a free win. Decide the limit with the people who own the measure, as Omar does. Second, every number on this page comes from a synthetic shop: our behaviour model gives new products a small bonus and gives regions no effect. Run the same measurements on your own shop before you copy the decisions.

Common mistake

zsh: no matches found

Symptom: on macOS, recs compare als als+new[slots=1] --period launch stops before recs runs:

zsh: no matches found: als+new[slots=1]

Cause: zsh, the default shell on macOS, reads square brackets as a pattern for file names. No file has that name.

Fix: put the run name in quotes: recs compare als "als+new[slots=1]" --period launch. Quotes work in bash and PowerShell too.

Common mistake

A kept place that looks free on the test period

Symptom: recs evaluate als+new on the test period prints exactly the same numbers as recs evaluate als.

Cause: the method keeps places only for products added in the 30 days before the period. No product was added in the 30 days before 22 June: the summer range is 52 days old by then.

Fix: measure a cold-start method on a period that has new products: --period launch.

Check your understanding

Name the four cells of the cold-start grid, and say which method serves each one at Larkfield.

Show the answer

Known customer and known product: history methods such as ALS (and trending, which is still strong). Known customer and new product: content, or one kept place for the new product closest to the customer’s taste. New customer and known product: popularity (trending), with context such as the page they are on. New customer and new product: on the launch period, none of the four methods we measured found any of its 259 orders. The only tools are exposure and learning from the first clicks. Exposure means kept places, or exploration: some places given on purpose to products the shop knows little about (Module 8).

Check your understanding

One kept place shows 186 of the 240 launched products but finds only 1.7% of the orders of new products. Is the kept place a failure? Explain what it buys.

Show the answer

Not a failure, if you judge it by what it is for. It buys exposure: customers see the new range from the first day, and the shop collects views and clicks for products that had none. After a week or two, the new products have a history, and ALS can learn them. It is not a good guess of which new product each person will buy, so do not sell it as one. Its price is 1.3% of the known customers’ recall@10, inside Omar’s 2% limit.

Check your understanding

Tomás wants to add the customer's region to the home page for new customers, because 'local lists are more relevant'. What does Larkfield's measurement say, and what should he do?

Show the answer

For new customers, trending computed per region was a little worse than trending for everybody. The difference was −0.011 (−0.019 to −0.003) on the validation period. On the test period, it was −0.001 (−0.007 to +0.005), not clear. In this synthetic shop, regions have no effect on taste, so a regional list only splits the orders into smaller, noisier counts. He should measure every context on the validation period before he adds it, by segment, and keep it only if it helps.

Recap

  • Cold start has two kinds: new products and new customers. The 2 × 2 grid shows which method serves which cell. For new customers buying new products, the four methods we measured found 0 of 259 orders.
  • One kept place for the closest new product shows 186 of the 240 launched products. It costs known customers 1.3% of recall@10, inside Omar’s 2% limit. Two places cost 8.6%.
  • Shown is not bought: one kept place finds 1.7% of the orders of new products. It buys exposure until the products have a history.
  • New customers start with trending. Test context before you trust it: a regional list was a little worse here.
  • ALS beats trending clearly only for customers with many events. A hard switch (hybrid-switch) gives 0.142 on the test period, below trending.
  • Your project now has methods/hybrid.py.

Each method is good for some customers and products. Next, you combine them in one list, and measure what each combination costs.

Next: Hybrids