Meet the shop log
- Lesson 3
- 40 min
- Module 1: The recommendation problem
Where we are
We know what success means for each page. Now we open the data. In this lesson, we set up the project, read the synthetic shop log, run the popularity baseline, and learn to reset the data. First, think of a new assistant's first morning in the shop.
Lesson 3 of 29 · Recommendation and Personalisation: Show Each Person What MattersMeet the shop log
1. The recommendation problem
2. Data, implicit feedback and an honest split
3. Measure recommendations offline
4. Neighbours and collaborative filtering
5. Matrix factorisation for implicit feedback
6. Content, hybrids and cold start
7. Two stages: candidates and ranking
8. From offline scores to the real shop
9. Run it in production
What you will be able to do
Set up Larkfield’s recommender project, read its synthetic shop log, and run the first score that every later method must beat. You cannot judge a method before you know the data under it and the score to beat.
NotebookDo this lesson in a notebook
The notebook reads the shop log, counts how often customers order and draws the long tail of products. Then you score the popular list yourself. Its setup cells install pandas, NumPy, SciPy and matplotlib and download the data (about 8 MB). So it needs no local setup. About 20 minutes. The project commands of this lesson run on your computer, not in the notebook.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- Larkfield’s three surfaces and its popular list, from A good shop assistant.
- recall@10, the business measure and the guardrails, from What success means.
- How to use a terminal, a virtual environment and
pip, from Isolate dependencies in Python for Practical AI Engineering. - How to read and change pandas code in a notebook, for the notebook route. Machine Learning: From Problem to Reliable Model uses it from its first lessons.
What you need on your computer
- Python 3.12, a terminal and VS Code. Git is useful, but a ZIP download also works.
- About 300 MB of free disk space for the project, its data and its packages. No account, no key and no money.
Tested on macOS with Python 3.12.13, NumPy 2.5.4, pandas 3.1.1 and SciPy 1.18.1. The Windows (PowerShell) commands are given but were not run. The notebook was tested in Jupyter, not in Colab or Kaggle.
A new assistant’s first morning
Section titled “A new assistant’s first morning”Think of a new assistant on her first morning in the garden shop. Before she suggests anything to anyone, she opens the sales book. Who buys here? How often? Which products sell, and in which months? She sees that most customers came only once, and that a few products sell far more than the rest.
Then she writes down the simplest rule she can think of: “Show everyone the bestsellers.” It is not clever, but now she has a number to beat. Every later idea must sell more than that rule.
This lesson is that first morning. You open Larkfield’s sales book, the shop log, and you score the simplest rule.
The project and the data
Section titled “The project and the data”The course gives you two things, in the labs repository on GitHub:
- The data, in the folder
C37/data: Larkfield’s shop log. It is synthetic: our code made it with a model of customer behaviour. It has 2,994 products, 40,000 customers and 805,316 events, from 1 February 2025 to 5 July 2026. - The project
larkfield-recs, in the folderC37/snapshots/start. Its commandrecsscores recommendation methods on the shop log, always with the same customers, period and measures. A snapshot is a copy of the project at one point of the course. Each later module has its own, so you can always start again from a clean copy.
You work in a folder larkfield-recs, with your own copy of the data inside it, in data/.
Set up the project
Section titled “Set up the project”-
Open a terminal. Make a folder for your projects, and go into it:
Terminal window mkdir -p ~/projectscd ~/projectsTerminal window New-Item -ItemType Directory -Force ~\projectscd ~\projects -
Get the files of the course. If you have Git, clone the labs repository. If you have the clone already, run
git -C nextia-learning-labs pullinstead.Terminal window git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.gitWithout Git, download the ZIP file of the repository, and unzip it in your projects folder. Its folder is then
nextia-learning-labs-main. Use that name in the next step. -
Copy the snapshot
startto a new folderlarkfield-recs. Then copy the data into it, asdata. Then go into the new folder.Terminal window cp -R nextia-learning-labs/C37/snapshots/start larkfield-recscp -R nextia-learning-labs/C37/data larkfield-recs/datacd larkfield-recsTerminal window Copy-Item -Recurse nextia-learning-labs\C37\snapshots\start larkfield-recsCopy-Item -Recurse nextia-learning-labs\C37\data larkfield-recs\datacd larkfield-recs -
Make a virtual environment, activate it, and install the packages. A virtual environment is a folder (
.venv) with its own Python packages, so this project’s versions do not mix with other projects.Terminal window python3.12 -m venv .venvsource .venv/bin/activatepip install -r requirements.txtpip install -e . --no-depsTerminal window py -3.12 -m venv .venv.venv\Scripts\Activate.ps1pip install -r requirements.txtpip install -e . --no-depsThe first
pipline installs the exact versions of NumPy, pandas, SciPy and pytest. The second line installs the project itself and makes the commandrecs. The-emeans “editable”: when you change the project’s code later,recsuses your change at once. The--no-depsmeans “install nothing else”. -
Check the setup:
Terminal window recs checkData: 12 files, all right (data)popular on the validation period (quick): recall@10 0.103 (expected 0.103)Setup check passed. -
Run the tests of the project:
Terminal window python -m pytest -qExpect
7 passed. The tests use a tiny made-up shop with 48 products, so they need no data and take less than a second.
recs check does two things. First, it compares each data file with its checksum: a short fingerprint of the file in data/SHA256SUMS. If one byte of a file is wrong, the fingerprint is different. Second, it scores the popular list on a period where the course knows the answer, 0.103. If both are right, your data and your code are the same as the course’s. The first command takes a few seconds, because it reads the whole log. It keeps a fast copy in outputs/cache/, so the next commands are quick.
In each new terminal, go to larkfield-recs and activate .venv again before you use recs.
What is in the shop log
Section titled “What is in the shop log”Run recs info:
recs infoProducts: 2,994 in 20 categories and 101 subcategories (synthetic)Customers: 40,000; 23,809 ordered at least once, 54% of them only onceEvents: 805,316 (572,465 views, 149,577 basket adds, 83,274 order lines in 57,418 orders), 2025-02-01 to 2026-07-05Periods: validation 2026-06-08 to 2026-06-21, test 2026-06-22 to 2026-07-05Customer-product pairs with any event: 488,128 of 119,760,000 (0.41% filled)The log has three files. The diagram shows how they connect:
Text description of the diagram
The diagram has three boxes. On the left, customers.csv has 40,000 rows, one for each customer, with columns such as shared_email and opted_out. In the middle, events.csv.gz has 805,316 rows. Each row is one thing that one customer did with one product: a view, a basket add or an order. On the right, products.csv has 2,994 rows, one for each product. It has the category, the price, the day it went on sale and the season. It also says if the product is in stock, sensitive or age-restricted. The column customer_id joins events to customers, and product_id joins events to products.
An event is one thing that one customer did with one product. Here are the first three events of the log, from section 1 of the notebook. The helper events() adds the column day: the days since 1 February 2025.
timestamp customer_id session_id product_id event quantity \0 2025-02-01 08:03:57 C133695 S0000001 P12071 view 01 2025-02-01 08:04:37 C133695 S0000001 P11787 view 02 2025-02-01 08:05:17 C133695 S0000001 P12808 view 0
price day0 NaN 01 NaN 02 NaN 0Customer C133695 looked at three products in one visit. A session is one visit, so the three events share S0000001. Views have no quantity and no price; only orders have them. An order line is one product in one order, so one order with three products makes three order lines.
There are no ratings in the log: what people do is the only signal. Module 2 is about this kind of data.
The last line of recs info is a first warning. Of the 40,000 × 2,994 = 119,760,000 possible pairs of a customer and a product, only 0.41% have any event. Almost everything a recommender must guess is a pair it has never seen.
The data card
Section titled “The data card”Open data/dataset-card.md in VS Code. A data card is a short document that comes with a dataset. It says where the data comes from, what one row means, its licence and its limits. Larkfield’s card says, among other things:
- The data is synthetic and has the licence CC0: you may copy and use it for anything.
splits.jsondefines three periods. Validation (8 to 21 June 2026) is for tuning. Test (22 June to 5 July 2026) is for the final report, once. Launch (1 to 14 May 2026) is the first two weeks of a new summer range of 240 products.- The practice problems we made on purpose. Winter coats sell only in winter. The new summer range has no history. A best-selling summer product is out of stock from 15 June. Some products are sensitive or age-restricted.
- Its limits: a method that finds these patterns shows that it can find patterns like them, not that it would win on a real shop. The case studies of Module 10 use real data.
Most customers buy once
Section titled “Most customers buy once”How many orders does a Larkfield customer make? The notebook counts the orders of each customer who ordered at least once:
Text description of the chart
A column chart with six columns: customers who made 1, 2, 3, 4, 5, and 6 or more orders. 1 order: 12,783 customers. 2 orders: 4,838. 3 orders: 2,296. 4 orders: 1,279. 5 orders: 773. 6 or more: 1,840. The first column is higher than the five other columns together: 53.7% of the buyers ordered only once.
Of the 23,809 customers who ordered, 53.7% ordered only once (recs info rounds it to 54%). And 40.5% of all 40,000 customers never ordered at all. So for most customers, a recommender knows one order, or only a few views. That is normal for a shop. It is also why the popular list is hard to beat. When you know almost nothing about a person, what sells to everyone is a reasonable guess.
A long tail of products
Section titled “A long tail of products”Now the products. Sort them from the bestseller to the worst, and add up their share of all order lines:
Text description of the chart
The line starts at 0 and rises very steeply at the left. The best-selling 1% of the products, 29 of 2,994, have 21.7% of all order lines. At 20% of the products, the line is at 76.8%. After that, it rises slowly and reaches 100% at about 86% of the products. The last 409 products, about 14%, were never ordered, so the line is flat there.
A few products sell a lot, and most products sell a little. This shape is called a long tail, after the long, flat right part of the curve. It matters in two ways. First, a list of the few bestsellers is right surprisingly often, because those products are a big share of all orders. Second, a recommender that shows only bestsellers never shows most of the catalogue. Coverage, from the last lesson, measures this.
Seasons: winter coats in summer
Section titled “Seasons: winter coats in summer”The shop log covers 17 months, so it has seasons. Here are the orders of winter coats by month:
Text description of the chart
The line shows the order lines of winter coats in each month, from February 2025 to July 2026. In February 2025 there are 170. They fall to almost zero from April to August 2025, with 1 to 3 a month. They rise from October 2025 (150) to a peak of 271 in January 2026, then 231 in February. In March 2026 there are 36, in April 12, in May 1, and in June and July 2026, 0. The log ends on 5 July 2026.
The coats sell in winter and almost never in summer. A count of orders over the whole log does not know that. So the duffle coat of A good shop assistant is in the popular list on 22 June. It had 857 buyers in the past, and no order in the test period.
The popularity baseline
Section titled “The popularity baseline”A baseline is the simple method that every better method must beat. Define a baseline in the machine learning course explains the idea for a model. Here, the first baseline is the poster: the method popular. The diagram shows how the project scores it:
Text description of the diagram
The method learns from every event before 22 June 2026. For each product, it counts how many different customers ordered it, and keeps the 10 products with the most customers. It shows this one list to each of the 2,118 customers who ordered between 22 June and 5 July 2026. Then the project compares each list with what that customer really ordered in those two weeks. It computes recall@10 for each customer, and takes the mean: 0.106.
The method learns only from the past, and it is scored on the next two weeks, as in real use. This is a time split. Module 2 explains why it matters so much for recommendations.
Run it on a quick sample first:
recs evaluate popular --quickMethod popular (the most ordered products over the whole training log), test periodCustomers scored: 1,000 (quick sample); trained on events before 2026-06-22recall@10 0.106 (95% interval 0.090 to 0.123), NDCG@10 0.061coverage 0.3% of the products on saletime: fit 0.0 s, suggestions 0.0 sRead it line by line:
- Customers scored: 1,000 (quick sample). With
--quick, the project scores a fixed sample of 1,000 of the 2,118 customers. It is faster, and the sample is always the same, so two quick runs compare fairly. - recall@10 0.106. On average, 10.6% of the products that a customer ordered were in the list. This is the mean that you computed by hand for three customers in What success means.
- 95% interval 0.090 to 0.123. The score depends on which customers happen to be in the test. The interval is the range where it would probably fall with other customers like these. A smaller difference between two methods may be chance.
- NDCG@10 0.061. A second measure that gives more credit to a right product near the top. Module 3 explains it.
- coverage 0.3%. The list shows 10 of the 2,994 products on sale, so 99.7% of the catalogue never appears.
Now score all 2,118 customers:
recs evaluate popularMethod popular (the most ordered products over the whole training log), test periodCustomers scored: 2,118; trained on events before 2026-06-22recall@10 0.106 (95% interval 0.095 to 0.118), NDCG@10 0.061coverage 0.3% of the products on saletime: fit 0.0 s, suggestions 0.0 sThe score is the same, and the interval is narrower: more customers, less chance. This is the first row of the course’s comparison table. Every module adds rows to it, always on the same 2,118 customers and the same two weeks.
Change one thing: random products
Section titled “Change one thing: random products”Is 0.106 good? Compare it with the floor. recs evaluate --list shows the methods of this stage:
recs evaluate --listpopular the most ordered products over the whole training lograndom random products on saleBefore you run the next command, predict: what recall@10 will 10 random products get?
recs evaluate randomMethod random (random products on sale), test periodCustomers scored: 2,118; trained on events before 2026-06-22recall@10 0.004 (95% interval 0.002 to 0.007), NDCG@10 0.002coverage 100.0% of the products on saletime: fit 0.0 s, suggestions 0.0 sRandom lists find almost nothing: 0.004. The popular list is about 26 times better. But random lists show every product to someone (coverage 100%), and the popular list shows 10. Module 3 comes back to this.
| Method | recall@10 (95% interval) | coverage |
|---|---|---|
popular |
0.106 (0.095 to 0.118) | 0.3% |
random |
0.004 (0.002 to 0.007) | 100.0% |
Guided practice: the shop log in the notebook
Section titled “Guided practice: the shop log in the notebook”The notebook does the same work in pandas, so you see how each number is computed. You can run it in Colab or Kaggle. To run it on your computer, put it in a folder of its own, not inside larkfield-recs.
- Open the notebook from the box Do this lesson in a notebook at the top of this page.
- Run the first setup cell. It prints one line per package, for example
pandas 3.1.1. - Run the second setup cell. It downloads four data files and checks their checksums. Its last line is
Ready: 4 files in larkfield-recs. - Run the helpers cell. It prints nothing.
- In section 1, run the cell. You see
805,316 events from 2025-02-01 to 2026-07-05and the first three events. - In section 2, predict the share of buyers who ordered once. Then run the cell. Its last line is
23,809 customers ordered; 54% of them only once. - In section 3, run the cell. It draws the long tail.
Your turn: score the popular list yourself
Section titled “Your turn: score the popular list yourself”In section 4 of the notebook, compute the popular list’s recall@10 on the test period without the project. Take the events before the test period, and find the 10 products that were ordered by the most customers (count each customer once). Then take the mean of recall_at_k(top10, ordered) over all customers who ordered in the test period. Then run the check cell.
Show the answer
train, truth = split("test", ev)o = train[train["event"] == "order"]top10 = o.groupby("product_id")["customer_id"].nunique().sort_values(ascending=False).index[:10].tolist()recall = np.mean([recall_at_k(top10, ordered) for ordered in truth.values()])print(round(recall, 3))0.106The check cell prints Check passed: recall is 0.106. The next cell shows the list. It is the list of A good shop assistant, with the duffle coat at place 6. nunique() counts different customers, so a customer who ordered the same gloves three times counts once.
Then change one thing in the project: the length of the list. Before you run them, predict recall@5 and recall@20.
recs evaluate popular --quick --k 5recs evaluate popular --quick --k 20Show the output
Method popular (the most ordered products over the whole training log), test periodCustomers scored: 1,000 (quick sample); trained on events before 2026-06-22recall@5 0.065 (95% interval 0.052 to 0.078), NDCG@5 0.047coverage 0.2% of the products on saletime: fit 0.0 s, suggestions 0.0 sMethod popular (the most ordered products over the whole training log), test periodCustomers scored: 1,000 (quick sample); trained on events before 2026-06-22recall@20 0.175 (95% interval 0.155 to 0.197), NDCG@20 0.080coverage 0.7% of the products on saletime: fit 0.0 s, suggestions 0.0 sA longer list catches more: 0.065 with 5 places, 0.106 with 10, 0.175 with 20. But it does not double when the list doubles, because each extra place holds a less popular product. And a surface does not have 20 good places. An email shows a few products, and most people look only at the first places of a list. So compare methods at the length that the surface really shows. This course uses 10.
Everything that recs writes goes to the folder outputs/: the fast copy of the log, and one saved file for each run. recs reset deletes outputs/ and checks the data files again:
recs resetReset: removed 9 files from outputs/; data: 12 files, all files are right.The number of files depends on the commands that you ran. Use recs reset when a result looks strange, or before a new module. It never deletes data/. If a data file is missing or changed, copy data from the labs folder again.
Common mistake
No course data found
Symptom: you open a new terminal, activate the environment, and run recs check from your projects folder. It stops at once:
No course data found. Put the course data folder next to the project as data/, or set LARKFIELD_DATA to its path.Cause: recs looks for the data in the folder where you run it: in data/, then ../data, then ../../data. In your projects folder, none of them has the shop log. The same happens if you forgot the second cp of step 3.
Fix: go into the project folder with cd larkfield-recs, and run the command again. If data/ is missing there, copy it from the labs folder, as in step 3. If you keep the data somewhere else, set the environment variable LARKFIELD_DATA to its path. A related error, command not found: recs, means that .venv is not active in this terminal. Activate it as in step 4.
Consequence in AI work: a scoring tool that silently used another copy of the data would give numbers that nobody can compare. recs stops instead, and recs check compares every file with its checksum.
Knowledge check
Section titled “Knowledge check”Check your understanding
recs info says that 54% of the customers who ordered did so only once. What does this mean for a recommender, and for the popular list?
Show the answer
For most customers, the recommender knows one order, or only views, and for new visitors nothing. Then a list of what sells to everyone is a reasonable guess. With the long tail (the best 1% of products have 21.7% of the order lines), it is right surprisingly often. So the popular list is a strong baseline, and later methods must beat it, also for new and occasional customers.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict and explain: the popular list gets recall@10 0.106 and coverage 0.3%; random products get 0.004 and 100%. Which is better for Larkfield's home page?
Show the answer
The popular list, by far, on the main measure: it finds about 26 times more of what people order. But it shows only 10 of 2,994 products, so the rest of the catalogue, including the new summer range, is never shown. A better method must keep recall high and show more of the catalogue. Coverage is a guardrail, not the main measure.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Why does recs evaluate train the popular list only on events before 22 June 2026, and score it on 22 June to 5 July?
Show the answer
Because that is how the list is used in real life: on 22 June, the shop knows only the past. If the method could count orders from the test weeks, it would know the answer, and its score would be too good. With the same split for every method, the rows of the comparison table compare fairly.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- The project
larkfield-recsand its commandrecsrun on your computer.recs checkchecks the data and a known result, andrecs resetdeletes everything that the project wrote. - The shop log is synthetic: 2,994 products, 40,000 customers and 805,316 events (views, basket adds, orders), from February 2025 to July 2026. It has no ratings. The data card explains it and its limits.
- Most customers buy once (53.7% of buyers), and a few products sell most: the best 1% have 21.7% of the order lines. This long tail makes the popular list a strong baseline.
recs evaluate popularscores recall@10 0.106 (0.095 to 0.118) with coverage 0.3%. Random products score 0.004. These are the first rows of the comparison table.
You can now run the project and read a score. But so far, the project counts only orders. The log also has 572,465 views and 149,577 basket adds, and none of them is a rating. Next, in Module 2, you decide what these events say about what a customer wants.