Learn from clicks
- Lesson 22
- 40 min
- Module 6: Re-rank the top results
Where we are
In the last lesson, LambdaMART learned from 13,763 judged labels and gave the best ranking within the budget so far. Judgments are slow and expensive to make. The query log has thousands of clicks for free. In this lesson, we see why a click is not a relevance label. We measure position bias in the log and correct for it. Then we find the bias that no correction can remove.
Lesson 22 of 30 · Search and Ranking: Find the Best MatchLearn from clicks
1. The search problem
2. Measure relevance before you change anything
3. Lexical search: the baseline to beat
4. Understand the query
5. Dense and hybrid retrieval
6. Re-rank the top results
7. The parts of search that people see
8. Run search in production
What you will be able to do
Use clicks without being fooled by them. You measure position bias in Larkfield’s query log and estimate how often people look at each position. You correct the clicks with inverse propensity weighting. Then you see why the correction does not rescue a model trained on clicks.
NotebookDo this lesson in a notebook
The notebook plots clicks by position in the synthetic log. You estimate how often people look at each position, and correct two clicks by hand. Then you count how many exact products the log ever showed. Its setup cells download the practice data, so it needs no local setup. About 35 minutes.
Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.
Before you startWhat to know and to have ready
What you need to know
- Learning to rank and the run
ltr, from Learning to rank. - The synthetic query log and how it was made, from Meet the catalogue and the query log.
- How NDCG discounts lower positions, from Ranking measures.
What you need on your computer (only for the guided practice)
- The project
larkfield-searchwith this module’s files, from Cross-encoder re-ranking, and the model of Learning to rank (search ltr train).
A shop assistant who watches what people pick up
Section titled “A shop assistant who watches what people pick up”A shop has no time to ask customers what they think of each product. So the assistant watches. Every evening, she counts which products people picked up from the shelves. She plans to move the most picked-up products to the front.
Then she notices two things. The products on the table at the entrance are picked up most, whatever they are. Everyone walks past that table. And the products in the back room are never picked up at all, because nobody can see them. Her count measures where products stand as much as what people like.
The problem: judgments are expensive, and clicks are biased
Section titled “The problem: judgments are expensive, and clicks are biased”The model of the last lesson learned from 13,763 judged labels on 260 queries. Real shops rarely have that many. Writing judgments is slow, as Relevance judgments showed. The query log is free: Larkfield’s log has 20,000 searches, and a click on 45.7% of them.
Tomás says: “Clicks are relevance labels. A clicked product is a good product. Let’s train on clicks.” Look at the clicks by position first.
Text description of the chart
A falling line. Click rate by position: 1: 0.446; 2: 0.232; 3: 0.148; 4: 0.112; 5: 0.09; 6: 0.07; 7: 0.06; 8: 0.054; 9: 0.052; 10: 0.041.
The first result is clicked 44.6% of the time, the tenth 4.1%. Is the first result ten times more relevant? Partly: today’s search puts some good products first. But people also click the top result because it is at the top. They look at it first, and many never scroll down. This is position bias: the position of a result changes its clicks, whatever its quality.
The idea: a click model
Section titled “The idea: a click model”A click model describes how people look and click. The simplest one says a click needs two things:
- The person looks at the result. The chance of that depends on the position k. It is called examine(k).
- If they look, they like it enough to click. That chance depends on relevance.
So: chance of a click = examine(k) × chance of a click when looked at.
Larkfield’s query log is synthetic: our code made every click with such a model. These are the course’s assumptions, not measurements of real people:
| Part | Value |
|---|---|
| examine(k) | 1/k: 1 at position 1, 0.5 at 2, 0.25 at 4, 0.1 at 10 |
| click when looked at | 0.65 for an exact product, 0.25 partial, 0.05 irrelevant or not judged |
| add to cart after a click | 0.30 exact, 0.08 partial, 0.01 irrelevant |
A tiny example
Section titled “A tiny example”Work out three cases with these numbers:
| Product | Position | examine(k) | Click when looked at | Chance of a click |
|---|---|---|---|---|
| Exact | 1 | 1 | 0.65 | 1 × 0.65 = 0.65 |
| Exact | 4 | 0.25 | 0.65 | 0.25 × 0.65 = 0.16 |
| Partial | 1 | 1 | 0.25 | 1 × 0.25 = 0.25 |
The same exact product gets 0.65 at the top and 0.16 at position 4. And a partial product at the top (0.25) gets more clicks than the exact product at position 4. If you rank by raw clicks, the top stays the top, whatever it is.
Estimate examine(k) from the log
Section titled “Estimate examine(k) from the log”With real people, you do not know examine(k). You can estimate it. Take only the exact products: their relevance is the same at every position, so any difference in their click rate comes from position. Divide the click rate of exact products at position k by their rate at position 1.
- estimated from the log
- the click model: 1/k
Text description of the chart
Two lines that almost overlap. Estimated: 1: 1; 2: 0.52; 3: 0.335; 4: 0.247; 5: 0.204; 6: 0.161; 7: 0.142; 8: 0.131; 9: 0.117; 10: 0.098. The click model: 1: 1; 2: 0.5; 3: 0.333; 4: 0.25; 5: 0.2; 6: 0.167; 7: 0.143; 8: 0.125; 9: 0.111; 10: 0.1.
The estimate gives 1, 0.52, 0.335, 0.247 … 0.098. The rule that made the log is 1, 0.5, 0.333, 0.25 … 0.1. The method works. With a real log, it gives your own users’ curve.
Correct the clicks: inverse propensity weighting
Section titled “Correct the clicks: inverse propensity weighting”Once you know examine(k), you can correct each click. The chance that a result was looked at is called its propensity. Inverse propensity weighting (IPW) divides each click by the propensity of its position. A click at position 4 counts 1 ÷ 0.25 = 4 times; a click at position 1 counts once.
A tiny example. Two products were each shown 4 times and clicked once.
| Product | Position | Shown | Clicks | Naive label (clicks ÷ shown) | examine(k) | IPW label |
|---|---|---|---|---|---|---|
| X | 1 | 4 | 1 | 0.25 | 1 | 0.25 ÷ 1 = 0.25 |
| Y | 4 | 4 | 1 | 0.25 | 0.25 | 0.25 ÷ 0.25 = 1.00 |
The naive label says they are equal. IPW says Y is four times stronger: it got the same clicks with a quarter of the attention. In the project, clicks.click_labels makes these labels. Then ltr.click_grades turns them into grades: an IPW label of 0.5 or more becomes 2, any click 1, no click 0.
Guided practice: measure the bias and train on clicks
Section titled “Guided practice: measure the bias and train on clicks”-
Show the position bias in the log:
Terminal window search clicks bias20,000 searches, 113,636 results shown (synthetic log; click model ETA 1.0)click rate exact shown click rate of exact estimated examine(k) click model examine(k)position1 0.446 8204 0.643 1.000 1.0002 0.232 7537 0.335 0.520 0.5003 0.148 7184 0.215 0.335 0.3334 0.112 7380 0.159 0.247 0.2505 0.090 6856 0.131 0.204 0.2006 0.070 6090 0.104 0.161 0.1677 0.060 5738 0.091 0.142 0.1438 0.054 5672 0.084 0.131 0.1259 0.052 5780 0.075 0.117 0.11110 0.041 5382 0.063 0.098 0.100ETAis the power in examine(k) = 1/k^ETA. The log was made with ETA 1, so examine(k) = 1/k. -
Train the same model on clicks instead of judgments. A candidate gets the label 1 if it was ever clicked when shown for that query, and 0 otherwise:
Terminal window search ltr train --labels clicksTrained LambdaMART on 260 training queries, 26,000 candidates (595 with a label above 0), labels: clicks. Saved outputs/ltr-clicks.txtFeature importance (share of the total gain):clicks_log 82.3%carts_log 2.9%rrf 2.8%(The list goes on; the other 16 features have 2.0% or less each.) Look at the second number: 595 candidates with a label above 0. With judgments, it was 13,763.
-
Measure it:
Terminal window search eval --run ltr-clicksltr-clicks on 243 test queries: NDCG@10 0.565 (95% interval 0.520 to 0.607)recall@100 0.540, MRR 0.656, zero results 0.0%, judged in the top 10 60.1%; 20.8 ms a query (p50), 71.6 ms (p95) -
Now correct for position bias, and train again:
Terminal window search ltr train --labels clicks-ipwsearch eval --run ltr-clicks-ipwTrained LambdaMART on 260 training queries, 26,000 candidates (595 with a label above 0), labels: clicks-ipw. Saved outputs/ltr-clicks-ipw.txtFeature importance (share of the total gain):clicks_log 80.5%carts_log 3.2%rrf 2.7%ltr-clicks-ipw on 243 test queries: NDCG@10 0.565 (95% interval 0.519 to 0.608)recall@100 0.540, MRR 0.658, zero results 0.0%, judged in the top 10 60.4%; 16.6 ms a query (p50), 34.3 ms (p95)The same 0.565. The correction changed almost nothing.
Text description of the chart
Four rows. hybrid-rrf 0.621 (0.576 to 0.663); ltr on judgments 0.668 (0.628 to 0.708); ltr on clicks 0.565 (0.52 to 0.607); ltr on clicks with IPW 0.565 (0.519 to 0.608).
Trained on clicks, the model is worse than hybrid search without any learning: −0.056 (95% interval −0.079 to −0.033). IPW does not change that: also −0.056. Why?
The bigger bias: most products were never shown
Section titled “The bigger bias: most products were never shown”Look at where the clicks came from. The synthetic log was made by Larkfield’s search today: SQLite FTS5, where every word must match. That search returns nothing for 30.9% of the searches: for 94.3% of the French searches, and for every misspelling and every product code. And when it returns something, it shows only its own top 10.
So most good products were never shown for their query. A product that is never shown gets no click, however good it is. This is selection bias: the log only contains what the old system chose to show.
Text description of the diagram
A decision flow. “Exact products in the training candidates: 3,893” → “Did today’s search ever show it? (every word must match)”. “No: 3,410” → a box with a red border: “Never shown, no click is possible, a correction cannot help”. “Yes: 483” → “Did people look at it and click it? examine(k) = 1/k”. “Yes: 436” → “A click label”. “No: 47” → “Never clicked, looks irrelevant, IPW corrects the part caused by position”.
The numbers, for the 260 training queries and their 26,000 candidates (the hybrid top 100 of each):
| Grade of the candidate | Candidates | Ever shown in the log | Ever clicked |
|---|---|---|---|
| Exact | 3,893 | 483 (12.4%) | 436 |
| Partial | 9,870 | 157 | 125 |
| Irrelevant or not judged | 12,237 | 86 | 34 |
Only 12.4% of the exact candidates were ever shown. For 138 of the 260 training queries, not one candidate was ever shown. They include all 80 French, misspelt and code queries of the training set: not one of their 676 exact candidates was ever shown. For these queries, every label is 0.
IPW divides the clicks that exist by the chance of being looked at. It cannot divide a click that never happened. It corrects for products shown low; it cannot correct for products not shown at all.
What the model learned instead
Section titled “What the model learned instead”With 595 positive labels, mostly on the English queries that today’s search could answer, the model found a shortcut: clicks_log takes 82.3% of the gain. That feature counts the clicks of a product on any query in the log. So the model learned “products that were clicked before”, which is the old search’s choice, again. It is popularity, not relevance for this query. It hurts most where the old search was weakest.
The course’s planted lesson: a click model is necessary, not sufficient. You need it to read clicks correctly. You also need clicks on the right products, and those come only from showing them.
What helps in practice
Section titled “What helps in practice”- Use clicks as features, and judgments as labels. The model of the last lesson already uses
clicks_logandcarts_logas two of its 19 features. - Show new products sometimes. An exploration slot is a place on the page for a product that the log knows little about. It collects clicks on products that the old search never showed. New products and the cold start builds one.
- Log what a better search shows. Once Larkfield runs hybrid search, its log will contain the French and misspelt queries’ products. Clicks from that log will be worth more.
- Test online. Interleaving and A/B tests compare rankings with real clicks while controlling for position (Test changes online).
Your turn: where do clicks hurt most?
Section titled “Your turn: where do clicks hurt most?”Predict first. Against the model trained on judgments, does the IPW model lose more on English WANDS queries or on French ones?
search compare ltr ltr-clicks-ipw --by sourceThe output, and what it shows
ltr-clicks-ipw minus ltr on 241 test queries: ndcg@10 -0.103 (95% interval -0.126 to -0.081); better on 39, worse on 147, the same on 55
source queries ltr ltr-clicks-ipw difference low high worse code 20 1.000 1.000 0.000 0.000 0.000 0 planted 3 0.571 0.544 -0.027 -0.158 0.156 2translated 59 0.474 0.287 -0.186 -0.244 -0.130 41 typo 40 0.615 0.540 -0.075 -0.120 -0.026 28 wands 119 0.729 0.638 -0.091 -0.117 -0.067 76It loses everywhere, and most on French: −0.186. Today’s search showed nothing for 94.3% of French searches, and nothing at all for the French training queries. So the click labels taught the model nothing about French. The model trained on judgments learned to trust the French-title BM25 score; the click model has no French examples to learn that from. On the real English queries, it loses −0.091: popularity is not relevance for a specific query.
Common mistake
Ranking by click rate
Symptom: Hana asks for a quick win: “Sort each query’s results by their click rate in the log.” Tomás builds it. The products at the top stay at the top. New and better products never climb.
Cause: the click rate mixes relevance and position. In Larkfield’s synthetic log, people click a partial product at position 1 24.0% of the time. They click an exact product at position 3 only 21.5% of the time, and at position 5 only 13.1%. Sorting by raw click rate rewards the position each product already had. The page then shows the same products at the top, they collect more clicks, and the loop closes. This feedback loop is sometimes called “the rich get richer”.
Fix: never rank by raw clicks. Correct for position with a click model (IPW). Use clicks as one feature among many. Give products that were never shown a chance to be seen.
Consequence in search work: a ranking trained on its own clicks learns its own past choices. Without correction and exploration, it can never discover that it was wrong.
When to learn from clicks, and what it costs
Section titled “When to learn from clicks, and what it costs”Use clicks when you have many of them, from a search that shows a fair range of products. They are cheap and they come from real people. A click model is the first step. Estimate examine(k) from your log (exact products, or a short randomised test). Then weight each click by its inverse.
Do not use clicks as labels when the old search hid most good products, as Larkfield’s search did: 30.9% empty pages, no French, no misspellings. Then the log mostly teaches the old search’s choices. On Larkfield’s test queries, clicks as labels give 0.565 against 0.668 with judgments, with or without the correction.
Remember too that Larkfield’s clicks are synthetic: made by a click model whose numbers we chose. Real clicks are noisier. People also click on price, pictures and curiosity, and the shape of examine(k) differs from shop to shop. The method stays the same.
Recommenders meet the same problem: a recommender learns from what it showed before (a planned course on recommendation). The fixes are the same family: correct for exposure, and explore.
Knowledge check
Section titled “Knowledge check”Check your understanding
With examine(k) = 1/k, an exact product at position 5 is clicked when looked at with chance 0.65. What is its chance of a click? And an irrelevant product at position 1 (0.05)?
Show the answer
Exact at position 5: (1/5) × 0.65 = 0.13. Irrelevant at position 1: 1 × 0.05 = 0.05. Position cuts the exact product’s clicks to a fifth, but it still gets more clicks than an irrelevant product at the top. A partial product at the top (0.25) would beat it.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Product P was shown 10 times at position 2 and clicked twice. Product Q was shown 10 times at position 1 and clicked three times. Using examine(k) = 1/k, which has the higher IPW label?
Show the answer
P: (2 ÷ 10) ÷ 0.5 = 0.4. Q: (3 ÷ 10) ÷ 1 = 0.3. P has the higher IPW label: it got almost as many clicks with half the attention.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
The IPW correction recovers examine(k) almost perfectly, yet ltr-clicks-ipw scores the same 0.565 as ltr-clicks. Explain why in two sentences.
Show the answer
IPW corrects for products that were shown low. But most good products were never shown at all: only 483 of 3,893 exact training candidates, and none for the French, misspelt or code training queries. A product that was never shown has no click to correct, so the labels stay wrong whatever the weights.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- Clicks are free but biased. In Larkfield’s synthetic log, the first result is clicked 44.6% of the time and the tenth 4.1%: position bias.
- A click model splits a click into looking, examine(k), and liking. From the exact products, the log gives examine(k) = 1, 0.52, 0.335 … 0.098; the rule that made it was 1/k.
- Inverse propensity weighting divides each click by examine(k) of its position.
- Trained on clicks, LambdaMART scores 0.565 on Larkfield’s test queries, with or without IPW: worse than hybrid search (0.621) and than judgments (0.668).
- The reason is selection bias: only 483 of 3,893 exact training candidates were ever shown, none for French, misspelt or code queries. A click model is necessary, not sufficient.
This module re-ranked the top of the list. Module 7 turns to what people see first: suggestions while they type, filters, empty pages, and the business rules that Hana wants.
Next: Autocomplete