Skip to content
Français

Learn from clicks

  • Lesson 22
  • 40 min
  • Module 6: Re-rank the top results

Where we are

In the last lesson, LambdaMART learned from 13,763 judged labels and gave the best ranking within the budget so far. Judgments are slow and expensive to make. The query log has thousands of clicks for free. In this lesson, we see why a click is not a relevance label. We measure position bias in the log and correct for it. Then we find the bias that no correction can remove.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Use clicks without being fooled by them. You measure position bias in Larkfield’s query log and estimate how often people look at each position. You correct the clicks with inverse propensity weighting. Then you see why the correction does not rescue a model trained on clicks.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook plots clicks by position in the synthetic log. You estimate how often people look at each position, and correct two clicks by hand. Then you count how many exact products the log ever showed. Its setup cells download the practice data, so it needs no local setup. About 35 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

What you need on your computer (only for the guided practice)

A shop assistant who watches what people pick up

Section titled “A shop assistant who watches what people pick up”

A shop has no time to ask customers what they think of each product. So the assistant watches. Every evening, she counts which products people picked up from the shelves. She plans to move the most picked-up products to the front.

Then she notices two things. The products on the table at the entrance are picked up most, whatever they are. Everyone walks past that table. And the products in the back room are never picked up at all, because nobody can see them. Her count measures where products stand as much as what people like.

The problem: judgments are expensive, and clicks are biased

Section titled “The problem: judgments are expensive, and clicks are biased”

The model of the last lesson learned from 13,763 judged labels on 260 queries. Real shops rarely have that many. Writing judgments is slow, as Relevance judgments showed. The query log is free: Larkfield’s log has 20,000 searches, and a click on 45.7% of them.

Tomás says: “Clicks are relevance labels. A clicked product is a good product. Let’s train on clicks.” Look at the clicks by position first.

Click rate by position in Larkfield's synthetic query log (20,000 searches, 113,636 results shown). The first result gets ten times the clicks of the tenth.
Text description of the chart

A falling line. Click rate by position: 1: 0.446; 2: 0.232; 3: 0.148; 4: 0.112; 5: 0.09; 6: 0.07; 7: 0.06; 8: 0.054; 9: 0.052; 10: 0.041.

The first result is clicked 44.6% of the time, the tenth 4.1%. Is the first result ten times more relevant? Partly: today’s search puts some good products first. But people also click the top result because it is at the top. They look at it first, and many never scroll down. This is position bias: the position of a result changes its clicks, whatever its quality.

A click model describes how people look and click. The simplest one says a click needs two things:

  1. The person looks at the result. The chance of that depends on the position k. It is called examine(k).
  2. If they look, they like it enough to click. That chance depends on relevance.

So: chance of a click = examine(k) × chance of a click when looked at.

Larkfield’s query log is synthetic: our code made every click with such a model. These are the course’s assumptions, not measurements of real people:

Part Value
examine(k) 1/k: 1 at position 1, 0.5 at 2, 0.25 at 4, 0.1 at 10
click when looked at 0.65 for an exact product, 0.25 partial, 0.05 irrelevant or not judged
add to cart after a click 0.30 exact, 0.08 partial, 0.01 irrelevant

Work out three cases with these numbers:

Product Position examine(k) Click when looked at Chance of a click
Exact 1 1 0.65 1 × 0.65 = 0.65
Exact 4 0.25 0.65 0.25 × 0.65 = 0.16
Partial 1 1 0.25 1 × 0.25 = 0.25

The same exact product gets 0.65 at the top and 0.16 at position 4. And a partial product at the top (0.25) gets more clicks than the exact product at position 4. If you rank by raw clicks, the top stays the top, whatever it is.

With real people, you do not know examine(k). You can estimate it. Take only the exact products: their relevance is the same at every position, so any difference in their click rate comes from position. Divide the click rate of exact products at position k by their rate at position 1.

  • estimated from the log
  • the click model: 1/k
examine(k) estimated from the clicks on exact products in the synthetic log, against the rule that made the log (1/k). The estimate recovers the rule from the clicks alone.
Text description of the chart

Two lines that almost overlap. Estimated: 1: 1; 2: 0.52; 3: 0.335; 4: 0.247; 5: 0.204; 6: 0.161; 7: 0.142; 8: 0.131; 9: 0.117; 10: 0.098. The click model: 1: 1; 2: 0.5; 3: 0.333; 4: 0.25; 5: 0.2; 6: 0.167; 7: 0.143; 8: 0.125; 9: 0.111; 10: 0.1.

The estimate gives 1, 0.52, 0.335, 0.247 … 0.098. The rule that made the log is 1, 0.5, 0.333, 0.25 … 0.1. The method works. With a real log, it gives your own users’ curve.

Correct the clicks: inverse propensity weighting

Section titled “Correct the clicks: inverse propensity weighting”

Once you know examine(k), you can correct each click. The chance that a result was looked at is called its propensity. Inverse propensity weighting (IPW) divides each click by the propensity of its position. A click at position 4 counts 1 ÷ 0.25 = 4 times; a click at position 1 counts once.

A tiny example. Two products were each shown 4 times and clicked once.

Product Position Shown Clicks Naive label (clicks ÷ shown) examine(k) IPW label
X 1 4 1 0.25 1 0.25 ÷ 1 = 0.25
Y 4 4 1 0.25 0.25 0.25 ÷ 0.25 = 1.00

The naive label says they are equal. IPW says Y is four times stronger: it got the same clicks with a quarter of the attention. In the project, clicks.click_labels makes these labels. Then ltr.click_grades turns them into grades: an IPW label of 0.5 or more becomes 2, any click 1, no click 0.

Guided practice: measure the bias and train on clicks

Section titled “Guided practice: measure the bias and train on clicks”
  1. Show the position bias in the log:

    Terminal window
    search clicks bias
    20,000 searches, 113,636 results shown (synthetic log; click model ETA 1.0)
    click rate exact shown click rate of exact estimated examine(k) click model examine(k)
    position
    1 0.446 8204 0.643 1.000 1.000
    2 0.232 7537 0.335 0.520 0.500
    3 0.148 7184 0.215 0.335 0.333
    4 0.112 7380 0.159 0.247 0.250
    5 0.090 6856 0.131 0.204 0.200
    6 0.070 6090 0.104 0.161 0.167
    7 0.060 5738 0.091 0.142 0.143
    8 0.054 5672 0.084 0.131 0.125
    9 0.052 5780 0.075 0.117 0.111
    10 0.041 5382 0.063 0.098 0.100

    ETA is the power in examine(k) = 1/k^ETA. The log was made with ETA 1, so examine(k) = 1/k.

  2. Train the same model on clicks instead of judgments. A candidate gets the label 1 if it was ever clicked when shown for that query, and 0 otherwise:

    Terminal window
    search ltr train --labels clicks
    Trained LambdaMART on 260 training queries, 26,000 candidates (595 with a label above 0), labels: clicks. Saved outputs/ltr-clicks.txt
    Feature importance (share of the total gain):
    clicks_log 82.3%
    carts_log 2.9%
    rrf 2.8%

    (The list goes on; the other 16 features have 2.0% or less each.) Look at the second number: 595 candidates with a label above 0. With judgments, it was 13,763.

  3. Measure it:

    Terminal window
    search eval --run ltr-clicks
    ltr-clicks on 243 test queries: NDCG@10 0.565 (95% interval 0.520 to 0.607)
    recall@100 0.540, MRR 0.656, zero results 0.0%, judged in the top 10 60.1%; 20.8 ms a query (p50), 71.6 ms (p95)
  4. Now correct for position bias, and train again:

    Terminal window
    search ltr train --labels clicks-ipw
    search eval --run ltr-clicks-ipw
    Trained LambdaMART on 260 training queries, 26,000 candidates (595 with a label above 0), labels: clicks-ipw. Saved outputs/ltr-clicks-ipw.txt
    Feature importance (share of the total gain):
    clicks_log 80.5%
    carts_log 3.2%
    rrf 2.7%
    ltr-clicks-ipw on 243 test queries: NDCG@10 0.565 (95% interval 0.519 to 0.608)
    recall@100 0.540, MRR 0.658, zero results 0.0%, judged in the top 10 60.4%; 16.6 ms a query (p50), 34.3 ms (p95)

    The same 0.565. The correction changed almost nothing.

The same LambdaMART model with three kinds of labels, on Larkfield's test queries (mean of 241 of the 243), with 95% intervals. Clicks as labels are worse than no learning at all, with or without the correction.
Text description of the chart

Four rows. hybrid-rrf 0.621 (0.576 to 0.663); ltr on judgments 0.668 (0.628 to 0.708); ltr on clicks 0.565 (0.52 to 0.607); ltr on clicks with IPW 0.565 (0.519 to 0.608).

Trained on clicks, the model is worse than hybrid search without any learning: −0.056 (95% interval −0.079 to −0.033). IPW does not change that: also −0.056. Why?

The bigger bias: most products were never shown

Section titled “The bigger bias: most products were never shown”

Look at where the clicks came from. The synthetic log was made by Larkfield’s search today: SQLite FTS5, where every word must match. That search returns nothing for 30.9% of the searches: for 94.3% of the French searches, and for every misspelling and every product code. And when it returns something, it shows only its own top 10.

So most good products were never shown for their query. A product that is never shown gets no click, however good it is. This is selection bias: the log only contains what the old system chose to show.

What the log can teach about the 3,893 exact products in the training candidates. Did today's search ever show it? No for 3,410: never shown, no click is possible, and a correction cannot help. Yes for 483. Of those, did people look at it and click it? Yes for 436: a click label. No for 47: never clicked, it looks irrelevant, and IPW corrects the part caused by position.
Two filters stand between an exact product and a click label. IPW corrects the second one, position. Nothing in the log corrects the first one: products that were never shown.
Text description of the diagram

A decision flow. “Exact products in the training candidates: 3,893” → “Did today’s search ever show it? (every word must match)”. “No: 3,410” → a box with a red border: “Never shown, no click is possible, a correction cannot help”. “Yes: 483” → “Did people look at it and click it? examine(k) = 1/k”. “Yes: 436” → “A click label”. “No: 47” → “Never clicked, looks irrelevant, IPW corrects the part caused by position”.

The numbers, for the 260 training queries and their 26,000 candidates (the hybrid top 100 of each):

Grade of the candidate Candidates Ever shown in the log Ever clicked
Exact 3,893 483 (12.4%) 436
Partial 9,870 157 125
Irrelevant or not judged 12,237 86 34

Only 12.4% of the exact candidates were ever shown. For 138 of the 260 training queries, not one candidate was ever shown. They include all 80 French, misspelt and code queries of the training set: not one of their 676 exact candidates was ever shown. For these queries, every label is 0.

IPW divides the clicks that exist by the chance of being looked at. It cannot divide a click that never happened. It corrects for products shown low; it cannot correct for products not shown at all.

With 595 positive labels, mostly on the English queries that today’s search could answer, the model found a shortcut: clicks_log takes 82.3% of the gain. That feature counts the clicks of a product on any query in the log. So the model learned “products that were clicked before”, which is the old search’s choice, again. It is popularity, not relevance for this query. It hurts most where the old search was weakest.

The course’s planted lesson: a click model is necessary, not sufficient. You need it to read clicks correctly. You also need clicks on the right products, and those come only from showing them.

  • Use clicks as features, and judgments as labels. The model of the last lesson already uses clicks_log and carts_log as two of its 19 features.
  • Show new products sometimes. An exploration slot is a place on the page for a product that the log knows little about. It collects clicks on products that the old search never showed. New products and the cold start builds one.
  • Log what a better search shows. Once Larkfield runs hybrid search, its log will contain the French and misspelt queries’ products. Clicks from that log will be worth more.
  • Test online. Interleaving and A/B tests compare rankings with real clicks while controlling for position (Test changes online).

Predict first. Against the model trained on judgments, does the IPW model lose more on English WANDS queries or on French ones?

Terminal window
search compare ltr ltr-clicks-ipw --by source
The output, and what it shows
ltr-clicks-ipw minus ltr on 241 test queries: ndcg@10 -0.103 (95% interval -0.126 to -0.081); better on 39, worse on 147, the same on 55
source queries ltr ltr-clicks-ipw difference low high worse
code 20 1.000 1.000 0.000 0.000 0.000 0
planted 3 0.571 0.544 -0.027 -0.158 0.156 2
translated 59 0.474 0.287 -0.186 -0.244 -0.130 41
typo 40 0.615 0.540 -0.075 -0.120 -0.026 28
wands 119 0.729 0.638 -0.091 -0.117 -0.067 76

It loses everywhere, and most on French: −0.186. Today’s search showed nothing for 94.3% of French searches, and nothing at all for the French training queries. So the click labels taught the model nothing about French. The model trained on judgments learned to trust the French-title BM25 score; the click model has no French examples to learn that from. On the real English queries, it loses −0.091: popularity is not relevance for a specific query.

Common mistake

Ranking by click rate

Symptom: Hana asks for a quick win: “Sort each query’s results by their click rate in the log.” Tomás builds it. The products at the top stay at the top. New and better products never climb.

Cause: the click rate mixes relevance and position. In Larkfield’s synthetic log, people click a partial product at position 1 24.0% of the time. They click an exact product at position 3 only 21.5% of the time, and at position 5 only 13.1%. Sorting by raw click rate rewards the position each product already had. The page then shows the same products at the top, they collect more clicks, and the loop closes. This feedback loop is sometimes called “the rich get richer”.

Fix: never rank by raw clicks. Correct for position with a click model (IPW). Use clicks as one feature among many. Give products that were never shown a chance to be seen.

Consequence in search work: a ranking trained on its own clicks learns its own past choices. Without correction and exploration, it can never discover that it was wrong.

When to learn from clicks, and what it costs

Section titled “When to learn from clicks, and what it costs”

Use clicks when you have many of them, from a search that shows a fair range of products. They are cheap and they come from real people. A click model is the first step. Estimate examine(k) from your log (exact products, or a short randomised test). Then weight each click by its inverse.

Do not use clicks as labels when the old search hid most good products, as Larkfield’s search did: 30.9% empty pages, no French, no misspellings. Then the log mostly teaches the old search’s choices. On Larkfield’s test queries, clicks as labels give 0.565 against 0.668 with judgments, with or without the correction.

Remember too that Larkfield’s clicks are synthetic: made by a click model whose numbers we chose. Real clicks are noisier. People also click on price, pictures and curiosity, and the shape of examine(k) differs from shop to shop. The method stays the same.

Recommenders meet the same problem: a recommender learns from what it showed before (a planned course on recommendation). The fixes are the same family: correct for exposure, and explore.

Check your understanding

With examine(k) = 1/k, an exact product at position 5 is clicked when looked at with chance 0.65. What is its chance of a click? And an irrelevant product at position 1 (0.05)?

Show the answer

Exact at position 5: (1/5) × 0.65 = 0.13. Irrelevant at position 1: 1 × 0.05 = 0.05. Position cuts the exact product’s clicks to a fifth, but it still gets more clicks than an irrelevant product at the top. A partial product at the top (0.25) would beat it.

Check your understanding

Product P was shown 10 times at position 2 and clicked twice. Product Q was shown 10 times at position 1 and clicked three times. Using examine(k) = 1/k, which has the higher IPW label?

Show the answer

P: (2 ÷ 10) ÷ 0.5 = 0.4. Q: (3 ÷ 10) ÷ 1 = 0.3. P has the higher IPW label: it got almost as many clicks with half the attention.

Check your understanding

The IPW correction recovers examine(k) almost perfectly, yet ltr-clicks-ipw scores the same 0.565 as ltr-clicks. Explain why in two sentences.

Show the answer

IPW corrects for products that were shown low. But most good products were never shown at all: only 483 of 3,893 exact training candidates, and none for the French, misspelt or code training queries. A product that was never shown has no click to correct, so the labels stay wrong whatever the weights.

Recap

  • Clicks are free but biased. In Larkfield’s synthetic log, the first result is clicked 44.6% of the time and the tenth 4.1%: position bias.
  • A click model splits a click into looking, examine(k), and liking. From the exact products, the log gives examine(k) = 1, 0.52, 0.335 … 0.098; the rule that made it was 1/k.
  • Inverse propensity weighting divides each click by examine(k) of its position.
  • Trained on clicks, LambdaMART scores 0.565 on Larkfield’s test queries, with or without IPW: worse than hybrid search (0.621) and than judgments (0.668).
  • The reason is selection bias: only 483 of 3,893 exact training candidates were ever shown, none for French, misspelt or code queries. A click model is necessary, not sufficient.

This module re-ranked the top of the list. Module 7 turns to what people see first: suggestions while they type, filters, empty pages, and the business rules that Hana wants.

Next: Autocomplete