Skip to content
Français

Search and Ranking: Find the Best Match

Make search find the best product first. Measure relevance with graded judgments, then compare BM25, query understanding, dense and hybrid retrieval, re-ranking and learning to rank on the same queries, and choose a design by relevance, speed and cost.

Read every lesson as a guest, or sign in to enroll and track your progress.

Is this the right course for you?What this course covers and does not cover, and 4 courses to take for other needs.

This course teaches ranking for people: relevance judgments and ranking measures with confidence intervals, lexical search with an inverted index and BM25, query understanding in English and French, dense and hybrid retrieval, re-ranking with a cross-encoder, learning to rank from judgments and clicks, autocomplete, facets, zero results and business rules, and latency, cost and online tests. It does not teach retrieval as a step before a language model, gradient boosting from the start, or how to operate a search server.

Take another course if you want to:

What you'll learn

By the end of this course, you will be able to:

  • Frame a search problem: who searches, for what, the cost of a bad result, a zero-result page and a slow page, and the measures that match those costs.
  • Build a relevance test set with graded judgments, and measure ranking with recall@k, MRR and NDCG, with confidence intervals.
  • Build lexical, dense and hybrid retrieval, and improve queries with normalisation, spelling correction, synonyms and stemming in English and French.
  • Re-rank candidates with a cross-encoder and with a learning-to-rank model trained on judgments and click features, and explain what each adds.
  • Handle the parts of search that people see: autocomplete, filters and facets, zero-result queries and business rules, without hiding relevance problems.
  • Choose a design by relevance, latency and cost per query, and test a change online with click logs, knowing the biases in clicks.

About this course

Larkfield, an online shop for home and garden products, has a search box that many people use and few people like. Someone types “grey outdor cushion” and sees an empty page. Someone types “sofa” and sees sofa covers first. A French customer types “coussin gris” and gets nothing useful. Each failed search is a lost basket or a message to support.

In this course, you follow Priya, Larkfield’s data scientist, Amira, the developer who owns the search service, and Omar, the product manager, as they fix it. You start by measuring: you use graded relevance judgments, compute NDCG and recall by hand, and build a test set with common and rare queries and confidence intervals. Then you change one thing at a time and measure it, on the same queries, in one comparison table.

You go from a classic inverted index and BM25 to query understanding in English and French, search by meaning with embeddings, hybrid search, re-ranking with a cross-encoder and a learning-to-rank model with LightGBM. You learn why clicks are not relevance labels, and how a click model corrects for position.

You build one project through the course: larkfield-search, a small search engine with a search command and a ready-made search page. You finish with the parts people see, such as autocomplete, facets and business rules, and with the production work: latency and cost per thousand queries, and online tests. The catalogue and the judgments come from a real, open furniture dataset; the query log and the clicks are synthetic, made by our code.

Before you start

You completed Machine Learning, From Problem to Reliable Model, or you have the same skills (train, validation and test sets, gradient boosting, metrics). You can read and change Python and pandas code, use a notebook and a terminal. The RAG and mathematics courses help but are not required; lessons that use their ideas explain them briefly and link to them. You need Python 3.12 on your computer, or a free Google or Kaggle account for the notebooks. You need no key, no graphics card and no money.

Courses to take first: Machine Learning: From Problem to Reliable Model.

Who this is for

AI engineers, backend developers and data scientists who must make search work for people, such as product search in a shop, help-centre search or internal search, and who want to change it by measurement, not by feel.

How we made this course

Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses

Inside the course

Module 3Lexical search: the baseline to beat

3 lessons

Lexical search, the baseline to beat. An inverted index by hand and in SQLite, BM25 with each part of the score visible and field weights, and exact-match rules for codes and numbers.

  1. The inverted index35 min
  2. BM2540 min
  3. Exact matches and rules30 min

Final module · choose at least oneCase studies

3 case studies

Three end-to-end case studies on real open data. Product search for a furniture shop, shopping queries with hard negatives, and search on an open retrieval benchmark. Complete at least one for the certificate; each one is a portfolio piece.

Complete at least one case study for the certificate. The others are optional. What each one covers

  1. Product search for a furniture shop150 min
  2. Shopping queries with hard negatives150 min
  3. Search a help centre or a scientific corpus150 min

After the case studiesFinal assignment

Use every module in one practice project. You collect evidence, and check it against the course outcomes.

  1. Final assignment180 min

Case studies

Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.

  1. Product search for a furniture shop2 hr 30 min

    On the full WANDS dataset: a BM25 baseline, query understanding, hybrid search, a cross-encoder and LambdaMART, compared on NDCG@10 and latency; recommend a design for a 200 ms budget.

  2. Shopping queries with hard negatives2 hr 30 min

    On the Amazon Shopping Queries dataset: rank products with exact, substitute, complement and irrelevant labels; measure the cost of showing substitutes; compare English and Spanish slices.

  3. Search a help centre or a scientific corpus2 hr 30 min

    On an open retrieval benchmark with a permissive licence: lexical, dense and hybrid retrieval, re-ranking, and when the cheap method is good enough.

Reviews

Loading the reviews…

    Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.

    Share this course