The same ideas in a search server
- Lesson 30
- 30 min
- Module 8: Run search in production
Where we are
We built every part of search in Python. Many teams run a search server instead. In this lesson, we see BM25, synonyms, vectors and hybrid queries in OpenSearch, and what a server adds and costs to run.
Lesson 30 of 30 · Search and Ranking: Find the Best MatchThe same ideas in a search server
1. The search problem
2. Measure relevance before you change anything
3. Lexical search: the baseline to beat
4. Understand the query
5. Dense and hybrid retrieval
6. Re-rank the top results
7. The parts of search that people see
8. Run search in production
What you will be able to do
Recognise every part of the search you built inside a search server. You map the project’s parts to OpenSearch: BM25, analyzers and synonyms, a vector field, hybrid queries and index versions. Then you decide when a server is worth its cost.
Before you startWhat to know and to have ready
What you need to know
- BM25, fields and analyzers, from BM25 and Normalise text.
- Hybrid search by reciprocal rank fusion, from Hybrid search.
- Index versions and the pointer, from Keep the index fresh.
What you need on your computer (only to try the steps)
- Docker, with about 2 GB of free memory for the container. The Docker and cloud essentials for AI developers course explains how to install and run it.
- The project, for its data and its bi-encoder. Reading the lesson needs nothing.
A home workshop and a factory machine
Section titled “A home workshop and a factory machine”You built a bookshelf at home with hand tools. You know every cut, because you made it. Now a company asks for a thousand bookshelves a week. You buy a factory machine. It makes the same cuts, faster, all day, and many people can use it.
The machine does nothing that you did not understand by hand. But it needs a room, power, and someone to maintain it. If you make three bookshelves a year, the machine is a bad choice.
The problem: one laptop, one process
Section titled “The problem: one laptop, one process”The larkfield-search project runs in one Python process on one computer. That was right for learning: every part is visible, and you measured each one. A busy shop needs more:
- Many searches at the same time, on several machines, without a slow search blocking the others.
- Updates while it serves: new products added, removed products deleted, in seconds, without a rebuild.
- No single point of failure: if one machine stops, search keeps working.
- Filters and facets on millions of products, fast.
Our project does none of this, on purpose. A search server does all of it. It is a program that keeps the indexes, takes updates over the network, answers queries, and runs on several machines as a cluster.
The idea: the same parts, in a server
Section titled “The idea: the same parts, in a server”OpenSearch is an open-source search server (Apache-2.0 licence). It began as a copy of Elasticsearch, and it uses Lucene, a search library, inside. You send it requests over HTTP, with JSON bodies. Every idea of this course has a place in it.
Text description of the diagram
Six rows, each a part of larkfield-search with an arrow to its OpenSearch feature. index.py (SQLite FTS5 + bm25s) to an index with BM25 similarity. normalise.py (lowercase, stemming) to an analyzer. synonyms.csv to a synonym_graph filter. products.npy (flat NumPy matrix) to a knn_vector field with an HNSW graph. rrf() to a hybrid query and a search pipeline. outputs/index/ (a folder per build) to index versions and an alias.
Here are the words that change:
| In this course | In OpenSearch | What it is |
|---|---|---|
| a product | a document | one JSON object with fields |
| the index folder | an index | the documents and their inverted indexes |
| the analyzer of a field | a field’s analyzer | the steps that turn text into tokens |
| the flat embedding matrix | a knn_vector field | a vector for each document, with a search structure |
a run like hybrid-rrf |
a query and a search pipeline | how the results are found and combined |
the pointer current |
an alias | a name that points to one index version |
Larkfield in OpenSearch, step by step (not run)
Section titled “Larkfield in OpenSearch, step by step (not run)”1. Start one node for practice
Section titled “1. Start one node for practice”This starts a single OpenSearch node on your computer, with security turned off. Never turn security off on a server that other people can reach.
docker run -d --name larkfield-os -p 9200:9200 \ -e discovery.type=single-node \ -e DISABLE_INSTALL_DEMO_CONFIG=true -e DISABLE_SECURITY_PLUGIN=true \ -e OPENSEARCH_JAVA_OPTS="-Xms1g -Xmx1g" \ opensearchproject/opensearch:2.19.1-Xms1g -Xmx1g gives the server 1 GB of memory for its Java heap. Stop and delete the container with docker rm -f larkfield-os when you finish.
2. Create the index: BM25, an analyzer, synonyms and a vector field
Section titled “2. Create the index: BM25, an analyzer, synonyms and a vector field”One request creates the index products-v1. It sets BM25 with Larkfield’s settings and an English analyzer with stemming. It adds a synonym filter for search time only, and a vector field for the bi-encoder’s 384 numbers.
curl -s -X PUT "http://localhost:9200/products-v1" -H "Content-Type: application/json" -d '{ "settings": { "index": { "knn": true, "similarity": { "default": { "type": "BM25", "k1": 1.2, "b": 0.75 } } }, "analysis": { "filter": { "en_stem": { "type": "stemmer", "language": "english" }, "larkfield_synonyms": { "type": "synonym_graph", "synonyms": ["couch, sofa", "wardrobe, armoire"] } }, "analyzer": { "en_index": { "tokenizer": "standard", "filter": ["lowercase", "en_stem"] }, "en_search": { "tokenizer": "standard", "filter": ["lowercase", "larkfield_synonyms", "en_stem"] } } } }, "mappings": { "properties": { "title": { "type": "text", "analyzer": "en_index", "search_analyzer": "en_search" }, "description": { "type": "text", "analyzer": "en_index", "search_analyzer": "en_search" }, "title_fr": { "type": "text", "analyzer": "french" }, "product_class": { "type": "keyword" }, "price": { "type": "float" }, "in_stock": { "type": "boolean" }, "embedding": { "type": "knn_vector", "dimension": 384, "method": { "name": "hnsw", "space_type": "cosinesimil", "engine": "lucene" } } } }}'Read it part by part:
similarity: the same BM25 formula as the project, with k1 1.2 and b 0.75. Lucene leaves out the k1 + 1 factor. The project’sbm25ssettings do the same.analyzer: lowercase, then the English stemmer, as in Normalise text. The search analyzer adds the synonyms, so a query for “couch” also looks for “sofa”. The synonyms are applied only to the query, as in the project. One difference: the project counts an added word half (Synonyms and query rewriting), and this filter gives it the same weight as the person’s own word.title_fruses OpenSearch’s built-in French analyzer.product_classis akeyword: exact values for filters and facets, as in Filters and facets.embeddingis a vector field with an HNSW graph. HNSW is an approximate method: it finds near neighbours fast by walking a graph, and it may miss a few. The project’s flat matrix compares every product, so it misses none.
3. Load the products with their embeddings
Section titled “3. Load the products with their embeddings”The project already has the embeddings in data/embeddings/products.npy, in catalogue order. A short Python script sends each product as a document. It uses the opensearch-py client (python -m pip install opensearch-py; the course does not pin or test it).
import numpy as npfrom opensearchpy import OpenSearch, helpersfrom larkfield_search import data
client = OpenSearch("http://localhost:9200")cat = data.catalogue()vectors = np.load("data/embeddings/products.npy").astype("float32")docs = ({"_index": "products-v1", "_id": row.product_id, "title": row.title, "description": row.description, "title_fr": row.title_fr, "product_class": row.product_class, "price": float(row.price), "in_stock": bool(row.in_stock), "embedding": vectors[i].tolist()} for i, row in enumerate(cat.itertuples()))print(helpers.bulk(client, docs, chunk_size=500))The bulk API sends many documents in one request. Later, the same API adds a new product or deletes a removed one, one document at a time, while the server keeps answering queries. That is the incremental update of Keep the index fresh.
4. A lexical query with BM25
Section titled “4. A lexical query with BM25”curl -s "http://localhost:9200/products-v1/_search" -H "Content-Type: application/json" -d '{ "size": 10, "query": { "multi_match": { "query": "couch", "fields": ["title", "description", "title_fr"] } }}'multi_match scores each field with BM25 and keeps the best field’s score by default. The project’s bm25-fields run adds weighted field scores instead. So the two orders will differ a little, even with the same formula.
5. Hybrid search: BM25 and vectors together
Section titled “5. Hybrid search: BM25 and vectors together”OpenSearch can combine a lexical query and a vector query in one hybrid query. A search pipeline combines the two lists. Your code encodes the query with the same bi-encoder as the products, then sends the vector.
Text description of the diagram
“outdoor cushion” splits in two. Top: “match query: BM25 on title, description; analyzer + synonyms”. Bottom: “Your code encodes the query: bi-encoder, 384 numbers” → “knn query: top 100 by vector, approximate (HNSW)”. Both go to “Search pipeline: score-ranker-processor, RRF, rank constant 60” → “One ranked list, top 10 to the page”.
First, create the pipeline once. The score-ranker-processor with RRF came in OpenSearch 2.19. rank_constant is the k of reciprocal rank fusion: 60, as in the project.
curl -s -X PUT "http://localhost:9200/_search/pipeline/larkfield-rrf" -H "Content-Type: application/json" -d '{ "phase_results_processors": [ { "score-ranker-processor": { "combination": { "technique": "rrf", "rank_constant": 60 } } } ]}'Then send the hybrid query. Here in Python, with the project’s bi-encoder:
from larkfield_search import densevector = dense.encode("outdoor cushion").astype("float32").tolist()body = {"size": 10, "query": {"hybrid": {"queries": [ {"multi_match": {"query": "outdoor cushion", "fields": ["title", "description", "title_fr"]}}, {"knn": {"embedding": {"vector": vector, "k": 100}}},]}}}hits = client.search(index="products-v1", body=body, params={"search_pipeline": "larkfield-rrf"})print([h["_id"] for h in hits["hits"]["hits"]])The project’s hybrid-blend adds scaled scores: 0.8 lexical and 0.2 dense, each scaled 0 to 1. For this, OpenSearch has the normalization-processor, with min_max scaling and weights [0.8, 0.2].
6. Versions and an alias
Section titled “6. Versions and an alias”An alias is the pointer of Keep the index fresh, inside the server. The shop’s code always searches the alias products. To release version 2, build products-v2 beside version 1, check it, and move the alias in one atomic request:
curl -s -X POST "http://localhost:9200/_aliases" -H "Content-Type: application/json" -d '{ "actions": [ { "remove": { "index": "products-v1", "alias": "products" } }, { "add": { "index": "products-v2", "alias": "products" } } ]}'Both actions happen together. To roll back, swap them.
Measure it the same way
Section titled “Measure it the same way”A server does not make search better by itself. It runs the same ideas. To know if it ranks as well as the project, measure it the same way. Run the 243 test queries through OpenSearch and collect each top 100. Then score them with compare.py against the same judgments. Expect small differences. The tokenizer and stemmer are not identical. multi_match combines fields in a different way. And HNSW is approximate.
Measure the time too, as in Latency and cost budgets. A request over HTTP adds network time to every search.
LambdaMART needs a plugin, or your own code after the server’s search. Many teams keep stage two in their own service: the server returns 100 candidates, and their code re-ranks them. That is the funnel of Two-stage ranking.
What a server adds, and what it costs
Section titled “What a server adds, and what it costs”| A server adds | It costs |
|---|---|
| Many searches at once, on several machines | Machines that run all the time |
| Updates while it serves, in seconds | Memory: the inverted indexes, the HNSW graphs and the Java heap |
| Copies of the data, so one machine can fail | Usually three nodes, not one, for a safe cluster |
| Filters, facets and aggregations on millions of documents | People who know how to run, upgrade and monitor it |
| Aliases, snapshots and backups | Upgrades, security settings and access control |
A worked example with the course’s example price: one server with 4 vCPUs and 16 GB costs US$0.17 an hour (an assumption). That is 0.17 × 24 × 30 ≈ US$122 a month. A cluster of three such nodes costs about US$367 a month. Storage, backups and the time of the people who run it come on top. Hosted search services exist too: they run the cluster for you, at their own price. Prices change: check the provider’s page.
Should Larkfield run a search server?
Section titled “Should Larkfield run a search server?”Not yet. Its indexes are 102.5 MB of lexical indexes and 31.3 MB of embeddings. LambdaMART answers in 36 ms on one thread. A full rebuild takes seconds, and the synthetic log has about 2,500 searches a week. The Python service on one small server fits the budget.
A server becomes worth its cost in three cases. The catalogue has millions of products, updates must show within seconds, or search must survive a failed machine. The ideas stay the same. You now know each of them by hand, so you can read a server’s settings and know what they do.
Common mistake
Moving to a server and assuming the ranking is the same
Symptom: the team moves Larkfield’s search to OpenSearch over a weekend. The settings look right: BM25 with k1 1.2 and b 0.75, the English stemmer, the synonyms, RRF with k 60. On Monday, Omar asks: “Is it as good as before?” Nobody knows.
Cause: the team copied the settings and skipped the measurement. Small parts differ: the tokenizer, how fields combine, an approximate vector search, and LambdaMART is missing. Each can move NDCG@10, on some slices more than others.
Fix: before any person sees it, run the same 243 test queries through the server. Score them with compare.py, and compare with the project’s run by slice. Then test it online, as in Test changes online.
Consequence in search work: a new tool is a ranking change like any other. Measure it offline, then online, then release it with a way back.
Knowledge check
Section titled “Knowledge check”Check your understanding
In OpenSearch, why is the synonym filter only in the search analyzer, not in the index analyzer?
Show the answer
So the synonyms expand the query, not the stored products. The stored tokens then do not depend on the rules, and the products are indexed the same way whatever the rules say. The project applies its synonym rules in the same place: to the query. (How a server picks up a changed rule list depends on its version: check its documentation.)
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
The project's dense search compares the query with all 40,066 products. OpenSearch's knn_vector field uses HNSW. What do you gain and what can you lose?
Show the answer
HNSW reads only a small part of the vectors by walking a graph, so it stays fast for millions of products. But it is approximate: it can miss a few true nearest products. At Larkfield’s size, the flat search takes 2.1 ms and misses nothing, so HNSW is not needed.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Which OpenSearch feature plays the role of the pointer file `current` from the last lesson, and why is it safe?
Show the answer
An alias. The shop’s code searches the alias, and one _aliases request removes it from the old index and adds it to the new one together, atomically. No search sees half of the switch, and moving it back is the rollback.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A search server (OpenSearch, Apache-2.0, with Lucene inside) runs the same ideas as the project. It has BM25 similarity, analyzers with stemming and synonyms, a vector field, hybrid queries with RRF, and aliases. The course team did not run it.
- It adds many searches at once, updates while serving, copies of the data and filters at scale. It costs machines that run all the time, memory and people. One example node at US$0.17 an hour costs about US$122 a month.
- Moving to a server is a ranking change: measure it with the same test queries and
compare.py, then test it online. - Larkfield does not need a server at its size. A catalogue of millions, updates in seconds, or a need to survive a failed machine would change that.
This is the end of the teaching modules. You can now frame a search problem, measure it, build every family of methods, and run the result in production. Next, the case studies: start with product search on the full WANDS data.