Skip to content
Français

The same ideas in a search server

  • Lesson 30
  • 30 min
  • Module 8: Run search in production

Where we are

We built every part of search in Python. Many teams run a search server instead. In this lesson, we see BM25, synonyms, vectors and hybrid queries in OpenSearch, and what a server adds and costs to run.

What you will be able to do

Recognise every part of the search you built inside a search server. You map the project’s parts to OpenSearch: BM25, analyzers and synonyms, a vector field, hybrid queries and index versions. Then you decide when a server is worth its cost.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need on your computer (only to try the steps)

  • Docker, with about 2 GB of free memory for the container. The Docker and cloud essentials for AI developers course explains how to install and run it.
  • The project, for its data and its bi-encoder. Reading the lesson needs nothing.

You built a bookshelf at home with hand tools. You know every cut, because you made it. Now a company asks for a thousand bookshelves a week. You buy a factory machine. It makes the same cuts, faster, all day, and many people can use it.

The machine does nothing that you did not understand by hand. But it needs a room, power, and someone to maintain it. If you make three bookshelves a year, the machine is a bad choice.

The larkfield-search project runs in one Python process on one computer. That was right for learning: every part is visible, and you measured each one. A busy shop needs more:

  • Many searches at the same time, on several machines, without a slow search blocking the others.
  • Updates while it serves: new products added, removed products deleted, in seconds, without a rebuild.
  • No single point of failure: if one machine stops, search keeps working.
  • Filters and facets on millions of products, fast.

Our project does none of this, on purpose. A search server does all of it. It is a program that keeps the indexes, takes updates over the network, answers queries, and runs on several machines as a cluster.

OpenSearch is an open-source search server (Apache-2.0 licence). It began as a copy of Elasticsearch, and it uses Lucene, a search library, inside. You send it requests over HTTP, with JSON bodies. Every idea of this course has a place in it.

Each part of larkfield-search maps to an OpenSearch feature. index.py with SQLite FTS5 and bm25s maps to an index with BM25 similarity. normalise.py, with lowercase and stemming, maps to an analyzer with lowercase and a stemmer. synonyms.csv maps to a synonym_graph filter. products.npy, a flat NumPy matrix, maps to a knn_vector field with an HNSW graph. rrf() in runs/dense.py maps to a hybrid query and a search pipeline. outputs/index/, a folder per build, maps to index versions and an alias.
Every part you built has a place in a search server.
Text description of the diagram

Six rows, each a part of larkfield-search with an arrow to its OpenSearch feature. index.py (SQLite FTS5 + bm25s) to an index with BM25 similarity. normalise.py (lowercase, stemming) to an analyzer. synonyms.csv to a synonym_graph filter. products.npy (flat NumPy matrix) to a knn_vector field with an HNSW graph. rrf() to a hybrid query and a search pipeline. outputs/index/ (a folder per build) to index versions and an alias.

Here are the words that change:

In this course In OpenSearch What it is
a product a document one JSON object with fields
the index folder an index the documents and their inverted indexes
the analyzer of a field a field’s analyzer the steps that turn text into tokens
the flat embedding matrix a knn_vector field a vector for each document, with a search structure
a run like hybrid-rrf a query and a search pipeline how the results are found and combined
the pointer current an alias a name that points to one index version

Larkfield in OpenSearch, step by step (not run)

Section titled “Larkfield in OpenSearch, step by step (not run)”

This starts a single OpenSearch node on your computer, with security turned off. Never turn security off on a server that other people can reach.

Terminal window
docker run -d --name larkfield-os -p 9200:9200 \
-e discovery.type=single-node \
-e DISABLE_INSTALL_DEMO_CONFIG=true -e DISABLE_SECURITY_PLUGIN=true \
-e OPENSEARCH_JAVA_OPTS="-Xms1g -Xmx1g" \
opensearchproject/opensearch:2.19.1

-Xms1g -Xmx1g gives the server 1 GB of memory for its Java heap. Stop and delete the container with docker rm -f larkfield-os when you finish.

2. Create the index: BM25, an analyzer, synonyms and a vector field

Section titled “2. Create the index: BM25, an analyzer, synonyms and a vector field”

One request creates the index products-v1. It sets BM25 with Larkfield’s settings and an English analyzer with stemming. It adds a synonym filter for search time only, and a vector field for the bi-encoder’s 384 numbers.

Terminal window
curl -s -X PUT "http://localhost:9200/products-v1" -H "Content-Type: application/json" -d '{
"settings": {
"index": { "knn": true, "similarity": { "default": { "type": "BM25", "k1": 1.2, "b": 0.75 } } },
"analysis": {
"filter": {
"en_stem": { "type": "stemmer", "language": "english" },
"larkfield_synonyms": { "type": "synonym_graph", "synonyms": ["couch, sofa", "wardrobe, armoire"] }
},
"analyzer": {
"en_index": { "tokenizer": "standard", "filter": ["lowercase", "en_stem"] },
"en_search": { "tokenizer": "standard", "filter": ["lowercase", "larkfield_synonyms", "en_stem"] }
}
}
},
"mappings": {
"properties": {
"title": { "type": "text", "analyzer": "en_index", "search_analyzer": "en_search" },
"description": { "type": "text", "analyzer": "en_index", "search_analyzer": "en_search" },
"title_fr": { "type": "text", "analyzer": "french" },
"product_class": { "type": "keyword" },
"price": { "type": "float" },
"in_stock": { "type": "boolean" },
"embedding": { "type": "knn_vector", "dimension": 384,
"method": { "name": "hnsw", "space_type": "cosinesimil", "engine": "lucene" } }
}
}
}'

Read it part by part:

  • similarity: the same BM25 formula as the project, with k1 1.2 and b 0.75. Lucene leaves out the k1 + 1 factor. The project’s bm25s settings do the same.
  • analyzer: lowercase, then the English stemmer, as in Normalise text. The search analyzer adds the synonyms, so a query for “couch” also looks for “sofa”. The synonyms are applied only to the query, as in the project. One difference: the project counts an added word half (Synonyms and query rewriting), and this filter gives it the same weight as the person’s own word.
  • title_fr uses OpenSearch’s built-in French analyzer.
  • product_class is a keyword: exact values for filters and facets, as in Filters and facets.
  • embedding is a vector field with an HNSW graph. HNSW is an approximate method: it finds near neighbours fast by walking a graph, and it may miss a few. The project’s flat matrix compares every product, so it misses none.

3. Load the products with their embeddings

Section titled “3. Load the products with their embeddings”

The project already has the embeddings in data/embeddings/products.npy, in catalogue order. A short Python script sends each product as a document. It uses the opensearch-py client (python -m pip install opensearch-py; the course does not pin or test it).

import numpy as np
from opensearchpy import OpenSearch, helpers
from larkfield_search import data
client = OpenSearch("http://localhost:9200")
cat = data.catalogue()
vectors = np.load("data/embeddings/products.npy").astype("float32")
docs = ({"_index": "products-v1", "_id": row.product_id,
"title": row.title, "description": row.description, "title_fr": row.title_fr,
"product_class": row.product_class, "price": float(row.price), "in_stock": bool(row.in_stock),
"embedding": vectors[i].tolist()}
for i, row in enumerate(cat.itertuples()))
print(helpers.bulk(client, docs, chunk_size=500))

The bulk API sends many documents in one request. Later, the same API adds a new product or deletes a removed one, one document at a time, while the server keeps answering queries. That is the incremental update of Keep the index fresh.

Terminal window
curl -s "http://localhost:9200/products-v1/_search" -H "Content-Type: application/json" -d '{
"size": 10,
"query": { "multi_match": { "query": "couch", "fields": ["title", "description", "title_fr"] } }
}'

multi_match scores each field with BM25 and keeps the best field’s score by default. The project’s bm25-fields run adds weighted field scores instead. So the two orders will differ a little, even with the same formula.

5. Hybrid search: BM25 and vectors together

Section titled “5. Hybrid search: BM25 and vectors together”

OpenSearch can combine a lexical query and a vector query in one hybrid query. A search pipeline combines the two lists. Your code encodes the query with the same bi-encoder as the products, then sends the vector.

The query outdoor cushion goes two ways. A match query runs BM25 on title and description with the analyzer and synonyms. Your code encodes the query with the bi-encoder into 384 numbers, and a knn query finds the top 100 by vector, approximately, with HNSW. A search pipeline with the score-ranker-processor combines both by RRF with rank constant 60. The result is one ranked list, and the top 10 go to the page.
A hybrid query in OpenSearch: the same reciprocal rank fusion as the project's hybrid-rrf, done by the server.
Text description of the diagram

“outdoor cushion” splits in two. Top: “match query: BM25 on title, description; analyzer + synonyms”. Bottom: “Your code encodes the query: bi-encoder, 384 numbers” → “knn query: top 100 by vector, approximate (HNSW)”. Both go to “Search pipeline: score-ranker-processor, RRF, rank constant 60” → “One ranked list, top 10 to the page”.

First, create the pipeline once. The score-ranker-processor with RRF came in OpenSearch 2.19. rank_constant is the k of reciprocal rank fusion: 60, as in the project.

Terminal window
curl -s -X PUT "http://localhost:9200/_search/pipeline/larkfield-rrf" -H "Content-Type: application/json" -d '{
"phase_results_processors": [
{ "score-ranker-processor": { "combination": { "technique": "rrf", "rank_constant": 60 } } }
]
}'

Then send the hybrid query. Here in Python, with the project’s bi-encoder:

from larkfield_search import dense
vector = dense.encode("outdoor cushion").astype("float32").tolist()
body = {"size": 10, "query": {"hybrid": {"queries": [
{"multi_match": {"query": "outdoor cushion", "fields": ["title", "description", "title_fr"]}},
{"knn": {"embedding": {"vector": vector, "k": 100}}},
]}}}
hits = client.search(index="products-v1", body=body, params={"search_pipeline": "larkfield-rrf"})
print([h["_id"] for h in hits["hits"]["hits"]])

The project’s hybrid-blend adds scaled scores: 0.8 lexical and 0.2 dense, each scaled 0 to 1. For this, OpenSearch has the normalization-processor, with min_max scaling and weights [0.8, 0.2].

An alias is the pointer of Keep the index fresh, inside the server. The shop’s code always searches the alias products. To release version 2, build products-v2 beside version 1, check it, and move the alias in one atomic request:

Terminal window
curl -s -X POST "http://localhost:9200/_aliases" -H "Content-Type: application/json" -d '{
"actions": [
{ "remove": { "index": "products-v1", "alias": "products" } },
{ "add": { "index": "products-v2", "alias": "products" } }
]
}'

Both actions happen together. To roll back, swap them.

A server does not make search better by itself. It runs the same ideas. To know if it ranks as well as the project, measure it the same way. Run the 243 test queries through OpenSearch and collect each top 100. Then score them with compare.py against the same judgments. Expect small differences. The tokenizer and stemmer are not identical. multi_match combines fields in a different way. And HNSW is approximate.

Measure the time too, as in Latency and cost budgets. A request over HTTP adds network time to every search.

LambdaMART needs a plugin, or your own code after the server’s search. Many teams keep stage two in their own service: the server returns 100 candidates, and their code re-ranks them. That is the funnel of Two-stage ranking.

A server adds It costs
Many searches at once, on several machines Machines that run all the time
Updates while it serves, in seconds Memory: the inverted indexes, the HNSW graphs and the Java heap
Copies of the data, so one machine can fail Usually three nodes, not one, for a safe cluster
Filters, facets and aggregations on millions of documents People who know how to run, upgrade and monitor it
Aliases, snapshots and backups Upgrades, security settings and access control

A worked example with the course’s example price: one server with 4 vCPUs and 16 GB costs US$0.17 an hour (an assumption). That is 0.17 × 24 × 30 ≈ US$122 a month. A cluster of three such nodes costs about US$367 a month. Storage, backups and the time of the people who run it come on top. Hosted search services exist too: they run the cluster for you, at their own price. Prices change: check the provider’s page.

Not yet. Its indexes are 102.5 MB of lexical indexes and 31.3 MB of embeddings. LambdaMART answers in 36 ms on one thread. A full rebuild takes seconds, and the synthetic log has about 2,500 searches a week. The Python service on one small server fits the budget.

A server becomes worth its cost in three cases. The catalogue has millions of products, updates must show within seconds, or search must survive a failed machine. The ideas stay the same. You now know each of them by hand, so you can read a server’s settings and know what they do.

Common mistake

Moving to a server and assuming the ranking is the same

Symptom: the team moves Larkfield’s search to OpenSearch over a weekend. The settings look right: BM25 with k1 1.2 and b 0.75, the English stemmer, the synonyms, RRF with k 60. On Monday, Omar asks: “Is it as good as before?” Nobody knows.

Cause: the team copied the settings and skipped the measurement. Small parts differ: the tokenizer, how fields combine, an approximate vector search, and LambdaMART is missing. Each can move NDCG@10, on some slices more than others.

Fix: before any person sees it, run the same 243 test queries through the server. Score them with compare.py, and compare with the project’s run by slice. Then test it online, as in Test changes online.

Consequence in search work: a new tool is a ranking change like any other. Measure it offline, then online, then release it with a way back.

Check your understanding

In OpenSearch, why is the synonym filter only in the search analyzer, not in the index analyzer?

Show the answer

So the synonyms expand the query, not the stored products. The stored tokens then do not depend on the rules, and the products are indexed the same way whatever the rules say. The project applies its synonym rules in the same place: to the query. (How a server picks up a changed rule list depends on its version: check its documentation.)

Check your understanding

The project's dense search compares the query with all 40,066 products. OpenSearch's knn_vector field uses HNSW. What do you gain and what can you lose?

Show the answer

HNSW reads only a small part of the vectors by walking a graph, so it stays fast for millions of products. But it is approximate: it can miss a few true nearest products. At Larkfield’s size, the flat search takes 2.1 ms and misses nothing, so HNSW is not needed.

Check your understanding

Which OpenSearch feature plays the role of the pointer file `current` from the last lesson, and why is it safe?

Show the answer

An alias. The shop’s code searches the alias, and one _aliases request removes it from the old index and adds it to the new one together, atomically. No search sees half of the switch, and moving it back is the rollback.

Recap

  • A search server (OpenSearch, Apache-2.0, with Lucene inside) runs the same ideas as the project. It has BM25 similarity, analyzers with stemming and synonyms, a vector field, hybrid queries with RRF, and aliases. The course team did not run it.
  • It adds many searches at once, updates while serving, copies of the data and filters at scale. It costs machines that run all the time, memory and people. One example node at US$0.17 an hour costs about US$122 a month.
  • Moving to a server is a ranking change: measure it with the same test queries and compare.py, then test it online.
  • Larkfield does not need a server at its size. A catalogue of millions, updates in seconds, or a need to survive a failed machine would change that.

This is the end of the teaching modules. You can now frame a search problem, measure it, build every family of methods, and run the result in production. Next, the case studies: start with product search on the full WANDS data.

Next: Product search for a furniture shop