Match products between two shops
- Case study 1
- 120 min
- Optional · 1 case study needed for the certificate
Where we are
In eight modules, we matched Larkfield's synthetic records. In this case study, we match real products between two shops from an open benchmark, compare several methods, and report precision at a fixed recall and the cost.
Case study 1 of 3 · Entity Resolution and Record Matching: Is This the Same Thing?Match products between two shops
1. The matching problem
2. Clean first, then match exactly
3. Blocking: do not compare everything
4. Compare records: similarity measures
5. Probabilistic linkage
6. Learned matching: classifiers, embeddings and language models
7. From pairs to entities
8. Run matching in production
What you will be able to do
Match real products between two online shops, from the raw files to a decision that a shop team can use. You will normalise titles, block, compare, classify and measure, as in the course. You will compare seven methods under one measure, and choose a design with its thresholds, its review band and its cost.
The notebook runs every step of this page, with the same code and the same output, plus three tasks with checks. Its setup cells install the course’s packages and download about 4 MB of data. The benchmark comes from its source page; the embeddings and recorded answers come from the course’s labs. It needs no account and no key. After the install, it runs in under a minute.
Before you startWhat to know and to have ready
What you need to know
This case study uses the whole course. These lessons matter most:
- Normalising text and parsing product titles, from Normalise and Parse into parts.
- Pair precision, recall and F1, from Measure a matcher.
- Pair completeness and reduction ratio, from What blocking loses.
- The comparison vector, from Compare by field, then combine.
- Splink without labels, from Learn the weights without labels.
- The pair classifier, embeddings and the unsure band, from A supervised pair classifier, Embeddings for names, addresses and products and A language model for the unsure pairs.
What you need on your computer (only for the local route; the notebook needs none of it)
- A virtual environment with the course’s packages from Module 6, plus Splink 5.0.0, DuckDB 2.0.0 and pyarrow 26.0.1 from Module 5. The environment of the
larkfield-matchproject has all of them. This case study does not change the project’s files. - The notebook, downloaded from the link above, in VS Code or JupyterLab. Its setup cell makes a new folder
product-matchingnext to it. To start again, delete that folder.
Tested on macOS with Python 3.12.13, in Jupyter from a fresh start. Windows and Linux are not tested.
The situation
Section titled “The situation”You want to buy a television. A price-comparison website shows the same model in two shops, with two prices. Someone had to decide that two offers with two different titles are one product.
Leila runs the catalogue team of a small price-comparison website. Ben is her data engineer. The site compares the offers of two online shops that sold electronics, Abt and Buy. (Leila, Ben and the website are invented. The two shops and their product titles are real.)
Each error has a price:
- A false match shows two different products as one. A reader sees a wrong comparison, for example a camera case next to a camera. Leila says: “One wrong match on the page costs us more trust than ten missing ones.”
- A missed match hides a comparison that the site could show. It costs a little traffic.
So Leila sets a rule: matches go live automatically only when at least 99 in 100 of them are right. A person checks the unsure ones.
What you will build
Section titled “What you will build”You will build a matcher that reads the two shops’ product lists. It puts every candidate pair in one of three groups: accept, review or no match. You will compare seven methods on the same pairs with the same measures, and choose one design. Then you run that design on products that no step has seen, and on a second, harder pair of shops.
The skills come from the whole course, Modules 2 to 8. One idea is new here: one-to-one matching.
The data
Section titled “The data”The data is a public benchmark: a dataset that researchers publish so that everyone can test methods on the same problem. It comes from the database group of the University of Leipzig. It has the product offers of Abt (1,081) and Buy (1,092), and a list of 1,097 true matches: pairs of offers that are the same product.
Real data, at last. Every record in Modules 1 to 8 was synthetic. This data is real: the shops typed these titles for their customers, and the Leipzig researchers made the list of true matches, not our code.
The data dictionary
Section titled “The data dictionary”These are the columns that the case study uses.
| Column | What it means | Use |
|---|---|---|
id |
The offer’s number in its shop | Key |
name |
The product title, as the shop wrote it | Normalised, compared, embedded |
description |
A longer text (in Buy, often empty) | Searched for model codes |
manufacturer |
The brand (Buy only) | Not used: Abt has no such column, so the first word of the title stands for the brand |
price |
The price, such as $399.00 (often empty) |
Compared when both offers have one |
idAbt, idBuy |
One true match per row (the third file) | Labels only, never an input |
First checks
Section titled “First checks”print("repeated IDs:", int(abt.id.duplicated().sum()), int(buy.id.duplicated().sum()))print("no price:", f"Abt {(abt.price == '').mean():.0%},", f"Buy {(buy.price == '').mean():.0%}")print("no description in Buy:", f"{(buy.description == '').mean():.0%}")print("products with a match:", f"Abt {abt.id.isin(matches.id_a).sum()} of {len(abt)},", f"Buy {buy.id.isin(matches.id_b).sum()} of {len(buy)}")print("products with two or more matches:", f"Abt {(matches.id_a.value_counts() > 1).sum()},", f"Buy {(matches.id_b.value_counts() > 1).sum()}")The output:
repeated IDs: 0 0no price: Abt 61%, Buy 46%no description in Buy: 40%products with a match: Abt 1081 of 1081, Buy 1092 of 1092products with two or more matches: Abt 16, Buy 5What this tells us. Prices are often missing, so price can only be weak evidence. And every product has a partner in the other shop, the most important fact about this benchmark. In real work, a shop sells many products that the other shop does not have.
Four of the true matches:
Sony Black 9 Megapixel Cyber-Shot Digital Camera - DSCH50B | Sony Cyber-shot DSC-H50 Digital Camera - Black - DSCH50BPanasonic 2-Line Integrated Telephone System - KXTS208W | Panasonic KX-TS208W Corded PhoneSpeck SeeThru Clear Hard Shell Case For 15' MacBook - MB15CLRSEEV2 | Speck Products SeeThru Case for Apple 15' MacBook Pro - MB15-CLR-SEE-V2VMware Fusion 2 For Mac - VMFM20BX2 | VMware Fusion 2 - VMFM20BX2What this tells us. Abt ends almost every title with the model number. Buy writes it with hyphens (KX-TS208W), in another place, or not at all. The other words differ too.
Frame the problem
Section titled “Frame the problem”- The unit is a pair: one Abt offer and one Buy offer. The target is “same product or not”, from Leipzig’s list.
- The split puts each product in train (50%), validation (25%) or test (25%), by group. Products joined by true matches stay in one split, so no answer leaks from training to test. A pair belongs to the split of its Abt product. The test split has 277 true matches. The split, its shares and its seed (39) are our choice.
- The measures. Every method is measured on the test split, with a threshold chosen on validation. We report pair precision, recall and F1, as in Measure a matcher. Leila’s rule needs a second measure: precision at a fixed recall. We take the threshold that finds 80% of the validation matches, and report the precision at that threshold on test. A rule that cannot reach 80% recall gets no value.
- The labels are Leipzig’s. We made no label ourselves. The 99% rule, the reviewer’s time and the prices are our assumptions.
Normalise: one product, two titles
Section titled “Normalise: one product, two titles”The simplest matcher says “same product when the names are the same”. On the test split, it finds 1 of 277 true matches. Shops almost never write a title in the same way.
So we normalise the titles first, as in Normalise. Normalised titles are in lower case, with no accents or signs. The inch mark 42' becomes 42 in, and no hyphen or slash stays inside a word. Then a small function keeps the words that look like model codes: at least 4 characters, with both letters and digits.
Text description of the diagram
Two columns, Abt and Buy, and three rows. Row 1: the title as each shop wrote it. Row 2: the title after normalise(). Row 3: the model codes that model_codes() found. At the bottom, a green box says that the codes are the same: th42px80u.
Exact normalised titles find 4 matches: still almost nothing. A rule on the codes works much better:
same model code: {'precision': 0.954, 'recall': 0.675, 'f1': 0.791, 'called': 196, 'right': 187, 'true': 277}What this tells us. When two titles share a model code, they are the same product 95 times in 100. But a third of the true matches share no code. Some Buy titles have none (“LG 5.0 cu.ft. Freestanding Gas Range”). Others write a shorter code (“JFS516” for “JFS516NA”).
Common mistake
A size that looks like a model number
Symptom: Ben’s first version of the code rule had a precision of 0.519: one call in two was wrong.
Cause: his code function kept every word with letters and digits. The most common “codes” in Abt were 1080p (28 products), 60gb and 16gb. Those are a screen resolution and sizes, shared by many different products. So every 1080p television “matched” every other 1080p product.
Fix: refuse a word that is a number followed by a unit (in, gb, p, hz and others). The precision rose to 0.954.
Consequence in AI work: before you trust a key, look at its most common values, as in What blocking loses. One common value can join thousands of pairs. In Module 7, one garden centre’s phone number joined 40 different people.
Blocking
Section titled “Blocking”The two shops make 1,081 × 1,092 = 1,180,452 pairs. A real catalogue has a million products, so we block, as in Module 3, and keep only candidate pairs.
The code rule is a precise blocking, but it keeps only 66% of the true matches. Two more blockings find titles that are similar, not equal:
- TF-IDF top 5. Think of a librarian who looks for a book by its rare words: “Acoustimass” says more than “speaker”. TF-IDF does the same. It cuts each title into short pieces of 3 to 5 letters. It gives a piece more weight when few titles have it. Two titles that share rare pieces get a high TF-IDF similarity, from 0 to 1. For each product, we keep the 5 products of the other shop with the highest similarity, in both directions. The search and ranking course ranks documents with the same idea: see BM25.
- Embeddings top 5. The same, with the cosine of the titles’ embeddings, from the course’s Granite model, as in Embeddings for names, addresses and products. The embeddings come with the notebook, so it does not run the model.
candidates true kept pair completeness reduction ratiosame model code 770.0 729.0 0.6645 0.99935TF-IDF top 5 7308.0 1077.0 0.9818 0.99381embeddings top 5 7374.0 1077.0 0.9818 0.99375all three together 10205.0 1087.0 0.9909 0.99136What this tells us. TF-IDF and embeddings keep the same share of true matches, but not the same matches. Together with the codes, they keep 1,087 of 1,097 with 10,205 candidates: less than 1% of all pairs. In the notebook, you try 3 neighbours instead of 5: 5,919 candidates, and 6 more true matches lost.
Compare and classify
Section titled “Compare and classify”The comparison vector
Section titled “The comparison vector”Each candidate pair gets nine similarities, as in Compare by field, then combine:
- two string similarities of the titles (token set ratio and Jaro-Winkler), and the TF-IDF similarity;
- “same code”, the best similarity between two codes, and “the Abt code appears in the Buy title or description”;
- the share of numbers that the two titles have in common, the price ratio, and the embedding cosine.
A missing value (no code, no price) stays missing.
Seven methods on the same pairs
Section titled “Seven methods on the same pairs”Every method below sees the same 10,205 candidates and is measured on the same 277 test matches.
- Rules: the exact title, and the same model code.
- One score alone: the TF-IDF similarity, or the embedding cosine, with the threshold that gave the best F1 on validation.
- Splink without labels, as in Module 5. It compares the title words, the first word (the brand) and the codes, and learns its weights with expectation maximisation.
- Logistic regression and LightGBM on the comparison vector, as in A supervised pair classifier. They learn from the 5,097 candidate pairs of the train split (539 true).
Text description of the chart
Nine dots, one per method, on a scale from 0 to 1. Exact title 0.028; rule same model code 0.791; embeddings alone 0.579; TF-IDF alone 0.707; Splink without labels 0.765; logistic regression 0.862; LightGBM 0.869; LightGBM with the embedding cosine 0.88; LightGBM with one-to-one matching 0.924.
What this tells us. The classifiers win clearly: LightGBM F1 0.869, logistic regression 0.862. The code rule (0.791) beats Splink without labels (0.765). The embedding cosine alone is the weakest score (0.579), and adds little to LightGBM (0.880). LightGBM puts 55% of its gain on one field: the Abt code found in the Buy text.
Why Splink without labels is weak here
Section titled “Why Splink without labels is weak here”About 67% of the true candidate pairs share a model code. So the m probability of “share a code” should be about 0.67: the share of true matches that agree on it. Splink learned 0.048.
Expectation maximisation looks for two groups of pairs, and it does not know which group we mean. Our reading, not measured: it probably found pairs of one brand and one product line: they share many words, but rarely a code. Module 5 saw the same kind of problem with the date of birth. On product titles, the evidence is in free text, and labels help a lot.
The measure that Leila needs
Section titled “The measure that Leila needs”Text description of the chart
Seven dots. TF-IDF alone 0.535; embeddings alone 0.378; Splink without labels 0.326; logistic regression 0.884; LightGBM 0.906; LightGBM with the cosine 0.92; LightGBM with one-to-one 0.983. The two rules have no value: they cannot reach recall 0.8.
At recall 0.8, about 1 of LightGBM’s matches in 10 is wrong (0.906); Leila needs 99 in 100. The next idea closes most of that gap.
One product, one partner
Section titled “One product, one partner”Dance partners
Section titled “Dance partners”At a dance class, each person dances with one partner. A product match is similar. An offer in one shop is the same product as at most one offer in the other shop. (In this benchmark, a few products have two matches, because one shop listed the same product twice.)
The problem
Section titled “The problem”The classifier scores every pair on its own. So it can give 0.7 to “camera and camera case”, and 0.9 to “camera and camera”. It does not know that the camera already has a better partner.
The idea: one-to-one matching
Section titled “The idea: one-to-one matching”One-to-one matching keeps a pair only when it is the best pair of both its products. It drops all other pairs. The pairs are the edges of a graph, as in Clustering pairs, but here each product may keep only one edge.
Text description of the diagram
Four boxes: A1 and A2 on the left, B1 and B2 on the right, and four lines with their scores. The line A1–B1 (0.90) is thick and green: kept. The other three lines are dashed: dropped. A note lists each pair and the reason.
On the real candidates, one-to-one keeps 1,030 of the 10,205 pairs. LightGBM’s F1 rises from 0.869 to 0.924, and its precision from 0.930 to 0.976. Its precision at recall 0.8 rises from 0.906 to 0.983. On logistic regression (the notebook’s task), the F1 becomes 0.915.
When to use it: when each entity can appear at most once in each list, such as two shops’ catalogues or two registers. When not to use it: inside one list with duplicates, such as TermLarkfieldA made-up Canadian online shop for home and garden products. It is not a real company. All Nextia courses use its help desk and data as the example.Learn more Meet Larkfield’s customers with up to 5 records each. Also not when many products have no partner. Then a wrong pair with a low score can still be “the best” of two products. Keep a threshold as well.
Decide: accept, review or no match
Section titled “Decide: accept, review or no match”The bands
Section titled “The bands”As in Human review, we cut the score into three bands. On the validation split, we tried three lower and three upper thresholds. Six of the nine rows:
accepted accepted right precision accepted to review true in review recall if reviews are rightlower upper0.01 0.50 221.0 213.0 0.964 25.0 21.0 0.854 0.80 201.0 200.0 0.995 45.0 34.0 0.854 0.95 182.0 181.0 0.995 64.0 53.0 0.8540.05 0.50 221.0 213.0 0.964 20.0 17.0 0.839 0.80 201.0 200.0 0.995 40.0 30.0 0.839 0.95 182.0 181.0 0.995 59.0 49.0 0.839The lowest upper threshold that meets Leila’s 99% is 0.8. For the lower threshold, we take 0.01. Compared with 0.05, it adds 5 pairs to review, and 4 of them are true matches. On the test split, used once:
accepted accepted right precision accepted to review true in review recall if reviews are rightvalidation 201.0 200.0 0.995 45.0 34.0 0.854test 218.0 216.0 0.991 31.0 27.0 0.877What this tells us. On test, 216 of the 218 automatic matches are right. 27 of the 31 review pairs are true. If the person decides them all correctly, the site shows 87.7% of the true matches.
Where do the other 34 true matches go? The blocking loses 1. One-to-one drops 7 because the product’s other true partner won, and 23 because a wrong pair won. 3 score below 0.01.
Text description of the diagram
Five steps from left to right, each with its main number: normalise, block, compare, classify, one-to-one. Under the last step, three boxes for the bands: accept, review and no match, each with its count of pairs.
A language model for the review band (recorded)
Section titled “A language model for the review band (recorded)”As in A language model for the unsure pairs, we asked a small chat model (Azure Termchat-smallThe name that Larkfield’s code gives to a small, low-cost hosted model: gpt-6-luna on Azure. The code uses the name, so the model can change without a code change.Learn more What an LLM API isBuilding Reliable Applications with LLM APIs) about every validation and test pair in the band. We asked once and recorded the answers. The model read both titles, the first 300 characters of each description, and the prices. Its recorded answers for the 76 band pairs:
| The model said | True match | Not a match |
|---|---|---|
| same | 49 | 0 |
| different | 3 | 5 |
| unsure | 9 | 10 |
What this tells us. When the model said “same”, it was right 49 times out of 49. For vague Buy titles, it said “unsure”: “Offer B does not specify the model or colour”. It also caught a colour variant, rightly: “A is white (WH) and B is black (BF)”.
One of its “mistakes” is worth a look. For “Canon 2GB SD Secure Digital Card - 3505B001” against “Simpletech 2GB SD Card - 3505B001AA”, the model said “different”. Its reason: “Offer A is branded Canon, while Offer B is branded Simpletech”. The benchmark says “same”, because the part number is the same. A person might agree with the model. A benchmark’s labels can be wrong too.
49 of 49 is a small sample: the true error rate could still be about 6% (a rough rule for zero errors: 3 divided by the number of answers). So the model cannot replace the person yet. It can sort the queue: its “same” pairs first, with its reason shown to the reviewer.
What it costs
Section titled “What it costs”The prices are assumptions from the course’s price table; check the providers’ current pages.
- A CPU server costs US$0.17 an hour.
- The chat model costs US$0.10 per million input tokens and US$0.50 per million output tokens.
- A reviewer takes 1.2 minutes a pair and costs US$40 an hour.
| Step | Measured | Cost (assumption × measurement) |
|---|---|---|
| Compare and classify on a CPU | under 0.5 s for 10,205 pairs | under US$0.000002 per 1,000 candidate pairs |
| Language model, band pairs only | 252 tokens in, 139 out; median 1.96 s a pair | US$0.095 per 1,000 pairs it reads; only 1.5% of candidates are in the band |
| A person reviews the band | 115 pairs per 1,000 Abt products | 2.3 hours, about US$92 per 1,000 products |
What this tells us. The computer is almost free; the person is the cost. The language model costs about one cent per 100 band pairs, so it pays for itself if it saves a reviewer a few seconds a pair. The whole recording (186 pairs, with a margin around the band and 20 sure pairs as a control) cost US$0.016.
The design on new input
Section titled “The design on new input”A design must work on products that it has never seen. The notebook’s match_shops() runs every step, from normalising to the bands. We run it on the 270 Abt products of the test split, as if they were new, against all of Buy:
270 new products, 0.3 s: {'accept': 220, 'review': 47} count truedecisionaccept 220 216review 47 30What this tells us. The precision is 0.982, a little below the 0.991 of the evaluation, and the review queue is longer. The reason is one-to-one. A Buy product now chooses only among the 270 new products, so a weaker new product can become its “best” pair. In daily use, match new products against the whole catalogue, with the products that already have a partner. New records every day shows how.
The harder pair of shops
Section titled “The harder pair of shops”The Leipzig page has a second product benchmark: Amazon (1,363 offers) against Google’s shopping service (3,226). It has 1,300 true matches, mostly software. Here, not every product has a partner: only 1,291 of the 3,226 Google offers have one. In the notebook’s last task, you run the whole design on it with one call. The result on its test split:
{'blocking': {'candidates': 28776, 'true kept': 1289, 'pair completeness': 0.9915, ...}, 'threshold': 0.05, 'test': {'precision': 0.683, 'recall': 0.59, 'f1': 0.633, ...}}Text description of the chart
Four dots. Larkfield products, synthetic, p-lgbm: 0.996. Abt–Buy, LightGBM with one-to-one: 0.924. Larkfield customers, synthetic, lgbm: 0.902. Amazon–Google with the same design: 0.633.
What this tells us. The blocking still works (99% of true matches kept), but the F1 falls to 0.633. Software titles have few model numbers. Many offers differ only by a year or an edition. For example, the design matched “peachtree by sage premium accounting for nonprofits 2007” to the 2008 edition. Most offers have no partner, so one-to-one can keep a wrong one. Some “false matches” look right to a person, such as two Red Hat licences with the same part number rl296aa; we counted them as errors, as the benchmark says. A good next step is a version and edition field, parsed from the title as Module 2 parsed sizes and packs.
Compared with Larkfield. On Larkfield’s synthetic supplier feeds, LightGBM reached F1 0.996: our generator wrote sizes and packs in a few known ways. Real shops are less tidy. Abt–Buy is harder than our feeds, and Amazon–Google is harder than our synthetic customers.
Recommendation
Section titled “Recommendation”Ben writes to Leila in her words:
- Use the design above. Match automatically when the score is 0.8 or more.
- Review the band from 0.01 to 0.8: about 2.3 hours for every 1,000 products, mostly true matches with a vague title.
- Use the language model to sort the queue, not to decide. Show its reason to the reviewer, and measure it again after a few hundred reviews.
- Match new products against the whole catalogue, never only against each other.
- Ask the shops for model numbers or barcodes. A shared barcode would make most of this work unnecessary.
What not to do:
- Do not use the 0.8 threshold for another pair of shops without new labels and a new validation.
- Do not show a match as certain. Show “compare offers”, and let readers report a wrong match.
Insights and limits
Section titled “Insights and limits”- Every product has a partner here. That makes one-to-one look better than it will be in real work. In a real catalogue, measure the share of products without a partner first.
- Old and small. The files date from 2010: two electronics shops, about 1,000 products each.
- Labels can be wrong. The Canon and Simpletech pair shows that a benchmark’s truth is someone’s judgement.
- The test split is small. One error moves the precision by about 0.4 points, so do not read the third decimal.
- The recorded answers are from one model, once. A new model version, or a new run, gives other answers.
- The prices are assumptions. Check the providers’ current pages.
Responsible use
Section titled “Responsible use”- Credit the source. CC BY 4.0 requires the attribution in the box above wherever you use or share the data.
- The brands are real. This case study says nothing about any shop or brand. A wrong match on a public page can misrepresent a shop’s price. That is why Leila’s rule is strict, and why readers can report errors.
- No personal data. These are product offers only. The same methods on customer records need the care of Privacy and matching.
- A language model sees the text you send. Here it was public product text. Never send customer records to a model service without the right agreement.
Make it a portfolio piece
Section titled “Make it a portfolio piece”This case study shows that you can take real, messy records to a decision. Make it yours with one change: add a version field for Amazon–Google, or a threshold for products without a partner, or try another open benchmark.
What to put in the repository:
README.md(template below).- The notebook, run from top to bottom, with its outputs.
- The comparison table as a CSV: every method, its threshold, precision, recall, F1 and precision at recall 0.8.
DECISION.md: the thresholds, the review band, its size and its cost, and the assumptions.- The data source and licence, with the attribution text. CC BY 4.0 lets you share the data with credit, but a link and the checksums are enough.
A README template:
# Matching products between two online shops
## ProblemA price-comparison site must decide which offers in two shops are the same product.Matches go live automatically only when 99 in 100 are right; a person checks the rest.
## Data and licenceAbt-Buy benchmark, University of Leipzig (Köpcke, Thor and Rahm, PVLDB 2010), CC BY 4.0,https://dbs.uni-leipzig.de/research/projects/benchmark-datasets-for-entity-resolution.1,081 + 1,092 offers, 1,097 true matches. Split 50/25/25 by group of products.
## ApproachNormalised titles and model codes; blocking with the same code, TF-IDF top 5 andembeddings top 5 (pair completeness 0.991, 10,205 of 1.18 million pairs); a comparisonvector of 9 similarities; seven methods compared; LightGBM with one-to-one matching.
## Results (test split)F1 0.924 (exact names 0.007, code rule 0.791, Splink without labels 0.765).Accept >= 0.8: 218 matches, 216 right. Review 0.01-0.8: 31 pairs.
## LimitsEvery product has a partner in this benchmark; old data, one category. On the harderAmazon-Google benchmark, the same design reached only F1 0.633.How to describe it in an interview, in two sentences: “I matched real products between two shops and compared seven methods on one test split; LightGBM with one-to-one matching reached F1 0.92 and 99% precision on the automatic matches. On a harder software benchmark, the same design fell to 0.63, because versions and editions need their own field.”
Go further:
- Parse the edition and the year from software titles, add them to the comparison vector, and run Amazon–Google again.
- Replace “best of both” with an optimal assignment (
scipy.optimize.linear_sum_assignment), and compare. - Ask a chat model about the 31 test review pairs with your own key, and compare its answers with the recorded ones.
Knowledge check
Section titled “Knowledge check”Check your understanding
The code rule has a precision of 0.954 but a recall of only 0.675. Give two reasons why a true match may share no model code, from the titles in this case study.
Show the answer
First, some Buy titles have no model code at all, for example “LG 5.0 cu.ft. Freestanding Gas Range”. Second, a shop may write a shorter or another form of the code: “JFS516” against “JFS516NA”, or “PMCII” against “PMC2”. Exact equality of codes misses both. That is why the comparison vector also has the best similarity between two codes, and “the Abt code appears in the Buy text”.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
One-to-one matching raised LightGBM's precision from 0.930 to 0.976 on Abt–Buy. Would you expect the same gain on Larkfield's customer records? Explain.
Show the answer
No. One-to-one assumes that each entity appears at most once in each list. Larkfield’s customers have up to 5 records of one person, often in the same source, so one person may need several partners. One-to-one would drop true pairs. It also helps most when every product has a partner, as in this benchmark. In lists where many records have no partner, keep a threshold as well.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
The recorded language model said “same” for 49 band pairs and was right every time. Leila asks: “Can we let it accept band pairs without a person?” What do you answer?
Show the answer
Not yet. 49 answers is a small sample: the true error rate could still be several percent, and Leila’s rule allows 1 error in 100. The answers are also from one model version, recorded once. Use the model to sort the queue and show its reason, and keep measuring its answers against the reviewers’ decisions. If it stays right on a few hundred more pairs, the team can decide again.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- On real product data, exact names found 1 of 277 test matches. Normalised model codes were precise (0.954) but found only two thirds.
- Three blockings together (codes, TF-IDF top 5, embeddings top 5) kept 99.1% of the true matches with less than 1% of the pairs.
- LightGBM on the comparison vector (F1 0.869) beat the rules, Splink without labels (0.765) and the embeddings (0.579). Splink learned a wrong weight for the codes.
- One-to-one matching raised the F1 to 0.924, because each product has at most one partner.
- The decision: accept at 0.8 or more (216 of 218 right on test). Review from 0.01 to 0.8: about 115 pairs per 1,000 products. Let the language model sort the queue, not decide it.
Next, real bibliographic records: titles, author lists and venues from two sources.