Link bibliographic records
- Case study 2
- 120 min
- Optional · 1 case study needed for the certificate
Where we are
In eight modules, we matched customers and products. In this case study, we link real bibliographic records from two open sources, compare Splink with a pair classifier, and study the errors in titles and author lists.
Case study 2 of 3 · Entity Resolution and Record Matching: Is This the Same Thing?Link bibliographic records
1. The matching problem
2. Clean first, then match exactly
3. Blocking: do not compare everything
4. Compare records: similarity measures
5. Probabilistic linkage
6. Learned matching: classifiers, embeddings and language models
7. From pairs to entities
8. Run matching in production
What you will be able to do
Link real bibliographic records from two libraries, from the raw files to a decision that a research office can use. You will normalise titles, authors and venues, block, compare, classify and measure, as in the course. You will compare rules, Splink and two classifiers under one measure. Then you study every error, fix the main one, and choose a design with its thresholds and its review band.
The notebook runs every step of this page, with the same code and the same output, plus three tasks with checks. Its setup cells install the course’s packages and download about 2 MB of data. The benchmark comes from its source page; the embeddings come from the course’s labs. It needs no account and no key. After the install, it runs in under a minute.
Before you startWhat to know and to have ready
What you need to know
This case study uses the whole course, and it follows the pattern of Match products between two shops. These lessons matter most:
- Normalising text, from Normalise.
- Pair precision, recall and F1, from Measure a matcher.
- Pair completeness and reduction ratio, from What blocking loses.
- The comparison vector, from Compare by field, then combine.
- Splink without labels, from Learn the weights without labels.
- The pair classifier, from A supervised pair classifier, and the review band, from Human review.
What you need on your computer (only for the local route; the notebook needs none of it)
- A virtual environment with the course’s packages from Module 6, plus Splink 5.0.0, DuckDB 2.0.0 and pyarrow 26.0.1 from Module 5. The environment of the
larkfield-matchproject has all of them. This case study does not change the project’s files. - The notebook, downloaded from the link above, in VS Code or JupyterLab. Its setup cell makes a new folder
bibliographic-recordsnext to it. To start again, delete that folder.
Tested on macOS with Python 3.12.13, in Jupyter from a fresh start. Windows and Linux are not tested.
The situation
Section titled “The situation”A library may have the same book twice: one card from the old catalogue, one from a donation list. The cards are not the same. One says “2nd ed.”, the other writes the author’s first name in full. A librarian decides that they are one book.
Ruth runs the research-information team of a university. Daniel is her data analyst. Each year, the team lists every paper that the university’s computer scientists published. It takes the records from two public libraries: DBLP, a free index of computer-science papers, and the ACM Digital Library. Many papers are in both. (Ruth, Daniel and the university are invented. The two libraries and their records are real.)
Each error has a price:
- A false link joins two different papers. One paper disappears from the report, and a researcher complains.
- A missed link counts one paper twice. The report is too high, and a reviewer may find it.
Ruth says: “A missing paper is the worse error. People notice it at once.” So she sets a rule: links go into the report automatically only when at least 995 in 1,000 of them are right. A person checks the unsure ones.
What you will build
Section titled “What you will build”You will build a linker that reads the two libraries. It puts every candidate pair in one of three groups: accept, review or no link. You will compare ten methods on the same pairs with the same measures, look at every error, and choose one design. Then you run that design on records that no step has seen.
What is different from the first case study. These records are much cleaner than product titles, with separate fields: title, authors, venue and year. So a simple rule may already do well. The interesting question is: where does it still fail, and why?
The data
Section titled “The data”The data is a public benchmark, from the database group of the University of Leipzig, like the shops of the first case study. It has 2,616 DBLP records and 2,294 ACM records of papers from five database conferences and journals, 1994 to 2003. A third file lists 2,224 true links: pairs of records that are the same paper. (The source page says 2,614 DBLP records; the file has 2,616 rows.)
The data dictionary
Section titled “The data dictionary”| Column | What it means | Use |
|---|---|---|
id |
The record’s key in its library (conf/sigmod/FranklinJK96 in DBLP, a number in ACM) |
Key |
title |
The title, as the library wrote it | Normalised, compared, embedded |
authors |
The author names, separated by commas | Surnames compared |
venue |
The conference or journal | Mapped to one key |
year |
The year of publication | Compared |
idDBLP, idACM |
One true link per row (the third file) | Labels only, never an input |
First checks
Section titled “First checks”print("records with a match:", f"DBLP {dblp.id.isin(matches.id_a).sum()} of {len(dblp)},", f"ACM {acm.id.isin(matches.id_b).sum()} of {len(acm)}")print("true matches with the same year:", f"{(pairs_true.year_a == pairs_true.year_b).mean():.0%}")print("true matches with the same title, as written:", f"{(pairs_true.title_a == pairs_true.title_b).mean():.0%}")print("true matches with the same author list, as written:", f"{(pairs_true.authors_a == pairs_true.authors_b).mean():.0%}")The output of the notebook’s cell, which also counts repeated IDs and empty author lists:
repeated IDs: 0 0no authors: DBLP 0, ACM 14records with a match: DBLP 2224 of 2616, ACM 2224 of 2294records with two or more matches: DBLP 0, ACM 0true matches with the same year: 100%true matches with the same title, as written: 41%true matches with the same author list, as written: 28%What this tells us. The year always agrees, but titles and author lists, as written, often differ. And not every record has a partner: 392 DBLP records and 70 ACM records have none. In the first case study, every product had one.
One paper in two libraries
Section titled “One paper in two libraries”Text description of the diagram
Four columns: DBLP and ACM as written, then DBLP and ACM after normalising. Four rows: title, authors, venue and year. The cells after normalising are green, because the two records agree in every field.
The differences are typical of this data:
- Capitals and signs. “Tradeoffs” against “tradeoffs”, “PESTO :” against “PESTO:”.
- Author order. DBLP and ACM often list the same authors in another order.
- HTML codes. ACM writes letters such as ö as
ö. DBLP writes “Þór”, ACMThór. - Venue names. DBLP writes “SIGMOD Conference”, ACM “International Conference on Management of Data”. In this data, each DBLP venue goes with exactly one ACM venue.
- Typing errors. A few ACM titles have one: “tumble” for “tunable”, “databates” for “databases”.
Frame the problem
Section titled “Frame the problem”- The unit is a pair: one DBLP record and one ACM record. The target is “same paper or not”, from Leipzig’s list.
- The split puts each record in train (50%), validation (25%) or test (25%), by group, as in the first case study. A pair belongs to the split of its DBLP record. The test split has 549 true links. The split, its shares and its seed (39) are our choice.
- The measures. We measure every method on the test split, with a threshold chosen on validation. We report pair precision, recall and F1, as in Measure a matcher. Ruth’s rule needs a second measure: the precision at recall 0.95. We take the threshold that finds 95% of the validation links, and report the precision at that threshold on test.
- The labels are Leipzig’s. We made no label ourselves. The 995-in-1,000 rule, the reviewer’s time and the prices are our assumptions.
Normalise, then a rule
Section titled “Normalise, then a rule”The exact title, as written, finds 230 of the 549 test links (precision 0.909, recall 0.419). So we normalise first, as in Normalise:
normalise()decodes HTML codes, writes lower case, drops accents, signs and apostrophes, and joins hyphenated words (“Client-Server” becomes “clientserver”).surnames()keeps the last word of each author name.VENUE_KEYS, a small table written by hand, gives both names of a venue one key, such assigmod.
Then two rules:
same normalised title: {'precision': 0.853, 'recall': 0.896, 'f1': 0.874, 'called': 577, 'right': 492, 'true': 549}same normalised title, year, venue: {'precision': 0.972, 'recall': 0.896, 'f1': 0.933, 'called': 506, 'right': 492, 'true': 549}What this tells us. On clean records, one rule already reaches F1 0.933: the year and the venue remove most false links of the title alone. On products, the best rule reached 0.791.
Common mistake
A title that many papers share
Symptom: the rule “same normalised title” made 255 false links in the whole data.
Cause: some titles are not the title of one paper. They are the name of a regular column. The most common titles in DBLP are “Editor’s notes” (30 records), “Reminiscences on influential papers” (19), “Chair’s message” (18) and “Book review column” (7). Each such title joins every record of one library to every record of the other. These four titles made 169 of the 255 false links.
Fix: add the year and the venue to the rule. That removes most of them, but not all: one journal has many columns with the same title in one year.
Consequence in AI work: before you trust a key, look at its most common values, as in What blocking loses. In the first case study, a screen size (1080p) looked like a model number. Here, a column name looks like a paper title.
Blocking
Section titled “Blocking”The two libraries make 2,616 × 2,294 = 6,001,104 pairs. We compare three blockings:
candidates true kept pair completeness reduction ratiosame year and venue 157871.0 2224.0 1.0000 0.97369TF-IDF top 5 17638.0 2219.0 0.9978 0.99706embeddings top 5 17860.0 2215.0 0.9960 0.99702TF-IDF and embeddings 28561.0 2221.0 0.9987 0.99524- Same year and venue keeps every true link, only because this data is clean. A preprint and its journal version have different years and venues.
- TF-IDF top 5 and embeddings top 5 keep, for each record, the 5 most similar titles of the other library, in both directions, as in the first case study. TF-IDF compares short pieces of 3 to 5 letters and gives rare pieces more weight. The search and ranking course uses the same idea: see BM25.
What this tells us. TF-IDF and embeddings together keep 2,221 of 2,224 true links with 28,561 candidates, about one fifth of the year-and-venue blocking. We use them. In the notebook, one neighbour instead of five gives 3,117 candidates and still keeps 99.1% of the true links.
Compare and classify
Section titled “Compare and classify”The comparison vector
Section titled “The comparison vector”Each candidate pair gets eight similarities, as in Compare by field, then combine, plus the embedding cosine for one test:
- Title: token set ratio, Jaro-Winkler, and the TF-IDF similarity.
- Authors: the share of shared surnames (Jaccard), and the token sort ratio of the two author strings, which ignores the order. Also the difference in the number of authors.
- Year and venue: the year difference, and “same venue key”.
An empty author list (14 ACM records) stays missing.
Ten methods on the same pairs
Section titled “Ten methods on the same pairs”Every method sees the same 28,561 candidates and is measured on the same 549 test links. (So the rules differ a little from the numbers above, which were on all pairs.)
- Rules: the exact title, the normalised title, and title with year and venue.
- One score alone: the TF-IDF similarity, or the embedding cosine.
- Splink without labels, as in Learn the weights without labels. It compares the title (Jaro-Winkler levels), the shared surnames, the year and the venue.
- Logistic regression and LightGBM on the comparison vector, as in A supervised pair classifier. They learn from the 14,420 candidate pairs of the train split (1,116 true).
- LightGBM + title count: the fix that the error analysis below finds.
Text description of the chart
Ten dots, one per method, on a scale from 0.5 to 1. Exact title as written 0.574; rule same title 0.876; rule same title, year and venue 0.933; TF-IDF alone 0.904; embeddings alone 0.915; Splink without labels 0.957; logistic regression 0.96; LightGBM 0.981; LightGBM with the embedding cosine 0.985; LightGBM with the title count 0.99.
What this tells us. Every learned method beats the best rule (0.933). Splink without labels reaches 0.957, close to logistic regression (0.960). LightGBM reaches 0.981. The embeddings alone (0.915) beat TF-IDF alone (0.904), and as one more feature they add little (0.985).
Why Splink works well here
Section titled “Why Splink works well here”In the first case study, Splink learned a much too low m probability for model codes (0.048 instead of about 0.67). Here, its values make sense:
field level m u title Exact match on title 0.8526 0.0004 year Exact match on year 0.9298 0.1033 venue Exact match on venue 0.9801 0.2888The m probability is the share of true links that agree on a field. The u probability is the share of other pairs that agree by chance. About 85% of true links have the same normalised title, so m = 0.85 is right. The year m (0.93) is a little low: in this data, every true link has the same year, so the right value is 1.0.
With clean, separate fields, the two groups that expectation maximisation finds are “same paper” and “not the same paper”. Splink without labels is a good first linker when the fields are clean and separate.
The measure that Ruth needs
Section titled “The measure that Ruth needs”| Method | Precision on test at recall 0.95 |
|---|---|
| TF-IDF alone | 0.846 |
| embeddings alone | 0.89 |
| Splink, no labels | 0.971 |
| logistic regression | 0.985 |
| LightGBM | 0.981 |
| LightGBM + title count | 1 |
At recall 0.95, about 2 of LightGBM’s links in 100 are wrong (0.981); Ruth needs 995 in 1,000. The rules cannot reach recall 0.95 at all. To close the gap, we must know which pairs are wrong.
Error analysis
Section titled “Error analysis”What it is
Section titled “What it is”A doctor does not stop at “80% of my patients get better”. She looks at the patients who did not, one by one, and asks why. Error analysis does the same for a matcher. You take every wrong decision on the validation or test split. You give each one a reason that you can see in the records. Then you count the reasons. The biggest group tells you what to fix first.
LightGBM’s errors
Section titled “LightGBM’s errors”The notebook’s reason() function gives each error one reason. It checks them in this order:
- a title that more than two records share;
- a false link for a DBLP record that has no partner;
- titles that differ (token set ratio below 0.9);
- author lists that differ (less than half the surnames shared).
On the test split, at LightGBM’s best-F1 threshold, the blocking lost 1 true link. The other errors:
| Reason | False links | Missed links |
|---|---|---|
| A title that many papers share | 11 | 0 |
| No partner in ACM | 2 | 0 |
| The title differs | 1 | 1 |
| The author lists differ | 1 | 2 |
| Other | 1 | 1 |
What this tells us. 11 of the 16 false links come from titles that many papers share, the problem of the failure case. The other errors are few and varied, for example:
- A part of a series. DBLP has “Database Tuning: Principles, Experiments, and Troubleshooting Techniques” twice: once in VLDB 2002, and once as “(part II)” in SIGMOD 2002. ACM has “(part I)”. The model linked both DBLP records to part I.
- A panel. DBLP lists “Index Research: Forest or Trees? (Panel Abstract)” under one author; ACM lists six panellists and writes “(panel session)”. The model missed it.
Why common titles fool the model
Section titled “Why common titles fool the model”Text description of the diagram
Three DBLP records on the left, two ACM records on the right, with six lines between them. Two lines are thick and green: the true links. The other four are dashed. A note says that no field tells the pairs apart.
“Reminiscences on influential papers” is a column in SIGMOD Record. In 2001, DBLP lists the column’s editor (Kenneth A. Ross) and the contributors; ACM lists the editor only.
LightGBM puts 91% of its split gain on the three title similarities, and less than 2% on the authors. The model has no way to know that, for these few titles, the title is worthless.
The fix: tell the model how common the title is
Section titled “The fix: tell the model how common the title is”We add one field, title_shared: the number of records, in both libraries, with the same normalised title. Three in four DBLP records have a title that exactly 2 records share (one in each library). 134 DBLP records have a title that more than 2 records share. Now the model can learn: “a common title is weak evidence; look at the other fields”.
LightGBM + how common the title is: {'precision': 1.0, 'recall': 0.98, 'f1': 0.99, 'called': 538, 'right': 538, 'true': 549, ...}kind missed matcha title that many papers share 12other 1the author lists differ 1the title differs 1What this tells us. No false link is left in the test split: F1 0.990. The model now misses 12 links with a common title instead of making false ones, and for Ruth a missed link costs less. With logistic regression, the same field gives 0.988 (the notebook’s task).
Can a rule do the same? In the notebook’s task, you keep only the rule’s pairs that share at least one surname. The F1 stays at 0.932, because the column’s editor is in every record. No rule on these four fields can tell two issues of one column apart. Only the issue number could, and neither library gives it.
One record, one partner
Section titled “One record, one partner”From the first case study: a paper is in each library at most once. One-to-one matching keeps a pair only when it is the best pair of both its records. It raises the first LightGBM from 0.981 to 0.985. After the fix, it changes nothing (0.990), but we keep it as a cheap guard. Because many records have no partner, it always works with a threshold.
Decide: accept, review or no link
Section titled “Decide: accept, review or no link”The bands
Section titled “The bands”As in Human review, we cut the score into three bands. On the validation split, we tried three lower and three upper thresholds. The lower threshold changes nothing in this table (0.01, 0.05 and 0.2 give the same rows), so we show the rows for 0.05:
accepted accepted right precision accepted to review true in review recall if reviews are rightlower upper0.05 0.50 556.0 555.0 0.9982 1.0 0.0 0.9946 0.80 556.0 555.0 0.9982 1.0 0.0 0.9946 0.95 549.0 549.0 1.0000 8.0 6.0 0.9946The lowest upper threshold that meets Ruth’s rule (0.995) is 0.5. For the lower threshold, we take 0.05. On the test split, used once:
accepted accepted right precision accepted to review true in review recall if reviews are rightvalidation 556.0 555.0 0.9982 1.0 0.0 0.9946test 539.0 539.0 1.0000 10.0 3.0 0.9872What this tells us. On test, all 539 automatic links are right. 10 pairs go to a person, and 3 of them are true links. If the person decides them all correctly, the report has 98.7% of the true links.
Zero errors in 539 is not proof. The true error rate could still be about 0.6% (a rough rule for zero errors: 3 divided by the number of links), and Ruth’s rule allows 0.5%. So the team must keep checking a sample.
Text description of the diagram
Five steps from left to right, each with its main number: normalise, block, compare, classify, one-to-one. Under the last step, three boxes for the bands: accept, review and no link, each with its count of pairs.
What it costs
Section titled “What it costs”The prices are assumptions from the course’s price table; check the providers’ current pages.
| Step | Measured | Cost (assumption × measurement) |
|---|---|---|
| Compare and classify on a CPU (US$0.17 an hour) | 0.6 s for 28,561 pairs | about US$0.000001 per 1,000 candidate pairs |
| A person reviews the band (1.2 minutes a pair, US$40 an hour) | 15 pairs per 1,000 DBLP records | 0.31 hours, about US$12 per 1,000 records |
What this tells us. The review queue is small: about 15 pairs for 1,000 records, against 115 for 1,000 products in the first case study. We used no language model: the band is small, and most band pairs have the same text in every field, so a model could not decide them either.
The design on new input
Section titled “The design on new input”The notebook’s link_records() runs every step, from blocking to the bands. We run it on the 654 DBLP records of the test split, as if they were new, against all of ACM:
654 new records, 0.5 s: {'accept': 539, 'review': 11} count truedecisionaccept 539 539review 11 3What this tells us. The automatic links are the same as in the evaluation. The review queue has one more pair, because an ACM record now chooses its top 5 only among the 654 new records.
The review queue is the most useful output. Here it is, shortened:
score same DBLP ACM 0.365 True Book review column Book review column 0.365 True Book Review Column Book review column 0.365 False Book Review Column Book review column ... (7 more "Book Review Column" pairs, 1 true) 0.085 False Database Tuning: Principles, Experiments, and Troubleshooting Techniques Database tuning: principles, experiments, and troubleshooting techniques (part I)What this tells us. 10 of the 11 pairs are one column, “Book Review Column”, with the same author, journal and year. The records are the same in every field, so the scores are equal, and one-to-one keeps them all. A person needs the issue number or the pages, from the library’s web page. A model can only use the fields that it gets.
Compared with the first case study and with Larkfield
Section titled “Compared with the first case study and with Larkfield”Text description of the chart
Five dots. TermLarkfieldA made-up Canadian online shop for home and garden products. It is not a real company. All Nextia courses use its help desk and data as the example.Learn more Meet Larkfield products, synthetic, p-lgbm: 0.996. DBLP–ACM, the chosen design: 0.99. Abt–Buy, LightGBM with one-to-one: 0.924. Larkfield customers, synthetic, lgbm: 0.902. Amazon–Google with the first case study’s design: 0.633.
What this tells us. DBLP–ACM is the easiest real problem of the course, close to Larkfield’s synthetic supplier feeds: the fields are separate and clean, and libraries curate their records. On Larkfield’s customers, LightGBM reached 0.902, because people share a household, an address and a surname. Free-text product titles on Amazon–Google were harder still.
Recommendation
Section titled “Recommendation”Daniel writes to Ruth in her words:
- Use the design above. Link automatically when the score is 0.5 or more.
- Review the band from 0.05 to 0.5: less than 20 minutes for every 1,000 DBLP records, mostly regular columns.
- For columns, open the record’s page and compare the issue number or the pages. Better: ask both libraries for the DOI (the paper’s permanent ID) or the issue, and add it as a field.
- Check a sample of automatic links every year, for example 200.
- Count the records without a partner. 292 of the 392 DBLP records without one are from 2002 and 2003. Before you report a paper as “only in DBLP”, check that ACM really does not have it.
What not to do:
- Do not use these thresholds for other libraries without new labels and a new validation. A preprint server has much messier records.
- Do not trust a title alone. It was the best evidence, and also the cause of most false links.
Insights and limits
Section titled “Insights and limits”- Clean data changes the question. On products, it was “which method works at all?”. Here, it is “which few pairs does it get wrong?”. Error analysis matters more than the choice of model.
- Rules did well, but not well enough. No rule on these fields reached Ruth’s 995 in 1,000.
- Splink without labels was good here, and weak on free-text product titles.
- Old and narrow. The records are database papers from five venues, 1994 to 2003. Today’s records have DOIs, which would make most of this work unnecessary.
- The test split is small. It has 549 true links. One error moves the precision by about 0.2 points.
- Labels can be wrong. People made Leipzig’s links. Some column pairs may be wrong in the benchmark itself.
- The prices are assumptions. Check the providers’ current pages.
Responsible use
Section titled “Responsible use”- Credit the source. CC BY 4.0 requires the attribution in the box above wherever you use or share the data.
- The authors are real people. Their names are in the records because they published the papers. This case study links papers, not people, and says nothing about any person. Linking authors (deciding that two names are one person) is a different task. It needs the care of Privacy and matching: a wrong author link gives a person someone else’s work.
- A report can affect careers. A missing paper in a university report can hurt a researcher. That is why Ruth’s rule is strict, and why researchers should be able to check and correct their list.
Make it a portfolio piece
Section titled “Make it a portfolio piece”This case study shows that you can find the cause of errors on real records and fix it.
A larger extension, with two CC0 sources. Take a slice of dblp (its data is CC0, see the dblp blog), for example one conference over five years. Find the same papers in OpenAlex (CC0) through its free API. Many records in both have a DOI. Hide it from your matcher, link on title, authors, venue and year, and use the DOI as the ground truth. Keep the slice small (a few thousand records), and write down the date and the query.
What to put in the repository:
README.md(template below).- The notebook, run from top to bottom, with its outputs.
- The comparison table as a CSV: every method, its threshold, precision, recall, F1 and precision at recall 0.95.
- The error table before and after your fix, with one example per reason.
DECISION.md: the thresholds, the review band, its size and its cost, and the assumptions.- The data source and licence, with the attribution text.
A README template:
# Linking bibliographic records from two libraries
## ProblemA research office lists every paper of its researchers from DBLP and the ACM Digital Library.Links go into the report automatically only when 995 in 1,000 are right; a person checks the rest.
## Data and licenceDBLP-ACM benchmark, University of Leipzig (Köpcke, Thor and Rahm, PVLDB 2010), CC BY 4.0,https://dbs.uni-leipzig.de/research/projects/benchmark-datasets-for-entity-resolution.2,616 + 2,294 records, 2,224 true links. Split 50/25/25 by group of records.
## ApproachNormalised titles, surnames and venue keys; blocking with TF-IDF top 5 and embeddings top 5(pair completeness 0.999, 28,561 of 6 million pairs); 10 methods compared; error analysis;one new field (how many records share the title); LightGBM with one-to-one matching.
## Results (test split)F1 0.990 (rule title + year + venue 0.933, Splink without labels 0.957, LightGBM 0.981).Accept >= 0.5: 539 links, 539 right. Review 0.05-0.5: 10 pairs.
## LimitsOld, clean records from five venues. Regular columns with the same title, author, venue andyear cannot be told apart without the issue number.How to describe it in an interview, in two sentences: “I linked real bibliographic records from two libraries, and error analysis showed that most false links came from titles that many papers share. One new field, how common the title is, removed every false link in the test split; the review queue holds the records that no field can tell apart.”
Go further:
- Run the same design on DBLP–Scholar, from the same Leipzig page (CC BY 4.0): 64,263 Google Scholar records, much messier. You will need a tighter blocking.
- Add a field for words such as “panel”, “tutorial”, “demonstration” or “part I”, and look at the error table again.
- Replace “best of both” with an optimal assignment (
scipy.optimize.linear_sum_assignment), and compare.
Knowledge check
Section titled “Knowledge check”Check your understanding
The rule “same normalised title, year and venue” reached F1 0.933 here, but the best rule on Abt–Buy reached only 0.791. Give two reasons from the data.
Show the answer
First, the bibliographic records have separate, clean fields: the year always agrees for a true link, and the venues map one to one. So the rule can check three independent fields. The product data had one free-text title, and the evidence (the model code) was written in many ways inside it. Second, the libraries curate their titles: after normalising, about 90% of the test links have the same title. Shops write titles for customers, in their own words, so equal titles were rare.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Splink without labels learned an m probability of 0.85 for “exact match on title” here, but 0.048 for “share a model code” on products. Why did expectation maximisation do well here?
Show the answer
Expectation maximisation looks for two groups of pairs that explain the agreements best. Here, the fields are clean and separate, and true links agree on almost all of them (title, year, venue, surnames), while other pairs rarely do. So the two groups it finds are “same paper” and “not the same paper”. On products, the evidence was mixed in free text. Our reading in the first case study: pairs of one brand and one product line probably formed another group that shared many words, and EM found that group instead.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
After the fix, the review queue had 10 “Book Review Column” pairs with the same score. Daniel says: “Train a better model.” What do you answer?
Show the answer
A better model will not help. The records are the same in every field that the model gets: title, author, venue and year. No model can tell them apart from these fields. The fix is more evidence: the issue number, the pages or the DOI. Until a library gives it, a person decides these pairs from the record’s web page. Error analysis tells you when the problem is the data, not the model.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- On clean bibliographic records, one rule (normalised title, year and venue) already reached F1 0.933, with no training.
- Splink without labels reached 0.957, because the fields are clean and separate. LightGBM reached 0.981.
- Error analysis showed that 11 of LightGBM’s 16 false links came from titles that many papers share.
- One new field, how common the title is, removed every false link in the test split (F1 0.990, precision 1.0).
- The decision: accept at 0.5 or more (539 of 539 right on test). Review from 0.05 to 0.5: about 15 pairs per 1,000 records. Regular columns need a field that the libraries do not give.
Next, people and companies: duplicates inside one list, and companies linked to an open register.