Skip to content
Français

Link bibliographic records

  • Case study 2
  • 120 min
  • Optional · 1 case study needed for the certificate

Where we are

In eight modules, we matched customers and products. In this case study, we link real bibliographic records from two open sources, compare Splink with a pair classifier, and study the errors in titles and author lists.

What you will be able to do

Link real bibliographic records from two libraries, from the raw files to a decision that a research office can use. You will normalise titles, authors and venues, block, compare, classify and measure, as in the course. You will compare rules, Splink and two classifiers under one measure. Then you study every error, fix the main one, and choose a design with its thresholds and its review band.

Case studyPortfolio piece

Licence
CC BY 4.0
Time
About 2 hours
Runs in
Colab, Kaggle or VS Code
Skills used
  • Normalise titles, authors and venues
  • Blocking with TF-IDF and embeddings
  • Splink without labels
  • LightGBM on a comparison vector
  • Error analysis
  • A field for common titles
  • Thresholds and a review band

Colab and Kaggle open your own copy. To use VS Code, download the notebook from GitHub. Stuck? See the solution notebook.

The notebook runs every step of this page, with the same code and the same output, plus three tasks with checks. Its setup cells install the course’s packages and download about 2 MB of data. The benchmark comes from its source page; the embeddings come from the course’s labs. It needs no account and no key. After the install, it runs in under a minute.

Before you startWhat to know and to have readyAbout 120 minutes

What you need to know

This case study uses the whole course, and it follows the pattern of Match products between two shops. These lessons matter most:

What you need on your computer (only for the local route; the notebook needs none of it)

  • A virtual environment with the course’s packages from Module 6, plus Splink 5.0.0, DuckDB 2.0.0 and pyarrow 26.0.1 from Module 5. The environment of the larkfield-match project has all of them. This case study does not change the project’s files.
  • The notebook, downloaded from the link above, in VS Code or JupyterLab. Its setup cell makes a new folder bibliographic-records next to it. To start again, delete that folder.

Tested on macOS with Python 3.12.13, in Jupyter from a fresh start. Windows and Linux are not tested.

A library may have the same book twice: one card from the old catalogue, one from a donation list. The cards are not the same. One says “2nd ed.”, the other writes the author’s first name in full. A librarian decides that they are one book.

Ruth runs the research-information team of a university. Daniel is her data analyst. Each year, the team lists every paper that the university’s computer scientists published. It takes the records from two public libraries: DBLP, a free index of computer-science papers, and the ACM Digital Library. Many papers are in both. (Ruth, Daniel and the university are invented. The two libraries and their records are real.)

Each error has a price:

  • A false link joins two different papers. One paper disappears from the report, and a researcher complains.
  • A missed link counts one paper twice. The report is too high, and a reviewer may find it.

Ruth says: “A missing paper is the worse error. People notice it at once.” So she sets a rule: links go into the report automatically only when at least 995 in 1,000 of them are right. A person checks the unsure ones.

You will build a linker that reads the two libraries. It puts every candidate pair in one of three groups: accept, review or no link. You will compare ten methods on the same pairs with the same measures, look at every error, and choose one design. Then you run that design on records that no step has seen.

What is different from the first case study. These records are much cleaner than product titles, with separate fields: title, authors, venue and year. So a simple rule may already do well. The interesting question is: where does it still fail, and why?

The data is a public benchmark, from the database group of the University of Leipzig, like the shops of the first case study. It has 2,616 DBLP records and 2,294 ACM records of papers from five database conferences and journals, 1994 to 2003. A third file lists 2,224 true links: pairs of records that are the same paper. (The source page says 2,614 DBLP records; the file has 2,616 rows.)

Column What it means Use
id The record’s key in its library (conf/sigmod/FranklinJK96 in DBLP, a number in ACM) Key
title The title, as the library wrote it Normalised, compared, embedded
authors The author names, separated by commas Surnames compared
venue The conference or journal Mapped to one key
year The year of publication Compared
idDBLP, idACM One true link per row (the third file) Labels only, never an input
print("records with a match:", f"DBLP {dblp.id.isin(matches.id_a).sum()} of {len(dblp)},",
f"ACM {acm.id.isin(matches.id_b).sum()} of {len(acm)}")
print("true matches with the same year:", f"{(pairs_true.year_a == pairs_true.year_b).mean():.0%}")
print("true matches with the same title, as written:", f"{(pairs_true.title_a == pairs_true.title_b).mean():.0%}")
print("true matches with the same author list, as written:", f"{(pairs_true.authors_a == pairs_true.authors_b).mean():.0%}")

The output of the notebook’s cell, which also counts repeated IDs and empty author lists:

repeated IDs: 0 0
no authors: DBLP 0, ACM 14
records with a match: DBLP 2224 of 2616, ACM 2224 of 2294
records with two or more matches: DBLP 0, ACM 0
true matches with the same year: 100%
true matches with the same title, as written: 41%
true matches with the same author list, as written: 28%

What this tells us. The year always agrees, but titles and author lists, as written, often differ. And not every record has a partner: 392 DBLP records and 70 ACM records have none. In the first case study, every product had one.

One paper in DBLP and ACM. Title: Performance Tradeoffs for Client-Server Query Processing, and Performance tradeoffs for client-server query processing. Authors: Donald Kossmann, Michael J. Franklin, Björn Þór Jónsson in DBLP; Michael J. Franklin, Björn Thór Jónsson, Donald Kossmann in ACM, with ö and ó written as HTML codes. Venue: SIGMOD Conference, and International Conference on Management of Data. Year 1996 in both. After normalising, the titles, the surnames (franklin, jonsson, kossmann), the venue key (sigmod) and the year agree.
One paper, two records. Capitals, author order, HTML codes and venue names differ. After normalising, all four fields agree.
Text description of the diagram

Four columns: DBLP and ACM as written, then DBLP and ACM after normalising. Four rows: title, authors, venue and year. The cells after normalising are green, because the two records agree in every field.

The differences are typical of this data:

  • Capitals and signs. “Tradeoffs” against “tradeoffs”, “PESTO :” against “PESTO:”.
  • Author order. DBLP and ACM often list the same authors in another order.
  • HTML codes. ACM writes letters such as ö as ö. DBLP writes “Þór”, ACM Thór.
  • Venue names. DBLP writes “SIGMOD Conference”, ACM “International Conference on Management of Data”. In this data, each DBLP venue goes with exactly one ACM venue.
  • Typing errors. A few ACM titles have one: “tumble” for “tunable”, “databates” for “databases”.
  • The unit is a pair: one DBLP record and one ACM record. The target is “same paper or not”, from Leipzig’s list.
  • The split puts each record in train (50%), validation (25%) or test (25%), by group, as in the first case study. A pair belongs to the split of its DBLP record. The test split has 549 true links. The split, its shares and its seed (39) are our choice.
  • The measures. We measure every method on the test split, with a threshold chosen on validation. We report pair precision, recall and F1, as in Measure a matcher. Ruth’s rule needs a second measure: the precision at recall 0.95. We take the threshold that finds 95% of the validation links, and report the precision at that threshold on test.
  • The labels are Leipzig’s. We made no label ourselves. The 995-in-1,000 rule, the reviewer’s time and the prices are our assumptions.

The exact title, as written, finds 230 of the 549 test links (precision 0.909, recall 0.419). So we normalise first, as in Normalise:

  • normalise() decodes HTML codes, writes lower case, drops accents, signs and apostrophes, and joins hyphenated words (“Client-Server” becomes “clientserver”).
  • surnames() keeps the last word of each author name.
  • VENUE_KEYS, a small table written by hand, gives both names of a venue one key, such as sigmod.

Then two rules:

same normalised title: {'precision': 0.853, 'recall': 0.896, 'f1': 0.874, 'called': 577, 'right': 492, 'true': 549}
same normalised title, year, venue: {'precision': 0.972, 'recall': 0.896, 'f1': 0.933, 'called': 506, 'right': 492, 'true': 549}

What this tells us. On clean records, one rule already reaches F1 0.933: the year and the venue remove most false links of the title alone. On products, the best rule reached 0.791.

Common mistake

A title that many papers share

Symptom: the rule “same normalised title” made 255 false links in the whole data.

Cause: some titles are not the title of one paper. They are the name of a regular column. The most common titles in DBLP are “Editor’s notes” (30 records), “Reminiscences on influential papers” (19), “Chair’s message” (18) and “Book review column” (7). Each such title joins every record of one library to every record of the other. These four titles made 169 of the 255 false links.

Fix: add the year and the venue to the rule. That removes most of them, but not all: one journal has many columns with the same title in one year.

Consequence in AI work: before you trust a key, look at its most common values, as in What blocking loses. In the first case study, a screen size (1080p) looked like a model number. Here, a column name looks like a paper title.

The two libraries make 2,616 × 2,294 = 6,001,104 pairs. We compare three blockings:

candidates true kept pair completeness reduction ratio
same year and venue 157871.0 2224.0 1.0000 0.97369
TF-IDF top 5 17638.0 2219.0 0.9978 0.99706
embeddings top 5 17860.0 2215.0 0.9960 0.99702
TF-IDF and embeddings 28561.0 2221.0 0.9987 0.99524
  • Same year and venue keeps every true link, only because this data is clean. A preprint and its journal version have different years and venues.
  • TF-IDF top 5 and embeddings top 5 keep, for each record, the 5 most similar titles of the other library, in both directions, as in the first case study. TF-IDF compares short pieces of 3 to 5 letters and gives rare pieces more weight. The search and ranking course uses the same idea: see BM25.

What this tells us. TF-IDF and embeddings together keep 2,221 of 2,224 true links with 28,561 candidates, about one fifth of the year-and-venue blocking. We use them. In the notebook, one neighbour instead of five gives 3,117 candidates and still keeps 99.1% of the true links.

Each candidate pair gets eight similarities, as in Compare by field, then combine, plus the embedding cosine for one test:

  • Title: token set ratio, Jaro-Winkler, and the TF-IDF similarity.
  • Authors: the share of shared surnames (Jaccard), and the token sort ratio of the two author strings, which ignores the order. Also the difference in the number of authors.
  • Year and venue: the year difference, and “same venue key”.

An empty author list (14 ACM records) stays missing.

Every method sees the same 28,561 candidates and is measured on the same 549 test links. (So the rules differ a little from the numbers above, which were on all pairs.)

  • Rules: the exact title, the normalised title, and title with year and venue.
  • One score alone: the TF-IDF similarity, or the embedding cosine.
  • Splink without labels, as in Learn the weights without labels. It compares the title (Jaro-Winkler levels), the shared surnames, the year and the venue.
  • Logistic regression and LightGBM on the comparison vector, as in A supervised pair classifier. They learn from the 14,420 candidate pairs of the train split (1,116 true).
  • LightGBM + title count: the fix that the error analysis below finds.
F1 of every method on the same candidate pairs of DBLP–ACM. On clean records, every method is good, and the differences are small. The last row is the fix from the error analysis.
Text description of the chart

Ten dots, one per method, on a scale from 0.5 to 1. Exact title as written 0.574; rule same title 0.876; rule same title, year and venue 0.933; TF-IDF alone 0.904; embeddings alone 0.915; Splink without labels 0.957; logistic regression 0.96; LightGBM 0.981; LightGBM with the embedding cosine 0.985; LightGBM with the title count 0.99.

What this tells us. Every learned method beats the best rule (0.933). Splink without labels reaches 0.957, close to logistic regression (0.960). LightGBM reaches 0.981. The embeddings alone (0.915) beat TF-IDF alone (0.904), and as one more feature they add little (0.985).

In the first case study, Splink learned a much too low m probability for model codes (0.048 instead of about 0.67). Here, its values make sense:

field level m u
title Exact match on title 0.8526 0.0004
year Exact match on year 0.9298 0.1033
venue Exact match on venue 0.9801 0.2888

The m probability is the share of true links that agree on a field. The u probability is the share of other pairs that agree by chance. About 85% of true links have the same normalised title, so m = 0.85 is right. The year m (0.93) is a little low: in this data, every true link has the same year, so the right value is 1.0.

With clean, separate fields, the two groups that expectation maximisation finds are “same paper” and “not the same paper”. Splink without labels is a good first linker when the fields are clean and separate.

Method Precision on test at recall 0.95
TF-IDF alone 0.846
embeddings alone 0.89
Splink, no labels 0.971
logistic regression 0.985
LightGBM 0.981
LightGBM + title count 1

At recall 0.95, about 2 of LightGBM’s links in 100 are wrong (0.981); Ruth needs 995 in 1,000. The rules cannot reach recall 0.95 at all. To close the gap, we must know which pairs are wrong.

A doctor does not stop at “80% of my patients get better”. She looks at the patients who did not, one by one, and asks why. Error analysis does the same for a matcher. You take every wrong decision on the validation or test split. You give each one a reason that you can see in the records. Then you count the reasons. The biggest group tells you what to fix first.

The notebook’s reason() function gives each error one reason. It checks them in this order:

  1. a title that more than two records share;
  2. a false link for a DBLP record that has no partner;
  3. titles that differ (token set ratio below 0.9);
  4. author lists that differ (less than half the surnames shared).

On the test split, at LightGBM’s best-F1 threshold, the blocking lost 1 true link. The other errors:

Reason False links Missed links
A title that many papers share 11 0
No partner in ACM 2 0
The title differs 1 1
The author lists differ 1 2
Other 1 1

What this tells us. 11 of the 16 false links come from titles that many papers share, the problem of the failure case. The other errors are few and varied, for example:

  • A part of a series. DBLP has “Database Tuning: Principles, Experiments, and Troubleshooting Techniques” twice: once in VLDB 2002, and once as “(part II)” in SIGMOD 2002. ACM has “(part I)”. The model linked both DBLP records to part I.
  • A panel. DBLP lists “Index Research: Forest or Trees? (Panel Abstract)” under one author; ACM lists six panellists and writes “(panel session)”. The model missed it.
Reminiscences on influential papers, SIGMOD Record, 2001. DBLP has three records with different author lists, each with Kenneth A. Ross. ACM has two records, each with the author Kenneth A. Ross only. Six candidate pairs; two are true links; one DBLP record has no partner. Every pair has the same title, year and venue and shares the surname Ross.
A title that many papers share: six pairs agree in every field, and only two are true links.
Text description of the diagram

Three DBLP records on the left, two ACM records on the right, with six lines between them. Two lines are thick and green: the true links. The other four are dashed. A note says that no field tells the pairs apart.

“Reminiscences on influential papers” is a column in SIGMOD Record. In 2001, DBLP lists the column’s editor (Kenneth A. Ross) and the contributors; ACM lists the editor only.

LightGBM puts 91% of its split gain on the three title similarities, and less than 2% on the authors. The model has no way to know that, for these few titles, the title is worthless.

The fix: tell the model how common the title is

Section titled “The fix: tell the model how common the title is”

We add one field, title_shared: the number of records, in both libraries, with the same normalised title. Three in four DBLP records have a title that exactly 2 records share (one in each library). 134 DBLP records have a title that more than 2 records share. Now the model can learn: “a common title is weak evidence; look at the other fields”.

LightGBM + how common the title is: {'precision': 1.0, 'recall': 0.98, 'f1': 0.99, 'called': 538, 'right': 538, 'true': 549, ...}
kind missed match
a title that many papers share 12
other 1
the author lists differ 1
the title differs 1

What this tells us. No false link is left in the test split: F1 0.990. The model now misses 12 links with a common title instead of making false ones, and for Ruth a missed link costs less. With logistic regression, the same field gives 0.988 (the notebook’s task).

Can a rule do the same? In the notebook’s task, you keep only the rule’s pairs that share at least one surname. The F1 stays at 0.932, because the column’s editor is in every record. No rule on these four fields can tell two issues of one column apart. Only the issue number could, and neither library gives it.

From the first case study: a paper is in each library at most once. One-to-one matching keeps a pair only when it is the best pair of both its records. It raises the first LightGBM from 0.981 to 0.985. After the fix, it changes nothing (0.990), but we keep it as a cheap guard. Because many records have no partner, it always works with a threshold.

As in Human review, we cut the score into three bands. On the validation split, we tried three lower and three upper thresholds. The lower threshold changes nothing in this table (0.01, 0.05 and 0.2 give the same rows), so we show the rows for 0.05:

accepted accepted right precision accepted to review true in review recall if reviews are right
lower upper
0.05 0.50 556.0 555.0 0.9982 1.0 0.0 0.9946
0.80 556.0 555.0 0.9982 1.0 0.0 0.9946
0.95 549.0 549.0 1.0000 8.0 6.0 0.9946

The lowest upper threshold that meets Ruth’s rule (0.995) is 0.5. For the lower threshold, we take 0.05. On the test split, used once:

accepted accepted right precision accepted to review true in review recall if reviews are right
validation 556.0 555.0 0.9982 1.0 0.0 0.9946
test 539.0 539.0 1.0000 10.0 3.0 0.9872

What this tells us. On test, all 539 automatic links are right. 10 pairs go to a person, and 3 of them are true links. If the person decides them all correctly, the report has 98.7% of the true links.

Zero errors in 539 is not proof. The true error rate could still be about 0.6% (a rough rule for zero errors: 3 divided by the number of links), and Ruth’s rule allows 0.5%. So the team must keep checking a sample.

The chosen design. Normalise titles, surnames and venue keys; block with TF-IDF top 5 and embeddings top 5 (28,561 of 6,001,104 pairs, 2,221 of 2,224 true kept); compare 9 fields, including how common the title is; classify with LightGBM; one-to-one. Then three bands on the test split: score 0.5 or more, accept, 539 links, 539 right; 0.05 to 0.5, a person reviews, 10 pairs, 3 true; below 0.05, no link.
The design that the case study recommends, with the counts of the test split.
Text description of the diagram

Five steps from left to right, each with its main number: normalise, block, compare, classify, one-to-one. Under the last step, three boxes for the bands: accept, review and no link, each with its count of pairs.

The prices are assumptions from the course’s price table; check the providers’ current pages.

Step Measured Cost (assumption × measurement)
Compare and classify on a CPU (US$0.17 an hour) 0.6 s for 28,561 pairs about US$0.000001 per 1,000 candidate pairs
A person reviews the band (1.2 minutes a pair, US$40 an hour) 15 pairs per 1,000 DBLP records 0.31 hours, about US$12 per 1,000 records

What this tells us. The review queue is small: about 15 pairs for 1,000 records, against 115 for 1,000 products in the first case study. We used no language model: the band is small, and most band pairs have the same text in every field, so a model could not decide them either.

The notebook’s link_records() runs every step, from blocking to the bands. We run it on the 654 DBLP records of the test split, as if they were new, against all of ACM:

654 new records, 0.5 s: {'accept': 539, 'review': 11}
count true
decision
accept 539 539
review 11 3

What this tells us. The automatic links are the same as in the evaluation. The review queue has one more pair, because an ACM record now chooses its top 5 only among the 654 new records.

The review queue is the most useful output. Here it is, shortened:

score same DBLP ACM
0.365 True Book review column Book review column
0.365 True Book Review Column Book review column
0.365 False Book Review Column Book review column
... (7 more "Book Review Column" pairs, 1 true)
0.085 False Database Tuning: Principles, Experiments, and Troubleshooting Techniques Database tuning: principles, experiments, and troubleshooting techniques (part I)

What this tells us. 10 of the 11 pairs are one column, “Book Review Column”, with the same author, journal and year. The records are the same in every field, so the scores are equal, and one-to-one keeps them all. A person needs the issue number or the pages, from the library’s web page. A model can only use the fields that it gets.

Compared with the first case study and with Larkfield

Section titled “Compared with the first case study and with Larkfield”
The course's methods on five problems: Larkfield's synthetic products and customers (from Module 6), and three real benchmarks.
Text description of the chart

Five dots. TermLarkfieldA made-up Canadian online shop for home and garden products. It is not a real company. All Nextia courses use its help desk and data as the example.Learn more Meet Larkfield products, synthetic, p-lgbm: 0.996. DBLP–ACM, the chosen design: 0.99. Abt–Buy, LightGBM with one-to-one: 0.924. Larkfield customers, synthetic, lgbm: 0.902. Amazon–Google with the first case study’s design: 0.633.

What this tells us. DBLP–ACM is the easiest real problem of the course, close to Larkfield’s synthetic supplier feeds: the fields are separate and clean, and libraries curate their records. On Larkfield’s customers, LightGBM reached 0.902, because people share a household, an address and a surname. Free-text product titles on Amazon–Google were harder still.

Daniel writes to Ruth in her words:

  • Use the design above. Link automatically when the score is 0.5 or more.
  • Review the band from 0.05 to 0.5: less than 20 minutes for every 1,000 DBLP records, mostly regular columns.
  • For columns, open the record’s page and compare the issue number or the pages. Better: ask both libraries for the DOI (the paper’s permanent ID) or the issue, and add it as a field.
  • Check a sample of automatic links every year, for example 200.
  • Count the records without a partner. 292 of the 392 DBLP records without one are from 2002 and 2003. Before you report a paper as “only in DBLP”, check that ACM really does not have it.

What not to do:

  • Do not use these thresholds for other libraries without new labels and a new validation. A preprint server has much messier records.
  • Do not trust a title alone. It was the best evidence, and also the cause of most false links.
  • Clean data changes the question. On products, it was “which method works at all?”. Here, it is “which few pairs does it get wrong?”. Error analysis matters more than the choice of model.
  • Rules did well, but not well enough. No rule on these fields reached Ruth’s 995 in 1,000.
  • Splink without labels was good here, and weak on free-text product titles.
  • Old and narrow. The records are database papers from five venues, 1994 to 2003. Today’s records have DOIs, which would make most of this work unnecessary.
  • The test split is small. It has 549 true links. One error moves the precision by about 0.2 points.
  • Labels can be wrong. People made Leipzig’s links. Some column pairs may be wrong in the benchmark itself.
  • The prices are assumptions. Check the providers’ current pages.
  • Credit the source. CC BY 4.0 requires the attribution in the box above wherever you use or share the data.
  • The authors are real people. Their names are in the records because they published the papers. This case study links papers, not people, and says nothing about any person. Linking authors (deciding that two names are one person) is a different task. It needs the care of Privacy and matching: a wrong author link gives a person someone else’s work.
  • A report can affect careers. A missing paper in a university report can hurt a researcher. That is why Ruth’s rule is strict, and why researchers should be able to check and correct their list.

This case study shows that you can find the cause of errors on real records and fix it.

A larger extension, with two CC0 sources. Take a slice of dblp (its data is CC0, see the dblp blog), for example one conference over five years. Find the same papers in OpenAlex (CC0) through its free API. Many records in both have a DOI. Hide it from your matcher, link on title, authors, venue and year, and use the DOI as the ground truth. Keep the slice small (a few thousand records), and write down the date and the query.

What to put in the repository:

  • README.md (template below).
  • The notebook, run from top to bottom, with its outputs.
  • The comparison table as a CSV: every method, its threshold, precision, recall, F1 and precision at recall 0.95.
  • The error table before and after your fix, with one example per reason.
  • DECISION.md: the thresholds, the review band, its size and its cost, and the assumptions.
  • The data source and licence, with the attribution text.

A README template:

# Linking bibliographic records from two libraries
## Problem
A research office lists every paper of its researchers from DBLP and the ACM Digital Library.
Links go into the report automatically only when 995 in 1,000 are right; a person checks the rest.
## Data and licence
DBLP-ACM benchmark, University of Leipzig (Köpcke, Thor and Rahm, PVLDB 2010), CC BY 4.0,
https://dbs.uni-leipzig.de/research/projects/benchmark-datasets-for-entity-resolution.
2,616 + 2,294 records, 2,224 true links. Split 50/25/25 by group of records.
## Approach
Normalised titles, surnames and venue keys; blocking with TF-IDF top 5 and embeddings top 5
(pair completeness 0.999, 28,561 of 6 million pairs); 10 methods compared; error analysis;
one new field (how many records share the title); LightGBM with one-to-one matching.
## Results (test split)
F1 0.990 (rule title + year + venue 0.933, Splink without labels 0.957, LightGBM 0.981).
Accept >= 0.5: 539 links, 539 right. Review 0.05-0.5: 10 pairs.
## Limits
Old, clean records from five venues. Regular columns with the same title, author, venue and
year cannot be told apart without the issue number.

How to describe it in an interview, in two sentences: “I linked real bibliographic records from two libraries, and error analysis showed that most false links came from titles that many papers share. One new field, how common the title is, removed every false link in the test split; the review queue holds the records that no field can tell apart.”

Go further:

  • Run the same design on DBLP–Scholar, from the same Leipzig page (CC BY 4.0): 64,263 Google Scholar records, much messier. You will need a tighter blocking.
  • Add a field for words such as “panel”, “tutorial”, “demonstration” or “part I”, and look at the error table again.
  • Replace “best of both” with an optimal assignment (scipy.optimize.linear_sum_assignment), and compare.

Check your understanding

The rule “same normalised title, year and venue” reached F1 0.933 here, but the best rule on Abt–Buy reached only 0.791. Give two reasons from the data.

Show the answer

First, the bibliographic records have separate, clean fields: the year always agrees for a true link, and the venues map one to one. So the rule can check three independent fields. The product data had one free-text title, and the evidence (the model code) was written in many ways inside it. Second, the libraries curate their titles: after normalising, about 90% of the test links have the same title. Shops write titles for customers, in their own words, so equal titles were rare.

Check your understanding

Splink without labels learned an m probability of 0.85 for “exact match on title” here, but 0.048 for “share a model code” on products. Why did expectation maximisation do well here?

Show the answer

Expectation maximisation looks for two groups of pairs that explain the agreements best. Here, the fields are clean and separate, and true links agree on almost all of them (title, year, venue, surnames), while other pairs rarely do. So the two groups it finds are “same paper” and “not the same paper”. On products, the evidence was mixed in free text. Our reading in the first case study: pairs of one brand and one product line probably formed another group that shared many words, and EM found that group instead.

Check your understanding

After the fix, the review queue had 10 “Book Review Column” pairs with the same score. Daniel says: “Train a better model.” What do you answer?

Show the answer

A better model will not help. The records are the same in every field that the model gets: title, author, venue and year. No model can tell them apart from these fields. The fix is more evidence: the issue number, the pages or the DOI. Until a library gives it, a person decides these pairs from the record’s web page. Error analysis tells you when the problem is the data, not the model.

Recap

  • On clean bibliographic records, one rule (normalised title, year and venue) already reached F1 0.933, with no training.
  • Splink without labels reached 0.957, because the fields are clean and separate. LightGBM reached 0.981.
  • Error analysis showed that 11 of LightGBM’s 16 false links came from titles that many papers share.
  • One new field, how common the title is, removed every false link in the test split (F1 0.990, precision 1.0).
  • The decision: accept at 0.5 or more (539 of 539 right on test). Review from 0.05 to 0.5: about 15 pairs per 1,000 records. Regular columns need a field that the libraries do not give.

Next, people and companies: duplicates inside one list, and companies linked to an open register.

Next: Find duplicate people and companies