Skip to content
Français

What blocking loses

  • Lesson 11
  • 35 min
  • Module 3: Blocking: do not compare everything

Where we are

We now have several ways to make candidate pairs. Each way can lose true pairs. In this lesson, we measure pair completeness and the reduction ratio, and we choose keys that also catch the customers who moved.

What you will be able to do

Measure what a blocking loses, with pair completeness and the reduction ratio. Find which true pairs a key loses, and choose keys that also catch TermLarkfieldA made-up Canadian online shop for home and garden products. It is not a real company. All Nextia courses use its help desk and data as the example.Learn more Meet Larkfield’s customers who moved.

NotebookFind what a blocking loses in a notebookColab · Kaggle · GitHub

The notebook measures the five keys on Larkfield’s records. You list the true pairs that the postcode loses, find what they have in common, and measure a blocking that catches them. Its setup cells download the data, so it needs no local setup.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 35 minutes

What you need to know

What you need on your computer

  • The project larkfield-match with the blocking files of this module. Or use the notebook only.

A fisher chooses the mesh of her net. A wide mesh lets the small fish swim out: the net is light, and she sorts the catch fast. A fine mesh keeps almost every fish, but the net is heavy with weed and stones, and sorting takes all day.

She cannot judge the net by the weight of the catch alone. She must also know which fish got away. If every fish that got away is of one kind, the net has a hole in one place. She can mend that place.

Blocking is the net. The candidate pairs are the catch. The true pairs that the blocking removes are the fish that got away, and they never come back.

The problem: “blocking is only about speed”

Section titled “The problem: “blocking is only about speed””

TermTomásLarkfield’s AI engineer. He often suggests a quick shortcut, and the lesson shows why it fails. has a fast matcher in mind. “Blocking is only about speed,” he says. “It removes pairs that are clearly different. It cannot lose matches.”

He blocks on the postcode. The candidate pairs fall from 1.8 billion to 89,630. A later model scores every candidate pair very well. Then TermOmarLarkfield’s product manager. He asks the business questions: what a change costs, and what it gives the shop. counts the customers. Many customers who moved still have two records. Every model that Tomás tries gives the same missed matches.

The model is not the problem. The postcode blocking removed those pairs before any model saw them. A missed match in blocking is a missed match of the whole matcher.

You met both numbers in Why all pairs is impossible. Pair completeness is the share of true pairs that the blocking keeps: the recall of the blocking. No later step can have a higher pair recall, because a model can only say “same” for a pair that it sees. The reduction ratio is the share of all pairs that the blocking removes: the saved work. A reduction ratio of 0.999 still leaves 1.8 million of Larkfield’s pairs; 0.9999 leaves 180,000.

Go back to the six records of Why all pairs is impossible: 15 pairs, 6 true pairs.

Blocking Candidates True pairs kept Pair completeness Reduction ratio
Postcode 6 3 3 / 6 = 0.50 1 − 6/15 = 0.60
Surname key 6 6 6 / 6 = 1.00 1 − 6/15 = 0.60
Both keys 9 6 6 / 6 = 1.00 1 − 9/15 = 0.40
Window of 3, sorted by name 9 5 5 / 6 = 0.83 1 − 9/15 = 0.40

On six records, every blocking removes few pairs. On 60,108 records, the same keys remove more than 99.9%.

A third number is sometimes useful: pair quality, the share of candidates that are true pairs. For the five keys it is 27,503 ÷ 1,108,591 = 0.025. Most candidates are still different people. That is normal: the comparison step must sort them.

The explorer shows all the blockings of this module, measured on Larkfield’s synthetic records. Each dot is one blocking: the number of candidate pairs (on a log scale, where each step is ten times more) against the pair completeness. Choose a blocking to see its exact numbers.

Explore

What each blocking keeps and loses

Candidate pairs
89,630
Pair completeness
0.845
True pairs lost
4,289
Reduction ratio
0.999950

postcode (same postcode): 89,630 candidate pairs out of 1,806,455,778. It keeps 23,388 of 27,677 true pairs and loses 4,289. True pairs with two different postcodes (mostly customers who moved): postcode keeps 0 of 4,289.

  • One key
  • Several keys
  • Sorted neighbourhood
All 10 measured blockings (synthetic records, 27,677 true pairs)
BlockingCandidate pairsTrue pairs keptPair completenessReduction ratio
postcode89,63023,3880.8450.999950
surname1,035,49823,4890.8490.999427
email13,95212,4720.4510.999992
phone19,87313,6780.4940.999989
dob8,7576,4950.2350.999995
postcode+surname1,103,62027,0190.9760.999389
keys1,108,59127,5030.9940.999386
keys-wide2,351,38227,5460.9950.998698
sorted-name540,92715,0230.5430.999701
sorted-name-502,944,06722,1440.8000.998370
Candidate pairs against pair completeness for ten blockings of the 60,108 synthetic customer records (1,806,455,778 possible pairs, 27,677 true pairs). Measured on the reference laptop.
Text description of the explorer

The explorer has one button for each blocking: postcode, surname, email, phone, dob, postcode+surname, keys, keys-wide, sorted-name and sorted-name-50. It starts on postcode. Four tiles show the candidate pairs, the pair completeness, the true pairs lost and the reduction ratio. A sentence says what the blocking is. For postcode and keys, it also says how many of the 4,289 true pairs with two different postcodes it keeps. A chart shows every blocking as a dot. The candidate pairs run from 1,000 to 10 million on a log scale, and the pair completeness from 0.2 to 1. The chosen blocking is a larger dark dot with its name. A table below lists every blocking. Postcode: 89,630 candidates, completeness 0.845, 4,289 true pairs lost, and 0 of the 4,289 movers kept. Surname: 1,035,498, 0.849. Email: 13,952, 0.451. Phone: 19,873, 0.494. Date of birth: 8,757, 0.235. Postcode or surname: 1,103,620, 0.976. The five keys: 1,108,591, 0.994, 174 lost, and 4,115 of the 4,289 movers kept. The five keys with a wider surname key: 2,351,382, 0.995. Sorted neighbourhood, window 10: 540,927, 0.543; window 50: 2,944,067, 0.800.

Things to try:

  • Choose postcode. It loses exactly 4,289 true pairs. Then choose keys. Read the sentence about the customers who moved for both.
  • Compare surname and postcode+surname. One extra key adds almost no candidates, and keeps 3,530 more true pairs.
  • Compare sorted-name-50 and keys. Which one is better on both numbers?

The prediction asked which true pairs the postcode loses. The answer is exact. The postcode loses 4,289 true pairs. Larkfield’s records have exactly 4,289 true pairs whose two records have different postcodes. Most of them, 3,776, are pairs of customers who moved. One record has the old address, and one has the new. Moving is a practice problem that we made on purpose. The other 513 pairs differ for another reason, such as a typing error in one postcode. Valérie Tremblay’s shop record has H6P 3H1, and her loyalty record H6P 3H7. In this lesson, we call all 4,289 pairs the movers’ pairs, for short.

The lost pairs have one thing in common, so we can mend the net in one place. Look at one of them.

Jacques Beaulieu: one customer, two records. A table with five keys. Postcode: G-009906 (garden centre) J8Y1L3, S-005041 (shop) J0C6V5: different, no block. Email: empty and jacques.beaulieu@mail.example: one is empty, no block. Phone: 5145550008 and empty: one is empty, no block. Surname start: beau|ja and beau|ja: same, a candidate pair. Date of birth: both empty, no block. Note: he moved; only the surname key puts the two records in one block; blocking on the postcode alone never compares them; the union of five keys does.
Jacques Beaulieu's two synthetic records, key by key. Only the surname key puts them in one block.
Text description of the diagram

Jacques Beaulieu has a garden-centre record, G-009906, and a shop record, S-005041. The table compares their five keys. The postcodes differ: J8Y1L3 and J0C6V5, because he moved. The garden-centre record has no email, and the shop record has no phone, so neither key can join them. Neither record has a date of birth. Only the surname key, beau|ja, is the same in both. So the postcode alone never compares his two records, and the five keys together do.

Jacques is typical. A customer who moves changes the address and the postcode, and often the phone. The name stays. So the keys that catch movers are the name keys.

The project has no command for this, so write a small script. Make a file movers.py in your project folder:

from larkfield_match import blocking, evaluate, prepared
postcode = prepared.customers().set_index("record_id").postcode
true = evaluate.true_pairs()
moved = true[true.record_id_l.map(postcode) != true.record_id_r.map(postcode)]
print(f"{len(moved):,} of {len(true):,} true pairs have two different postcodes")
for name in ["postcode", "surname", "keys", "keys-wide"]:
kept = moved.merge(blocking.candidates(name), on=["record_id_l", "record_id_r"])
print(f" {name:<10} keeps {len(kept):,} of them")

The script reads the prepared records and the true pairs. It keeps the true pairs whose postcodes differ. Then it checks which of them are candidates of each blocking. blocking.candidates reads the pairs that match block saved, or makes them the first time. Run it:

Terminal window
python movers.py
4,289 of 27,677 true pairs have two different postcodes
postcode keeps 0 of them
surname keeps 3,631 of them
keys keeps 4,115 of them
keys-wide keeps 4,158 of them

The chart shows the same measure for more blockings:

  • One key or sorted names
  • Five keys
The 4,289 true pairs with two different postcodes in the synthetic records (mostly customers who moved), and the share that each blocking keeps. Wider = the five keys with the first letter of the first name in the surname key.
Text description of the chart

A dot chart with eight rows, one dot per blocking, on an axis from 0 to 1. Postcode 0.00; date of birth 0.22; email 0.43; phone 0.50; sorted names with window 50: 0.79; surname key 0.85; the five keys 0.96; the five keys with the wider surname key 0.97.

The postcode keeps none of them, by definition. The surname key keeps 85%. The five keys keep 96%. Every key adds a few movers that the others miss: a mover with a nickname may still have the same phone or email.

The five keys lose 174 true pairs: 27,677 − 27,503. And the five keys lose exactly 174 of the movers’ pairs: 4,289 − 4,115. So every true pair that keys loses has two different postcodes: 156 of the 174 belong to customers who moved. These pairs differ in the postcode, and also in each of the other four keys. They share no email and no phone. The date of birth is empty or different. And the surname key differs: a nickname, a typing error in the first letters, or an empty name.

How could we catch more? Two options, both measured:

Blocking Candidates True pairs kept Pair completeness Movers kept
keys 1,108,591 27,503 0.9937 4,115
keys+street (adds the street number and street) 1,120,529 27,517 0.9942 4,129
keys-wide (surname key with 1 letter of the first name) 2,351,382 27,546 0.9953 4,158

keys-wide catches 43 more true pairs, but makes 2.1 times more candidates. keys+street catches 14 more, with 1% more candidates. A street key cannot help most movers: their street changed too.

Terminal window
match block --blocking keys-wide
match block --blocking keys+street
keys-wide: 2,351,382 candidate pairs out of 1,806,455,778 possible pairs (11.16 s)
pair completeness 0.995 (27,546 of 27,677 true pairs kept); reduction ratio 0.998698
keys+street: 1,120,529 candidate pairs out of 1,806,455,778 possible pairs (6.31 s)
pair completeness 0.994 (27,517 of 27,677 true pairs kept); reduction ratio 0.999380

Which to choose? Ask what each option costs.

  • More candidates cost compute time. In our bench on the course data, the comparison step of Module 4 took 2.5 seconds for the 1,108,591 candidates of keys. Twice the candidates take about twice the time: a few seconds more. On Larkfield’s data, that is cheap.
  • A lost true pair costs a missed match. The course’s assumed cost of a missed match is 3 dollars a year: double emails and split loyalty points.

At 60,108 records, keys-wide would be a fair choice. The project keeps keys for one reason: growth. A wider key grows much faster with more records, because its blocks are bigger. Scale in Module 8 shows that at 1.1 million records, the five keys already make about 340 million candidate pairs. A wider surname key would make that much worse.

Every run from Module 4 on compares only the candidates of keys. On the test split, keys keeps 5,531 of the 5,564 true pairs: a pair completeness of 0.9941. So no later matcher can reach a pair recall above 0.9941 on the test split. When a model in Module 6 misses a true pair, first check whether the pair was a candidate at all.

  • Every time you change a key, a window or a threshold. Measure the pair completeness and the reduction ratio together.
  • By group, not only in total. Total pair completeness of 0.845 hides that the postcode loses all movers. Measure the groups that matter to the business: movers, each source, each region. The project’s match eval --by does this for runs in Module 2; the notebook does it for blocking.
  • On all records, with true pairs that you trust. Here the synthetic records have known true customers. On real data, you need labelled pairs, and the pairs that blocking removes are exactly the ones a labeller never sees. Label a few random pairs that blocking removes, too.

Common mistake

Judge a blocking by its total alone

Symptom: Tomás compares the postcode (pair completeness 0.845) and the surname key (0.849). “They are the same,” he says, “so choose the faster one.” He chooses the postcode: 89,630 candidates against 1,035,498.

Cause: the two totals hide different losses. The postcode loses all 4,289 true pairs with two different postcodes: 0 of them are candidates. Most of them are customers who moved. The surname key loses other pairs, mostly names with errors in their first letters, and keeps 3,631 of the movers’ pairs. Larkfield’s movers are not a random group. They are customers whose address changed: the people who most need their records joined, so that the old address gets no more catalogues.

Fix: measure what each blocking loses by group, as the script above does. Then combine keys that lose different groups. The postcode and the surname key together keep 0.976 of all true pairs and 3,631 of the movers.

Consequence in AI work: an average can hide a group that a system always fails. A fair evaluation looks at slices. Machine Learning: From Problem to Reliable Model teaches the same lesson: one class of a classifier can fail while the accuracy looks good.

Check your understanding

A blocking makes 180,000 candidate pairs from Larkfield's 60,108 records and keeps 22,000 of the 27,677 true pairs. Compute its pair completeness and its reduction ratio. Is it better than the postcode?

Show the answer

Pair completeness = 22,000 / 27,677 = 0.795. Reduction ratio = 1 − 180,000 / 1,806,455,778 = 0.999900. The postcode keeps 23,388 true pairs (0.845) with only 89,630 candidates (0.999950). So this blocking is worse on both numbers: more candidates and fewer true pairs. Before you discard it, check which true pairs it keeps. If it keeps the movers that the postcode loses, it can still be useful as a second key.

Check your understanding

Every true pair that the five keys lose has two different postcodes. What must be true of such a pair for all five keys to miss it?

Show the answer

The two records differ in every key, or the key is empty in one of them. The postcode differs, mostly because the customer moved. The email is different or empty: for example a family email in one list and an empty email in the other. The phone is different or empty: a new phone after the move, or no phone in the shop record. The date of birth is empty or different. And the surname key differs. The cause is a nickname with other first letters, a typing error in the first four letters of the surname, or an empty name. The notebook shows one such pair: Jim Walker (S-000281) and James Walker (S-010340). Their postcodes differ by one character, and their emails differ. One has no phone, and neither has a date of birth. The surname keys are walk|ji and walk|ja.

Check your understanding

Omar asks: “Why not use keys-wide? It loses fewer customers.” Give him a short, honest answer.

Show the answer

keys-wide keeps 43 more true pairs (27,546 against 27,503), and 43 more of the movers’ pairs. It makes 2.1 times more candidate pairs, which costs a few seconds more on today’s 60,108 records. So today it is a fair choice. But its surname key has bigger blocks, and bigger blocks grow with the square of their size. When Larkfield grows to a million records, the five keys already make about 340 million candidates; a wider key makes the comparison step far slower. The project keeps the narrower key, and measures the cost of the lost pairs instead.

Recap

  • Pair completeness is the recall of the blocking: no later step can have a higher pair recall. The reduction ratio is the work saved. Measure both, every time.
  • The postcode keeps 0.845 of the true pairs. It loses exactly the 4,289 true pairs with two different postcodes, mostly customers who moved: it keeps none of them.
  • Name keys catch movers. The five keys keep 0.9937 of all true pairs and 4,115 of the 4,289 movers’ pairs. Every true pair that they lose has two different postcodes.
  • keys-wide keeps 43 more true pairs with 2.1 times more candidates. Choose by the cost of a missed match against the cost of the candidates, and think about growth.
  • On the test split, keys keeps 0.9941 of the true pairs: the ceiling for the pair recall of every later run.

You now have 1,108,591 candidate pairs that keep almost every true pair. Next, Module 4 compares the two records of each pair, field by field: how similar are two names, two addresses, two emails?

Next: String similarity