Skip to content
Français

Privacy and sensitive suggestions

  • Lesson 29
  • 40 min
  • Module 9: Run it in production

Where we are

In the last two lessons, we served the recommender and kept it fresh. But every list is built from what customers did, and that is personal data. In this lesson, we keep only the data we need, respect opt-outs, apply deletion requests that reach the model, and keep sensitive products out of suggestions and shared family emails.

What you will be able to do

Use personal data in a recommender the way a careful shop would. Keep only what the lists need. Respect customers who opt out. Make a deletion request reach the model, not only the data. And keep sensitive products out of places where the wrong person sees them.

NotebookDo this lesson in a notebookColab · Kaggle · GitHub

The notebook lists the personal data in the shop log and finds a column that no method uses. It trains a small item-item model, removes one customer, trains it again, and counts the products whose neighbours change. Then it keeps sensitive products out of an email to a shared family inbox. Its setup cells download about 8 MB. About 15 minutes.

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

What you need on your computer (only for the project steps)

  • Your larkfield-recs project after the last lesson, with its virtual environment active.
  • The practice files in ../nextia-learning-labs, beside the project folder. You copy one command and one test file from its snapshot end-of-m09.

Tested on macOS with Python 3.12. The Windows (PowerShell) commands were not run.

Think of a good pharmacist in a small town. She remembers your prescriptions. When you come in, she says, “Your usual is ready,” and suggests the right cream for the winter. That is helpful.

She is also careful. She never calls across a full shop, “Your continence pads are ready!” When your teenage son comes to collect something for the family, she does not mention your pregnancy test. If you move away and ask her to forget you, she shreds your card. She also stops telling other customers, “People like you often buy…” based on what you bought.

A recommender knows as much as that pharmacist, about many more people. It needs the same care.

The problem: three ways a good recommender hurts people

Section titled “The problem: three ways a good recommender hurts people”

Here are three real failures from Larkfield’s data and project. The shop log is synthetic, but each failure is measured.

Sensitive products everywhere. Larkfield sells health products in three sensitive groups: mobility aids, continence care, and pregnancy and fertility. Without business rules, 5.7% of the places in the two-stage lists were these products. Continence care is bought again and again, so it is popular. It is number 4 in the all-time popular list, so even a list that is the same for everybody shows it.

A family inbox. 3,178 of Larkfield’s 40,000 customers share one email address with their family. Customer C100081 is one of them. Suppose the shop allows health products on the customer’s own pages. Then C100081’s home page list starts with a dressing stick, a knee scooter and a walking cane. Sent by email, the same list tells the whole family about a health problem.

A deletion that does not reach the model. Customer C100123 asks Larkfield to delete their data. The shop deletes their 22 events. But the item-item model was trained on those events. Trained again without them, the model is different for 394 products. Until it is retrained, the old model still uses what C100123 did. And a service that is still running keeps C100123’s saved list in memory. The course team checked this: it served their personal list after the deletion.

Personal data is any data about a person who can be identified, for example a customer ID linked to what that customer bought. A careful recommender has four habits.

  1. Data minimisation: collect and keep only the personal data that the purpose needs.
  2. Opting out: a customer can say “no personal suggestions”. They then get the same list as everybody, the fallback.
  3. Deletion that reaches the model: a deletion request removes the customer’s data, and then every model and saved list built from it is rebuilt.
  4. Categories never used: some products never appear in suggestions or emails. For others, the place matters: a product that is fine on the customer’s own page can be wrong in a shared inbox.

The service applies these checks at every request, in this order:

A request for one customer reaches a first question: did they ask us to delete their data? If yes, the fallback list, with no personal data used. If no, a second question: did they opt out of personal suggestions? If yes, the fallback list. If no, their personal list. Both lists then reach a third question: is this an email to an address shared with family? If yes, sensitive health products are removed. Then the answer.
The privacy checks of Larkfield's service, applied to every request.
Text description of the diagram

The picture shows the privacy checks of Larkfield’s service. A request for one customer reaches a first question: did they ask us to delete their data? If yes, the service uses the fallback list, so no personal data is used. If not, a second question: did they opt out of personal suggestions? If yes, the fallback list again. Otherwise, their personal list. Then a third question, for both lists: is this an email to an address that a family shares? If yes, the service removes sensitive health products. Then it answers.

The business rules from Module 7 come on top of these checks. By default, they remove sensitive and age-restricted products from every list, on every page.

Which personal data do Larkfield’s methods use? Here are the columns of the two files about people:

File Column Used by the project?
events.csv.gz customer_id, product_id, event, timestamp Yes: every method
events.csv.gz session_id Yes: counting orders for the segments, and the session suggestions
events.csv.gz quantity, price Read, for orders and revenue
customers.csv shared_email, opted_out Yes: the service’s checks
customers.csv region, first_seen No project method uses them

The log has no names, no email addresses and no payment details. The recommender does not need them, so they were never put in. And region and first_seen are not used by any method in the project. (Cold start tested a regional trending list once, and it did not help.) For this purpose, they are data you could stop keeping.

Minimisation is also about time. Trending uses the last 28 days; ALS and item-cf use the whole log. How long to keep old events is a decision for the shop and its privacy team, not for the model alone. The AI security course covers this in Data inventory and Minimize and retain deliberately.

A tiny example: why the model must be retrained

Section titled “A tiny example: why the model must be retrained”

Here is a constructed shop with three customers and three products:

Customer Trowel Gloves Seed tray
A bought bought
B bought bought
C bought bought

An item-item model counts the customers who bought both products. Trowel and gloves: 2 (A and B). Trowel and seed tray: 1 (C). So the product page of the seed tray shows the trowel as “people also bought”.

Now C asks to be deleted. You delete C’s row. But the model was computed before, and it still says “trowel and seed tray: 1”. The seed tray page still shows the trowel, only because of C. Only when you count again without C does the pair fall to 0. So the model keeps what C did until it is retrained.

A deletion request from C100123 leads to: add the ID to outputs/deleted.txt; the project reads the log without their 22 events; retrain, and the item-item model changes for 394 products; rebuild the batch, with no list for C100123; the service gives their requests the fallback with the reason deleted. A dashed note points at the retrain: until the retrain, the old model and lists still hold what they did.
How a deletion request reaches Larkfield's model and service. The data is only the first step.
Text description of the diagram

The picture shows how a deletion request reaches Larkfield’s model and service. The request from C100123 adds their ID to the file outputs/deleted.txt. From then on, the project reads the log without their 22 events. The model is retrained: the item-item model changes for 394 products. The batch is rebuilt, with no list for C100123. The service then gives their requests the fallback, with the reason “deleted”. A dashed note points at the retrain step: until the retrain, the old model and lists still hold what they did.

recs forget does every step of the picture. It removes the customer from the data the project reads, measures how much the item-item model changes, and rebuilds the batch. By default, it rebuilds with item-cf because it takes two seconds. Add --method two-stage to rebuild the two-stage lists instead (about two minutes).

  1. In your larkfield-recs folder, with the virtual environment active, copy the command and this module’s tests:

    Terminal window
    S=../nextia-learning-labs/C37/snapshots/end-of-m09
    cp $S/larkfield_recs/commands/forget.py larkfield_recs/commands/
    cp $S/tests/test_m09.py tests/
  2. Apply C100123’s deletion request:

    Terminal window
    recs forget C100123
    C100123: 22 events before; 0 left in the data the project reads (outputs/deleted.txt)
    The item-item model changes for 394 products when it is trained again without this customer: a model trained before the request still holds their data until it is retrained.
    Batch rebuilt (item-cf); C100123 is in the lists: False; their suggestions now come from: fallback (deleted)
  3. Start the service, as in the first lesson of this module: recs serve --port 8037. In a second terminal, ask for C100123’s suggestions. On Windows, write curl.exe:

    Terminal window
    curl -s "http://127.0.0.1:8037/recommend/C100123?k=3"
    {"customer_id":"C100123","surface":"home","source":"fallback","reason":"deleted","products":[{"product_id":"P11118","title":"Lumi Green Nordic Coffee Table","price":174.99},{"product_id":"P10614","title":"Bramble Terracotta Mosaic Side Table","price":241.99},{"product_id":"P10219","title":"Navy Nitrile-Grip Garden Gloves","price":7.99}],"ms":0.119}

    Their history no longer affects their suggestions: they get the trending list, like a new visitor.

  4. Ask for an email list for C100019, a customer who opted out:

    Terminal window
    curl -s "http://127.0.0.1:8037/recommend/C100019?k=3&surface=email"

    The answer has "source":"fallback" and "reason":"opted out of personal suggestions".

  5. Stop the service with Ctrl+C. Run the tests:

    Terminal window
    python -m pytest -q

    You see 33 passed, 1 warning. The warning is the harmless StarletteDeprecationWarning. The new tests check the fallback without a model and with a broken batch file, the shared inbox, and that a deletion reaches the lists.

recs forget writes the deleted IDs to outputs/deleted.txt. In a real shop, a deletion is never undone. In this course, recs reset deletes the outputs folder, and C100123 comes back for the other lessons.

By default, Larkfield’s rules remove sensitive products from every list. Hana suggests a change: show health products on the customer’s own pages. A customer who buys continence care every month may want a reminder. Rafael, who owns the email, agrees only if a shared family inbox never gets them.

You can try Hana’s change without editing rules.toml. The project reads another rules file when you set LARKFIELD_RULES to its path.

  1. In the project folder, create a file rules-site.toml with this text. It is the course’s rules.toml with one change: remove_sensitive = false.

    [rules]
    remove_out_of_stock = true
    remove_age_restricted = true
    remove_already_bought = true
    remove_sensitive = false # Hana: health products may show on the customer's own pages
    min_margin = 0.0
    max_per_subcategory = 3
  2. Start the service with this rules file:

    Terminal window
    LARKFIELD_RULES=rules-site.toml recs serve --port 8037
  3. In a second terminal, ask for C100081’s home page list and email list:

    Terminal window
    curl -s "http://127.0.0.1:8037/recommend/C100081?k=3"
    curl -s "http://127.0.0.1:8037/recommend/C100081?k=3&surface=email"
    {"customer_id":"C100081","surface":"home","source":"model","reason":null,"products":[{"product_id":"P12612","title":"Natural Wood Dressing Stick","price":32.99},{"product_id":"P11414","title":"Grey Knee Scooter","price":55.99},{"product_id":"P10437","title":"Oak Wooden Walking Cane","price":67.99}],"ms":5.181}
    {"customer_id":"C100081","surface":"email","source":"model","reason":null,"products":[{"product_id":"P12024","title":"Charcoal Coastal Stemmed Dessert Glass","price":17.99},{"product_id":"P12540","title":"Mono Black Martini Glass","price":30.99},{"product_id":"P12010","title":"Sage Green Ribbed Rocks Glass","price":26.99}],"ms":0.142}
  4. Stop the service with Ctrl+C. On Windows, also run Remove-Item Env:LARKFIELD_RULES, so the next service uses the course’s rules again.

Both answers come from the model. The home page shows three mobility aids. The email skips every sensitive product and starts with glasses, the next products in C100081’s list.

The course team counted the whole effect with Hana’s rules and the item-cf lists. 2,899 of the 40,000 customers had at least one sensitive product in their top 10 on the home page. Of the 3,178 customers with a shared inbox, 225 did. In their emails: 0.

So Hana’s change is possible, but it moves the risk to every place a list is shown. A list on a tablet that the family shares has the same problem as the email. Larkfield keeps remove_sensitive = true in its rules for now.

Some products should never be suggested, whatever the page. Larkfield’s rules always remove two groups:

  • Age-restricted products (58 wines and spirits): the shop does not know the customer’s age. Without the rule, 0.8% of the two-stage places were alcohol.
  • Sensitive health products (49 products): mobility aids (18), continence care (18), and pregnancy and fertility (13). Without the rule, 5.7% of the places.

The rules remove these products from the lists. But the models still learn from these purchases. For example, a customer’s continence-care orders still shape their other suggestions. A stricter design removes these events from the training data too. It costs some accuracy for those customers; the course did not measure how much.

An LLM step needs the same care. In Sessions, sequences and LLMs, briefly, the provider’s content filter refused 3 recorded requests: 2 of 100 for re-ranking and 1 of 30 free-form. All 3 involved health products. So that step needs a fallback too. And sending a customer’s history to an outside model is itself a decision about personal data.

Customer C100032 also asks to be deleted. They share an email address with their family.

  1. Predict: how many of their events are removed, and does the item-item model change for more or fewer products than for C100123?
  2. Run recs forget C100032.
  3. Start the service and check that both C100123 and C100032 get the fallback with the reason deleted.
Show the answer

The course team’s run, after C100123’s request:

C100032: 18 events before; 0 left in the data the project reads (outputs/deleted.txt)
The item-item model changes for 324 products when it is trained again without this customer: a model trained before the request still holds their data until it is retrained.
Batch rebuilt (item-cf); C100032 is in the lists: False; their suggestions now come from: fallback (deleted)

18 events change 324 products. A customer with fewer events usually touches fewer products, so fewer neighbour lists change. outputs/deleted.txt now holds both IDs, and both customers get the fallback. C100032’s emails were already free of sensitive products. Now they are not personal at all.

When to apply each habit, and what it costs

Section titled “When to apply each habit, and what it costs”
  • Minimise before you build. A column that never enters the log cannot leak, and needs no deletion later.
  • Opting out costs one check per request. The customer gets the fallback: in this course, the trending list, which is a good list.
  • Deletion requests cost a retrain. For Larkfield, that is 2 seconds for item-cf and about 2 minutes for the two-stage batch. So the shop can process the day’s requests with the nightly batch. Do not forget the other copies. Old batch files kept to roll back, backups and logs of answers also hold the customer’s data.
  • Sensitive categories cost a little accuracy: the business rules cost 0.019 recall@10 for the two-stage system, for all the rules together. That price buys customers’ trust.

Common mistake

The deleted customer still gets their personal list

Symptom: after recs forget C100123, the running service still answers (the line is shortened here):

{"customer_id":"C100123","surface":"home","source":"model","reason":null,"products":[{"product_id":"P10065","title":"Silver Lamb's Ear Perennial","price":20.99},...

Cause: the service read the lists and deleted.txt when it started. It keeps them in memory. You get this answer from a service started before the deletion.

Fix: restart the service after every rebuild of the batch. Then the answer has "source":"fallback","reason":"deleted". A real service reloads its files after each night job.

Common mistake

No customer C999999.

Symptom: recs forget C999999 prints No customer C999999. and changes nothing.

Cause: the ID is not in the shop log. Often, it is a typing mistake in the request.

Fix: check the ID with the person who sent the request. Do not mark a request as done until recs forget prints 0 left.

Check your understanding

Larkfield deletes C100123's 22 events from the data. Why is the deletion not finished yet?

Show the answer

The models and the saved lists were built from those events before the deletion. The item-item model trained again without them is different for 394 products, so the old model still uses what C100123 did. Their saved list is also still in the batch files and in the memory of the running service. The deletion is finished only after four more steps. Retrain the model, rebuild the batch, restart the service, and handle old copies such as rollback files.

Check your understanding

Hana wants health products on customers' own pages. Why does Rafael ask for a separate rule for emails, and what did the course team's run show?

Show the answer

An email to an address that a family shares can be read by other people, so a health product there tells them something private. With Hana’s rules, 225 of the 3,178 customers who share an inbox had a sensitive product in their home page top 10. Their email lists, with the service’s shared-inbox check, had none. C100081’s home page started with mobility aids; the email started with glasses.

Check your understanding

In your project, run recs forget C100123, start the service, and ask for /recommend/C100123. What shows that their history no longer affects their suggestions?

Show the answer

recs forget prints 0 left in the data the project reads and C100123 is in the lists: False. The service answers with "source":"fallback" and "reason":"deleted". The products are the trending list (P11118, P10614, P10219…), as for a customer the shop has never seen. The test test_forget_reaches_the_model checks the same on the small fake shop.

Recap

  • Data minimisation: Larkfield’s log has no names, emails or payments, and no method uses region or first_seen.
  • Opting out: a customer who opts out gets the fallback, with the reason in the answer.
  • Deletion that reaches the model: removing C100123’s 22 events changes the item-item model for 394 products. recs forget removes the data, retrains and rebuilds the batch; restart the service after it.
  • Sensitive categories: the rules remove age-restricted and sensitive products everywhere (without them, 0.8% and 5.7% of the places). A shared family inbox never gets sensitive products, even if the site shows them.
  • This lesson is engineering, not legal advice: ask your privacy team what applies.

In this course, you have built Larkfield’s recommender from a popularity list to a served, monitored and careful system. But Larkfield is our own synthetic shop. Next, you take the same steps to real order data from an online retailer, with a time split, customer segments and the comparison table.

Next: Recommend for an online retailer