Conditional probability

What you will be able to do

Find out what an alert really tells you when it fires. A rare problem with a good alert can still give mostly false alarms. A team that misses this sends the wrong tickets to the wrong people.

NotebookOptional: check the numbers in a notebookColab · Kaggle · GitHub

The notebook is optional: this lesson works with a calculator. It counts the payment alert on 10,000 tickets and changes the base rate, as the explorer does. It also has the guided practice and the variation, with checks that say if your answer is right without showing it. It uses only Python’s standard library, so it needs no install.

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• How to find a probability by counting, with the 100 Tuesday tickets, from Events and uncertainty: Count first. This lesson uses the same tickets.
• Joint events (“and”), from Events and uncertainty: Joint events.
• Independence, from Events and uncertainty: Independence.

What you need to have ready (the notebook needs none of it)

• A calculator. A phone calculator is enough.
• Your c03-notes.md file, with the ## Probability heading from the “Before you start” box of Events and uncertainty. Add a heading: ### Conditional probability. If you cannot find the file, make a new empty file with that name: this lesson does not use earlier answers.

The notebook was tested in Jupyter with Python 3.14.6 (standard library only).

Where you meet this in AI

• Fraud alerts. A bank’s model flags card payments that look like fraud. Most payments are not fraud, so most flags are checked and released.
• Spam and abuse filters. A filter catches almost all spam. If it also catches a small share of normal mail, a person may lose important messages.
• Medical-style screening. A test finds a rare condition in 90 of 100 people who have it. Many people with a positive result still do not have it. AI tools that screen images or records have the same problem.
• Larkfield’s payment alert. A keyword alert flags tickets that look like payment problems. This lesson finds out how many of its flags are real.

Probability inside a group

Go back to the 100 Tuesday tickets from the last lesson. Here they are as a contingency table: a table that counts the tickets for each combination of two properties.

	Urgent	Not urgent	Total
Payment ticket	4	11	15
Other team	4	81	85
Total	8	92	100

Now ask two questions that sound almost the same:

1. Of the payment tickets, how many are urgent? Look only at the payment row: 4 of 15. That is 4/15 = 0.27.
2. Of the urgent tickets, how many are payment tickets? Look only at the urgent column: 4 of 8. That is 4/8 = 0.50.

Each answer is a conditional probability: the probability of an event when you look only at the cases in one group. The group is the condition. You write it with a vertical bar, read as “given”:

• P(urgent | payment) = 0.27, read “the probability of urgent, given a payment ticket”.
• P(payment | urgent) = 0.50, read “the probability of a payment ticket, given an urgent ticket”.

The top of the fraction is the same in both: the 4 tickets that are both. The bottom is different: the size of the group you look in. The condition decides what you divide by. So P(A | B) and P(B | A) are different numbers, and mixing them up is the most common mistake in this module.

The payment alert

About 2 in 100 Larkfield tickets are about a payment problem: a customer was charged twice, or sees a charge they do not know. These tickets must go to the payment team fast.

Tomás builds a keyword alert. It looks for phrases such as “charged twice” and “unknown charge”. He tests it on old tickets:

• Of 100 real payment problems, the alert flags 90. This share is the recall: P(flag | payment problem) = 0.90.
• Of 100 other tickets, the alert flags 5 by mistake. This share is the false-positive rate: P(flag | other ticket) = 0.05. A wrong flag is a false alarm.

Tomás writes in the team chat: “The alert is 90% accurate. When it flags a ticket, send the ticket straight to the payment team.”

Grace asks the question that matters to her team: when the alert flags a ticket, how often is it really a payment problem? That is P(payment problem | flag). It is the reverse of the 90%, so you cannot read it from Tomás’s test. You have to count.

Count 10,000 tickets

Priya imagines 10,000 tickets, about two months of Larkfield’s work. Whole numbers of tickets are easier to think about than small decimals. This way of counting is called natural frequencies.

1. How many are payment problems? 2% of 10,000 = 200. The other 9,800 are other tickets.
2. Of the 200 payment problems, how many does the alert flag? 90% of 200 = 180. It misses the other 20.
3. Of the 9,800 other tickets, how many does it flag? 5% of 9,800 = 490 false alarms. It correctly leaves 9,310.
4. How many flags in total? 180 + 490 = 670.
5. Of the 670 flags, how many are real? 180. So 180/670 = 0.27, about 27 in 100.

The payment alert on 10,000 tickets. Each branch shows a share; each box shows how many tickets reach it. The two coloured boxes are all the flagged tickets.

Text description of the chart

A tree that starts with 10,000 tickets. 2% (200) are payment problems and 98% (9,800) are other tickets. Of the 200 payment problems, 90% (180) are flagged, real flags, and 10% (20) are not flagged, missed. Of the 9,800 other tickets, 5% (490) are flagged, false alarms, and 95% (9,310) are not flagged. The flagged tickets are the 180 real flags plus the 490 false alarms: 670 in total.

The alert catches 9 in 10 real problems. But only about 1 flag in 4 is real. The other 3 are false alarms. The reason is the base rate: the share of all tickets that have the problem before any alert looks at them. Here it is 2%. A small false-alarm rate on a very large group (5% of 9,800) gives more flags than a high recall on a small group (90% of 200).

The same counts as a screening matrix

The same four numbers fit in a 2 × 2 table. The rows are the truth; the columns are what the alert does. In Module 6, the same table is called a confusion matrix.

Real condition	Flagged	Not flagged	Total
Payment problem	180real flag	20missed	200total
Other ticket	490false alarm	9,310correctly left	9,800total
Total	670total	9,330total	10,000total

• Share of real cases that are flagged (recall): 180 ÷ 200 = 90%
• Share of flags that are real (precision): 180 ÷ 670 = 26.9%
• Share of other tickets flagged by mistake (false-positive rate): 490 ÷ 9,800 = 5%
• Share of all tickets labelled correctly (accuracy): 9,490 ÷ 10,000 = 94.9%

The payment alert on 10,000 tickets. The outlined column is every flagged ticket: 180 real flags and 490 false alarms.

Text description of the table

A table with two rows and two columns, plus totals. Payment problems: 180 flagged (real flags) and 20 not flagged (missed), 200 in total. Other tickets: 490 flagged (false alarms) and 9,310 not flagged, 9,800 in total. Flagged in total: 670; not flagged: 9,330; all tickets: 10,000. The shares under the table: recall 180 ÷ 200 = 90%. Share of flags that are real: 180 ÷ 670 = 26.9%. False-positive rate: 490 ÷ 9,800 = 5%. Accuracy: (180 + 9,310) ÷ 10,000 = 94.9%.

Each share divides by a different total:

• Recall looks along the payment-problem row: 180 of 200.
• The share of flags that are real looks down the flagged column: 180 of 670. Its technical name is precision. Module 6 uses it again.
• Accuracy looks at the whole table: (180 + 9,310) of 10,000 = 94.9%.

How “90% accurate” misleads

Tomás’s “90% accurate” was the recall. The real accuracy is even higher, 94.9%. And still, 73 of every 100 flags are wrong. Accuracy is high because 9,310 easy “not flagged” tickets fill the table.

An alert that never flags anything is correct for all 9,800 other tickets. Its accuracy is 98%, better than Tomás’s alert, and it catches no payment problem at all. When one class is rare, accuracy hides the errors that matter. Module 6 comes back to this.

Explore: change how common the problem is

Explore

How the base rate changes the meaning of a flag

Payment problems in 10,000

200

Real flags

180

False alarms

490

All flags

670

Share of flags that are real

26.9%

Accuracy (all tickets labelled correctly)

94.9%

Of 670 flagged tickets, 180 are payment problems: about 27 in 100. Most flags are false alarms.

1,000 squares. Each square is 10 tickets.

• Flagged, payment problem
• Flagged, false alarm
• Missed payment problem
• Not flagged, correctly

The same tickets as a frequency tree

The payment alert with 90% recall and a 5% false-alarm rate. Only the base rate changes. Each square is 10 of the 10,000 tickets.

Text description of the explorer

The explorer has one slider: how common payment problems are, from 0% to 50%. The alert’s recall stays 90% and its false-alarm rate stays 5%. It shows 1,000 squares, each one 10 tickets. Orange squares are real flags and blue squares are false alarms. Orange outlines are missed payment problems, and grey squares are other tickets that the alert does not flag. Below the grid, a frequency tree shows the same counts for 10,000 tickets. At the start (2%), there are 670 flags and 180 are real: 26.9%. At 20%, there are 2,000 payment problems, 1,800 real flags and 400 false alarms, so 81.8% of the 2,200 flags are real. At 0.5%, 45 real flags and 498 false alarms: 8.3%. At 0%, every flag is a false alarm. The alert is the same in every case; only the base rate changes the meaning of a flag.

Bayes’ rule: the same count as a formula

You found 0.27 without a formula. Bayes’ rule writes the same steps as one line. It turns P(flag | problem), which a test measures, into P(problem | flag), which the team needs.

$$P(\text{problem} \mid \text{flag}) = \frac{P(\text{flag} \mid \text{problem}) \times P(\text{problem})}{P(\text{flag})}$$

Read it as: “the chance of a real problem, given a flag, is the share of tickets that are a problem and flagged, divided by the share of all flagged tickets”. Each part matches a step in the tree:

Part	In words	Value	In the tree
P(problem)	the base rate	0.02	200 of 10,000
P(flag | problem)	the recall	0.90	180 of 200
top line	problem and flagged	0.90 × 0.02 = 0.018	180 of 10,000
P(flag)	all flags: real + false alarms	0.018 + 0.05 × 0.98 = 0.018 + 0.049 = 0.067	670 of 10,000
result		0.018 / 0.067 = 0.269	180 of 670

The formula and the tree give the same answer, because they are the same count. When you are unsure, draw the tree.

Worked example: Priya answers Tomás

Priya replies in the team chat with the counts, not the formula:

“In 10,000 tickets, the alert flags 670. 180 of them are payment problems, and 490 are not. If we send every flag straight to the payment team, they get 490 wrong tickets every two months, about 60 a week. Those customers wait longer, because their ticket goes to the wrong team first.”

Omar asks what to do. Priya gives three options, each with its own number:

1. Use the flag to sort, not to decide. Flagged tickets go to the top of the queue, and a person checks each one. The 20 missed problems still arrive by the normal route.
2. Reduce false alarms. If the false-alarm rate falls, the share of real flags rises fast. You calculate this in the guided practice.
3. Measure the base rate again. In some weeks, payment problems are more common. Then the same alert gives a higher share of real flags.

Guided practice: a better keyword list

Mei finds that many false alarms contain the word “charge” in another sense (“free delivery charge”). She removes those phrases. The new alert has the same recall (90%), and the false-alarm rate falls to 1 in 100.

1. Start with 10,000 tickets. Write the number of payment problems and other tickets. Expected: 200 and 9,800.
2. Calculate the real flags: 90% of the payment problems. Expected: 180.
3. Calculate the false alarms: 1% of the other tickets. Expected: 98.
4. Add them to get all flags. Expected: 278.
5. Divide the real flags by all flags. Expected: about 0.65.
6. Write one sentence for Grace: what changed, and why?

Show an example answer

Step	Calculation	Count
Payment problems	2% of 10,000	200
Other tickets	10,000 − 200	9,800
Real flags	90% of 200	180
False alarms	1% of 9,800	98
All flags	180 + 98	278
Share of flags that are real	180 ÷ 278	0.647, about 65%

“The alert finds the same 180 payment problems, but it now flags 98 other tickets instead of 490. About 2 of every 3 flags are now real, not 1 in 4.” The recall did not change. The false-alarm rate did, and with a rare problem the false alarms decide what a flag means.

Check: your real flags plus your false alarms equal your total flags.

Independent variation: customers change their words

Change one condition. A new payment app sends customers messages with new wording, and customers copy it. Tomás tests the original alert again: it now catches only 60 of 100 payment problems. The false-alarm rate is back at 5 in 100, and 2% of tickets are payment problems.

Draw the frequency tree for 10,000 tickets. How many flags are there, and what share of them is real? How many payment problems does the alert miss?

Show an example answer

• Payment problems: 200. Other tickets: 9,800.
• Real flags: 60% of 200 = 120. Missed: 200 − 120 = 80.
• False alarms: 5% of 9,800 = 490.
• All flags: 120 + 490 = 610.
• Share of flags that are real: 120 ÷ 610 = 0.197, about 20%.

The share of real flags falls from 27% to 20%, and the alert now misses 80 payment problems instead of 20. The number of false alarms does not change, because they come from the other tickets, and the false-alarm rate did not change.

Common mistake

Reading the alert's number the wrong way round

Symptom: a plan says “the alert is 90% accurate, so 90% of flagged tickets are payment problems”. The payment team receives about 60 wrong tickets a week and starts to ignore the flag.

Cause: 90% is P(flag | payment problem), the recall. The plan needed P(payment problem | flag), the reverse. With a 2% base rate, the reverse is 27%, not 90%.

Fix: before you use a number from a test, write it with the bar: P( … | … ). Ask which group it divides by. Then draw the frequency tree with the real base rate, and read the share of flags that are real from the flagged boxes.

Check your understanding

Grace says: 'Half of the urgent tickets are payment tickets, so half of the payment tickets are urgent.' Use the Tuesday table (4 urgent payment tickets, 15 payment tickets, 8 urgent tickets) to explain her mistake.

Show the answer

P(payment | urgent) = 4/8 = 0.50 is true. But P(urgent | payment) = 4/15 = 0.27. Both have the same top number, the 4 tickets that are both, but they divide by different groups: 8 urgent tickets, or 15 payment tickets. A conditional probability and its reverse are usually different numbers.

Check your understanding

Predict: the base rate of payment problems falls from 2% to 1%. The alert keeps 90% recall and a 5% false-alarm rate. Does the share of flags that are real go up or down? Check with 10,000 tickets.

Show the answer

It goes down. 1% of 10,000 = 100 payment problems, so 90 real flags. False alarms: 5% of 9,900 = 495. All flags: 585. Share real: 90 ÷ 585 = 0.154, about 15%. The rarer the problem, the more the false alarms from the large group of other tickets dominate the flags.

Check your understanding

Why can an alert with 94.9% accuracy still be wrong for most of the tickets it flags?

Show the answer

Accuracy counts all tickets, and most tickets are other tickets that the alert correctly leaves alone (9,310 of 10,000). Those easy cases make accuracy high. The flagged tickets are a small group (670), and because the problem is rare, 490 of them are false alarms. Accuracy and the share of flags that are real divide by different groups, so they answer different questions.

Recap

• A conditional probability looks only at one group: P(A | B) divides by the cases where B is true.
• P(flag | problem) and P(problem | flag) are different numbers. A test measures the first; a team usually needs the second.
• Count 10,000 cases in a frequency tree: base rate first, then recall and the false-alarm rate. The flagged boxes give the share of flags that are real.
• With a rare problem, a small false-alarm rate on many other tickets can produce most of the flags.
• Bayes’ rule is the same count as a formula.

Next, you describe whole patterns of chance: how many urgent tickets arrive in a batch, and when the normal curve fits.

Next: Distributions in practice