Describe a whole pattern of chance, not one probability. Then you can say how many urgent tickets to plan for, how much that number varies, and when the normal curve gives a wrong answer.
NotebookOptional: check the numbers in a notebookColab · Kaggle · GitHub
The notebook is optional: this lesson works with a calculator and the explorer. It calculates the binomial chances with math.comb, and it has the simulated data from the charts on this page. It also has the guided practice, the variation and the module practice, with checks that say if your answer is right without showing it. It uses only Python’s standard library, so it needs no install.
The mean and the standard deviation (SD), from Describe a dataset: Spread. The SD is the typical distance of a value from the mean. This lesson uses Monday’s 12 tickets again: mean 28.75 minutes, SD 21.1 minutes.
What you need to have ready (the notebook needs none of it)
A calculator with a square-root key. A phone calculator is enough.
Your c03-notes.md file, with the ## Probability heading from the “Before you start” box of Events and uncertainty. Add a heading: ### Distributions. If you cannot find the file, make a new empty file with that name: this lesson does not use earlier answers.
About 15 minutes at the end for the module practice and the module check.
The notebook was tested in Jupyter with Python 3.14.6 (standard library only).
One prediction is one yes-or-no event. The urgency model labels each ticket urgent or not urgent. A fraud model labels each payment fraud or not fraud.
Counts for planning and monitoring. “How many urgent tickets arrive per hour?” decides how many senior agents to schedule. “12 urgent tickets today: is that unusual?” decides if somebody looks for an outage.
Means and measurement noise. The accuracy on a test set changes a little each time you measure it. So do the mean loss on a batch of training data and a measured response time. Their spread often follows the normal curve.
Skewed times. Reply times, processing times and request latencies have a long right tail. The normal curve describes them badly, and a team that uses it makes wrong promises.
A distribution lists every possible value and how likely each one is. You already used one in Lesson 1: the team model’s five outputs are a distribution over the five teams. This lesson shows three distributions that appear again and again in AI work.
Look at one new ticket. It is urgent or it is not. Write 1 for urgent and 0 for not urgent. At Larkfield, 8% of tickets are urgent:
Value
Meaning
Chance
1
urgent
0.08
0
not urgent
0.92
That is the whole distribution: a Bernoulli distribution, for one yes-or-no event with a chance p of “yes”. Its mean is p, here 0.08. If you write 1 for every urgent ticket and 0 for the rest, the mean of many tickets is close to 0.08.
Tickets arrive in batches of about 50 an hour. Grace asks: “How many of the 50 are urgent?” The answer is not one number. Each hour is a little different. The binomial distribution gives the chance of each count, from 0 to 50.
It fits when three conditions are true:
A fixed number of tickets, n. Here n = 50.
The same chance for each ticket, p. Here p = 0.08.
The tickets are independent: one urgent ticket does not make the next one more likely.
The third condition is the one from Lesson 1. During an outage, urgent tickets arrive together, so the binomial distribution does not describe that hour.
Here are the chances for n = 50 and p = 0.08:
Urgent tickets
0
1
2
3
4
5
6
7
8
9
10 or more
Chance (%)
1.5
6.7
14.3
19.9
20.4
16.3
10.6
5.8
2.7
1.1
0.6
Three things to read from the table:
The center. The most likely counts are 3 and 4. The mean is n × p = 50 × 0.08 = 4 urgent tickets an hour.
The spread. The SD is n×p×(1−p)=50×0.08×0.92=3.68=1.92. Read it as “the square root of n times p times one minus p”.
The shape. The table is not symmetric. Counts can go up to 10 or more, but they cannot go below 0. So the right side is longer.
Where does one value come from? Take 0 urgent tickets: each of the 50 tickets is not urgent, with a chance of 0.92 each. If the tickets are independent, multiply: 0.92 multiplied by itself 50 times is 0.9250 = 0.0155, the 1.5% in the table. The other values need one more step (how many ways to choose which tickets are urgent). The optional Python box below calculates them. In professional work, scipy.stats.binom does the same.
Explore
How many urgent tickets in one batch?
Predict first: The batch grows from 50 to 500 tickets (10 times as many). Does the SD of the urgent count also grow 10 times?
Mean = n × p
4
SD = √(n × p × (1 − p))
1.92
Most likely count
4
Range in about 95% of batches
1 to 8 (96.8%)
Chance of more than capacity
10.2%
Expect about 4 urgent tickets; most batches have 1 to 8. More than 6: 10.2% of batches, about 1 in 10.
Each bar is the chance of one count of urgent tickets. Orange bars are counts above the capacity line.Text description of the explorer
The explorer has three sliders: the batch size n (1 to 1,200 tickets), the chance p for one ticket (0 to 1), and the capacity. The capacity is the most urgent tickets the team can handle in one batch. A switch draws the normal curve with the same mean and SD. It starts at n = 50, p = 0.08 and capacity 6. The mean is 4, the SD is 1.92 and the most likely count is 4. 96.8% of batches have 1 to 8 urgent tickets, and 10.2% of batches have more than 6. At n = 500, the mean is 40 and the SD is 6.07: about 3.2 times larger, not 10 times. At n = 1,200, the mean is 96 and the SD is 9.4, and the normal curve follows the bars closely. At n = 50 and p = 0.08, the normal curve fits poorly, because n × p = 4 is small. At p = 0, every batch has 0 urgent tickets and the SD is 0.
The SD grows with the square root of n. Ten times as many tickets give about √10 ≈ 3.2 times the spread. So a larger batch is more predictable relative to its size. 4 ± 1.9 is a large range around 4, but 40 ± 6.1 is a small range around 40.
Worked example: Priya plans the senior agent’s hour
Grace has one senior agent for urgent tickets. The agent can handle 6 urgent tickets an hour. Grace asks: “How often is that not enough?”
Priya reads the binomial table for n = 50 and p = 0.08:
Question
Calculation
Result
Expected urgent tickets an hour
50 × 0.08
4
Typical spread
√(50 × 0.08 × 0.92)
1.92
Chance of 6 or fewer
1.5 + 6.7 + 14.3 + 19.9 + 20.4 + 16.3 + 10.6
89.8% (the rounded values in the table add up to 89.7%)
Chance of more than 6
100 − 89.8
10.2%, about 1 hour in 10
Chance of more than 7
100 − 89.8 − 5.8
4.4%, about 1 hour in 23
Priya’s answer: “Plan for 4, but the agent’s capacity of 6 is not enough in about 1 hour of every 10. In an 8-hour day, at least one such hour happens on about 6 days in 10. With capacity for 7, it happens in about 1 hour of 23.” Omar can now compare the cost of a second agent with the cost of those hours.
The mean alone (4 an hour) would hide the problem. Staffing for the mean means the team is too small in many hours.
The normal distribution is the symmetric, bell-shaped curve. It is described by two numbers: its mean (the center) and its SD (the width). It is useful because of one rule of thumb:
About 68% of values are within 1 SD of the mean.
About 95% of values are within 2 SD of the mean.
It is also used too often. Here is where it fits, and where it does not.
Priya takes 300 samples of 30 tickets each, and calculates the mean reply time of each sample. Each single reply time is skewed, but the mean of 30 of them is not.
The mean reply time of each of 300 samples of 30 tickets (simulated). The means follow the normal curve with the same mean and SD.Text description of the chart
A histogram of 300 sample means, from about 19 to 40 minutes. The normal curve has the same mean (28.0 minutes) and SD (3.4 minutes). The bars rise and fall almost symmetrically around 28 minutes, and they follow the curve closely. 95.7% of the sample means are within 2 SD of the mean, close to the 95% that the normal curve predicts. The data is simulated with a fixed seed.
This is a general pattern: the mean of many independent values is close to normal, even when the values themselves are not. Module 5 uses it to judge how far a measured mean can be from the truth.
The normal curve also fits two other cases you meet in AI work:
Measurement noise. Measure the time to load a model 100 times. The results differ by small amounts that come from many small causes. Their spread is often close to normal.
Counts with a large mean. Take a week of 1,200 tickets. The urgent count has mean 1,200 × 0.08 = 96 and SD √(1,200 × 0.08 × 0.92) = 9.4. In the explorer, set n = 1,200 and switch on the normal curve: it follows the bars closely. The 95% rule gives 96 ± 2 × 9.4, about 77 to 115 urgent tickets a week. The explorer’s exact range is 78 to 115.
As a rule of thumb, the normal curve fits a binomial count when n × p and n × (1 − p) are both above about 5. For a batch of 50, n × p = 4, so it fits poorly, as the explorer shows.
Go back to the 12 Monday tickets from Module 1. Their reply times have a mean of 28.75 minutes and an SD of about 21.1 minutes, and T-2052 waited 95 minutes. What does a normal curve with this mean and SD predict?
Prediction of the normal curve
Value
Reality
Share of tickets with a reply time below 0 minutes
8.7%, about 1 in 12
impossible: a reply cannot come before the ticket
How far 95 minutes is from the mean
(95 − 28.75) ÷ 21.1 = 3.1 SD
Chance of 95 minutes or more
0.08%, about 1 in 1,200
it happened to 1 ticket in 12
The chart shows the same problem with a larger, simulated sample of 400 tickets.
400 simulated reply times. The bars have a long right tail; the normal curve with the same mean and SD is symmetric and puts some tickets below 0 minutes.Text description of the chart
A histogram of 400 simulated reply times, from 3 to 105 minutes. The median is 23 minutes, the mean is 26.4 minutes and the SD is 15.2 minutes. Most bars are between 10 and 40 minutes, and a long, thin tail reaches past 100 minutes. The normal curve with the same mean and SD is symmetric. It is too high on the left: it predicts 14.2% of tickets below 10 minutes, but only 5.8% are. It puts 4.2% of tickets below 0 minutes, shown as a shaded area. On the right, it predicts almost no ticket above 80 minutes (0.02%), but 3 of the 400 tickets waited longer than that.
Reply times are skewed: most replies are quick, a few take much longer, and none can be negative. The normal curve is symmetric, so it gets both ends wrong. For skewed data, describe the data with the median and percentiles, as in Module 1, not with “mean ± 2 SD”.
Sunday mornings are quiet: about 20 tickets an hour. The chance that a ticket is urgent is still 0.08.
Write the mean number of urgent tickets in the hour: n × p. Expected: 20 × 0.08 = 1.6.
Write the SD: √(n × p × (1 − p)). Expected: √(20 × 0.08 × 0.92) = √1.472 = 1.21.
Calculate the chance of no urgent ticket at all: 0.9220. Expected: 0.189, about 19%.
In the explorer, set n = 20 and the capacity to 3. Read the chance of more than 3. Expected: 7.1%.
Switch on the normal curve. Write one sentence: does it fit? Why or why not?
Show an example answer
Question
Result
Mean
20 × 0.08 = 1.6 urgent tickets
SD
√1.472 = 1.21
No urgent ticket
0.9220 = 0.189: about 1 Sunday hour in 5
More than 3
7.1%: about 1 hour in 14
The normal curve fits badly. n × p = 1.6 is far below 5. The bars are piled up at 0, 1 and 2, and the curve spills below 0 urgent tickets, which is impossible.
Check: your mean is less than the mean for a batch of 50 (4), and your SD is smaller too (1.21 against 1.92).
Change one condition. During a payment outage, 1 ticket in 5 is urgent: p = 0.20, and the batch is still 50 tickets. Use the explorer.
What are the mean and the SD? How often is the capacity of 6 not enough? What capacity keeps the overflow chance near 5%? Then write one sentence: why might the binomial numbers still be too low for a real outage?
More than 6: 89.7%. A capacity of 6 is not enough in almost every outage hour.
Capacity 14: more than 14 has a chance of 6.1%. Capacity 15: 3.1%. So plan for about 14 or 15.
The binomial distribution assumes independent tickets. In an outage, tickets are not independent: many customers write about the same problem at the same time. So the real counts can be even higher and more variable than the binomial numbers.
Run Python in your browser
Optional: calculate the binomial table
This box is optional. It calculates the binomial chances for a batch of 50 with math.comb, which counts the ways to choose which k tickets are urgent. Run it and compare the output with the table above. Then change p to 0.20 and check your answer for the outage hour.
Tab adds four spaces. Press Esc, then Tab, to leave the box. Ctrl+Enter runs the code.
Common mistake
Using the normal curve for skewed times
Symptom: a dashboard promises “99.9% of tickets get a reply within mean + 3 SD, about 92 minutes”. In the first week, about 1 ticket in 12 waits longer than that, and customers complain.
Cause: the promise uses the normal curve, which is symmetric. Reply times are skewed: they cannot be below 0, and they have a long right tail. The normal curve puts too few tickets in that tail.
Fix: for skewed measurements, read the percentiles from the data. Write “95% of tickets get a reply within X minutes”, where X is the 95th percentile of the measured times. Use the normal curve for means of many values and for counts with a large n × p. First check its fit with a histogram.
Check your understanding
Predict: in the explorer, the batch grows from 50 to 500 tickets. Does the SD of the urgent count grow 10 times? What does this mean for planning?
Show the answer
No. The SD grows with the square root of n: from 1.92 to 6.07, about 3.2 times (√10 ≈ 3.16). The mean grows 10 times, from 4 to 40. So a larger batch is more predictable compared with its size. Planning for a whole day or week is easier than planning for one hour.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Which of these can you describe with a binomial distribution? (a) Urgent tickets among 50 tickets on a normal day. (b) Urgent tickets among 50 tickets in the first hour of an outage. (c) The reply time of one ticket.
Show the answer
Only (a). For (b), the tickets are not independent: one cause makes many tickets urgent at once, so the binomial distribution gives counts that are too low. For (c), a reply time is a measurement in minutes, not a count of yes-or-no events, and its distribution is skewed.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
The module practice ties the three lessons together: calculate how a rare event changes the meaning of a positive result. First follow a worked example, then do the same task with a different base rate.
Larkfield’s urgency model flags tickets for the senior agent. Suppose it has the same quality as the payment alert: it flags 90 of 100 urgent tickets, and 5 of 100 other tickets by mistake. Urgent tickets are 8% of all tickets. Priya counts 10,000 tickets:
Step
Calculation
Count
Urgent tickets
8% of 10,000
800
Not urgent
10,000 − 800
9,200
Real flags
90% of 800
720
Missed urgent tickets
800 − 720
80
False alarms
5% of 9,200
460
All flags
720 + 460
1,180
Share of flags that are real
720 ÷ 1,180
0.610, about 61%
The urgent flag on 10,000 tickets: base rate 8%, recall 90%, false-alarm rate 5%.Text description of the chart
A tree that starts with 10,000 tickets. 8% (800) are urgent and 92% (9,200) are not. Of the 800 urgent tickets, 90% (720) are flagged and 10% (80) are missed. Of the 9,200 other tickets, 5% (460) are flagged by mistake and 95% (8,740) are not flagged. All flags: 720 + 460 = 1,180.
Real condition
Flagged
Not flagged
Total
Urgent
720real flag
80missed
800total
Not urgent
460false alarm
8,740correctly left
9,200total
Total
1,180total
8,820total
10,000total
Share of real cases that are flagged (recall): 720 ÷ 800 = 90%
Share of flags that are real (precision): 720 ÷ 1,180 = 61%
The same counts as a screening matrix. The outlined column is every flagged ticket.Text description of the table
Urgent tickets: 720 flagged (real flags) and 80 not flagged (missed), 800 in total. Not urgent: 460 flagged (false alarms) and 8,740 not flagged, 9,200 in total. Flagged in total: 1,180. Recall: 720 ÷ 800 = 90%. Share of flags that are real: 720 ÷ 1,180 = 61.0%.
The flag has the same recall and false-alarm rate as the payment alert. But urgent tickets (8%) are 4 times as common as payment problems (2%), so 61% of the flags are real, not 27%. The quality of a test does not decide the meaning of a positive result alone. The base rate decides it too.
Some customers write that they will ask their bank to reverse a payment (a chargeback). These tickets are rare: 1 in 100. Tomás proposes a flag for them with the same quality: 90% recall and a 5% false-alarm rate.
In c03-notes.md, under a new heading ### Module practice: Probability:
Draw the frequency tree for 10,000 tickets, with every count.
Calculate the share of flags that are real.
Write P(flag | chargeback) and P(chargeback | flag) as numbers, and say which one the team that reviews the flags needs.
Check your counts with the explorer below.
Write two sentences for Omar: should a flag send a ticket straight to the bank-dispute process? What would you change?
Explore
Check your tree: base rate, recall and false alarms
Predict first: Before you move the sliders: the base rate falls from 8% to 1%, and nothing else changes. Will the share of real flags fall by about the same factor (8 times), by less, or by more?
Real cases in 10,000
800
Real flags
720
False alarms
460
All flags
1,180
Share of flags that are real
61%
Accuracy (all tickets labelled correctly)
94.6%
Of 1,180 flagged tickets, 720 are real cases: about 61 in 100. Most flags are real, but not all.
1,000 squares. Each square is 10 tickets.
Flagged, real case
Flagged, false alarm
Missed real case
Not flagged, correctly
The same tickets as a frequency tree
Any flag with a recall and a false-alarm rate. It starts at the worked example (8%). Each square is 10 of the 10,000 tickets.Text description of the explorer
The explorer has three sliders: how common the real cases are (0% to 50%), the recall and the false-alarm rate (0% to 30%). It shows 1,000 squares, each one 10 tickets, and a frequency tree for 10,000 tickets. It starts at the worked example: 8% real cases, 90% recall and 5% false alarms. That gives 720 real flags, 460 false alarms and 61.0% real flags. At a 1% base rate, it shows 90 real flags, 495 false alarms and 15.4% real flags. The share falls by a factor of about 4, not 8, because the false alarms change only a little. With a 0% false-alarm rate, every flag is real. With a 0% recall, every flag is a false alarm. If nothing is flagged, the explorer says that there is no share to show.
Show the model answer
Step
Calculation
Count
Chargeback tickets
1% of 10,000
100
Other tickets
10,000 − 100
9,900
Real flags
90% of 100
90
Missed
100 − 90
10
False alarms
5% of 9,900
495
Correctly left
9,900 − 495
9,405
All flags
90 + 495
585
Share of flags that are real
90 ÷ 585
0.154, about 15%
P(flag | chargeback) = 0.90: of 100 chargeback tickets, the flag catches 90. This is what Tomás’s test measures.
P(chargeback | flag) = 0.154: of 100 flagged tickets, about 15 are chargebacks. This is what the review team needs, because they see only flagged tickets.
“Do not send flagged tickets straight to the bank-dispute process: about 85 of every 100 flags are not chargebacks. Use the flag to put tickets at the top of a person’s queue, and ask Tomás to reduce the false-alarm rate. At 1%, the share of real flags rises to about 48% (90 of 189).”
Review: the tree has all six counts and the share is 0.15. The two conditional probabilities have the right names, in the right order.
Retrieval: what is a base rate? Where does it go in a frequency tree?
Show the answer
The base rate is the share of all cases that have the condition, before any test or model looks at them. For example, 2% of tickets are payment problems. It goes on the first branches of the tree: it splits the 10,000 tickets into 200 with the problem and 9,800 without. The recall and the false-alarm rate come after it, on the second branches.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: a new alert has 99% recall and a 2% false-alarm rate. It looks for a problem with a base rate of 0.1% (1 ticket in 1,000). Will most of its flags be real? Explain with counts for 10,000 tickets.
Show the answer
No. 0.1% of 10,000 = 10 tickets with the problem, and the alert flags about 10 of them (99%). It also flags 2% of the other 9,990 tickets, about 200. So about 10 of 210 flags are real: under 5%. With a very rare problem, even a small false-alarm rate on the large group gives most of the flags.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Completion check: a fraud model's documentation says 'given a fraudulent payment, the model raises an alert in 95% of cases'. A manager writes in a report: 'when the model raises an alert, the payment is fraud in 95% of cases'. Write both statements as probabilities with a bar, say what is wrong, and say what you need to find the second number.
Show the answer
The documentation gives P(alert | fraud) = 0.95, the recall. The manager wrote P(fraud | alert) = 0.95, which is the reverse. The two are different numbers: one divides by all fraudulent payments, the other by all alerts. To find P(fraud | alert), you need two more numbers. One is the base rate of fraud among all payments. The other is the false-alarm rate, P(alert | not fraud). Then draw a frequency tree for 10,000 payments and divide the real alerts by all alerts. If fraud is rare, P(fraud | alert) can be much lower than 95%.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
In Module 3, you learned to reason about chance by counting tickets:
A probability is a share of cases. Complements, “and”, “or” and independence come from counts. Events that share a cause are not independent.
A conditional probability divides by one group. P(result | condition) and P(condition | result) are different numbers; a frequency tree for 10,000 cases turns one into the other.
With a rare condition, most positive results can be false alarms, even from a good test. The base rate decides what a positive result means.
Bernoulli describes one yes-or-no event. Binomial describes the count in a batch of independent events, with mean n × p and SD √(n × p × (1 − p)).
The normal curve fits means of many values, measurement noise and large counts. It does not fit skewed times, which need the median and percentiles.
Next, Module 4 shows how a model learns: you follow a loss curve downhill, one step at a time.
