Capacity plan from a public traffic trace

What you will be able to do

Turn a real public request log into a capacity plan for AI work. You will find the bursts that an average hides, and replay them through a queue with the course’s measured job times. You will choose the number of workers for a target wait and test that choice once on months that you kept aside. Then you write a recommendation that says what it assumes and where it stops.

Case studyPortfolio piece

Dataset

San Francisco 311 Cases (DataSF), 2025 and June 2026

Licence

ODC Public Domain Dedication and License 1.0

Time

About 2.5 hours

Runs in

Colab, Kaggle or VS Code

Skills used

• Workload model from a trace
• Percentiles at three time scales
• Burst detection
• Queue simulation
• Design point
• Held-out check
• Quota and outage
• Capacity recommendation

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. To use VS Code, download the notebook from GitHub. Stuck? See the solution notebook.

This case study is a notebook, not a project folder. The work is analysis: read a year of timestamps, count, simulate and compare. A notebook shows each step with its output. It runs on free Colab in about 2 minutes and needs no Docker and no database. One optional part, Check the model on your computer, replays a few minutes of the trace against your own ticket-api. A model of a queue should meet a real queue at least once.

Before you startWhat to know and to have readyAbout 150 minutes

What you need to know

This case study uses the whole course. These lessons matter most:

• A workload stated in numbers (arrival rate, bursts, task time), from Workload model.
• Percentiles, and why you promise a p95 or p99 and not a mean, from Latency and capacity.
• Queued and running jobs, and the oldest job’s age, from Job lifecycle.
• The queue bound, 503 queue_full and Retry-After, from Backpressure.
• What the quota does to more workers, from Scaling signals.
• A recommendation that does not extrapolate, from Capacity recommendation.

The page teaches one thing that the course did not: a queue simulation that replays real arrival times.

What you need on your computer

• For the notebook: nothing. It opens in Colab or Kaggle, installs pandas 3.0.6, and downloads the data (about 4.4 MB). In VS Code or Jupyter, use a virtual environment with pandas 3.0.6, numpy and matplotlib. The notebook works in a new folder capacity-plan; to start again, delete that folder.
• For the optional replay only: the course project at the end of Capacity recommendation, with PostgreSQL and Valkey running. If yours is broken, download the course’s snapshot end-of-m06 and follow its README.

Tested on macOS with Python 3.12.13, pandas 3.0.6, numpy 2.5.3 and matplotlib 3.11.2, in Jupyter from a fresh start. The replay ran on macOS (Apple silicon) with the project at the end of Module 6. Windows and Linux are not tested; the commands are the course’s commands.

The situation

Nadia runs the digital team of a city’s 311 line. Residents call, use an app or fill in a web form to report a pothole, graffiti or a blocked sidewalk. Each report becomes a service request, and agents route it to the right department.

Nadia wants the AI work of this course on every new request. The service classifies it, drafts a reply for the agent, and computes an embedding to find similar requests. It is Larkfield’s ticket-api design: a request is a ticket, and its AI work is a job in the queue. She asks her engineer, Leo, two questions:

• “How many workers do we need, so that no request waits more than 10 seconds before its AI work starts?”
• “What happens when the AI provider is down for a while?”

Leo has no traffic of his own yet, but San Francisco publishes every 311 request with its time. So he plans with that trace, and says so: Nadia’s city must check its own numbers before it relies on the plan.

What you will build

You will build a capacity plan from a real trace. You check the data, measure the arrivals at three time scales and find the bursts. A model of a queue replays the arrivals with the course’s recorded AI job times, and you choose a number of job slots for the 10-second target. You test that choice once on months that you kept aside, add an outage and a queue bound, and run the method on new input with plan().

The data

The data is 311 Cases from DataSF, the open-data portal of the City and County of San Francisco. It has one row per request since 2008. The case study uses the 872,427 requests of 2025, and the requests of June 2026 as new input at the end.

Source and licence

“311 Cases”, City and County of San Francisco, DataSF, dataset vw6y-z8j6, under the Open Data Commons Public Domain Dedication and License 1.0. You may use, change and share it without conditions. Nextia Learning kept three columns (the time, the channel and the request type). We removed the request ID and the milliseconds, which were always zero, and froze a copy, because the live dataset changes every day. Nothing else changed.

Leo compared two other open traces. Boston’s 311 data has the same licence. But it has about a third of San Francisco’s volume. And in 2025 it changed systems, so the year is split between two files with different columns. The NASA web server log of 1995 is a classic traffic trace. But it has no named licence, and its rows are web hits, not tickets. Every line also holds a visitor’s host name. New York’s 311 data is used by the next case study, Queue and cache for an open-data API.

The notebook’s setup cell downloads the files and checks their checksums. Leo loads the 2025 file:

raw = pd.read_csv("sf311_2025_arrivals.csv.gz", parse_dates=["requested_datetime"],
                  keep_default_na=False, na_values=[""])
print(raw.shape)
print(raw.head(3).to_string(index=False))
print(raw.requested_datetime.min(), "to", raw.requested_datetime.max())

The output:

(872427, 3)
 requested_datetime source                 service_name
2025-01-01 00:00:36 Mobile Street and Sidewalk Cleaning
2025-01-01 00:01:08 Mobile Street and Sidewalk Cleaning
2025-01-01 00:05:36 Mobile Street and Sidewalk Cleaning
2025-01-01 00:00:36 to 2025-12-31 23:58:10

What this tells us. One row is one request: when the city’s system recorded it, the channel, and the type. The times are to the second. There are no names, addresses or descriptions in this copy, so no person can be found from it.

The data dictionary

Column	What it means	Use
requested_datetime	When the city’s system recorded the request, in San Francisco local time, to the second	The arrival time of one ticket
source	The channel: Phone, Mobile (the app), Web, Integrated Agency, Twitter, Email, or Test	Checks, and finding the bursts
service_name	The type, for example Graffiti Public or Street and Sidewalk Cleaning	Explaining a burst

First checks

Before any counting, Leo checks the channels, empty values, repeats and one day with a clock change:

print(raw.source.value_counts(dropna=False).to_string())
print("rows with no source:", raw.source.isna().sum())
print("same time, channel and type as another row:", raw.duplicated().sum())
t = raw.requested_datetime
march_9 = raw[t.dt.date == pd.Timestamp("2025-03-09").date()]
print("2025-03-09, requests per hour 0-4:", march_9.groupby(march_9.requested_datetime.dt.hour).size().head(4).to_dict())

The output:

source
Mobile               534032
Web                  181124
Phone                147841
Integrated Agency      6139
NaN                    1985
Twitter                 653
Test                    512
Email                   141
rows with no source: 1985
same time, channel and type as another row: 20446
2025-03-09, requests per hour 0-4: {0: 21, 1: 23, 3: 9, 4: 9}

What this tells us. Each finding changes a later step:

• 512 Test rows. Someone tested the system; no resident sent them. They would never reach a real queue, so Leo removes them.
• 1,985 rows without a channel. They are real requests of every type, spread over the whole year. A missing label does not make a request less real, so they stay.
• 20,446 rows repeat another row’s time, channel and type. They are not copies: two requests about graffiti can arrive in the same second. These groups are the bursts that this case study is about, so they stay.
• No hour 2 on 9 March. That night the clocks moved forward one hour. The times are local and have no time zone. In November one hour happens twice, so it holds two hours of requests. Both are night hours with about 20 requests, so Leo keeps local time and notes it.

Frame the problem

The unit. One request is one ticket and one AI job. The question. How long does a job wait in the queue before a worker takes it? That wait is the number to control. The oldest job’s age, the signal from Scaling signals, is the same idea seen from the queue. It reaches its peak just before the oldest job starts, so the peak age equals the longest wait.

The target. No ticket waits more than 10 seconds. A job then takes about 2 to 3 seconds more, so the agent sees the category and the draft reply within about 13 seconds.

A split in time. A plan made from one period must work in the next one. So Leo uses January to September, the design months, to choose the plan. He keeps October to December for one final check, as a model is tested once on data it never saw. A random split would be wrong here: bursts come in runs, and the plan must face months that it did not see.

df = raw[raw.source != "Test"].copy()
T0 = pd.Timestamp("2025-01-01")
df["sec"] = (df.requested_datetime - T0).dt.total_seconds()  # seconds since 1 January
design = df[df.requested_datetime < "2025-10-01"]
check = df[df.requested_datetime >= "2025-10-01"]
sec = design.sec.to_numpy()
print(len(df), "requests:", len(design), "design months,", len(check), "check months")

The output:

871915 requests: 663926 design months, 207989 check months

What this tells us. Leo has 273 days to plan with and 92 days to check, and the 512 test rows are gone. Every number from here on is for the design months, until the final check.

Explore what the decision needs

The day and the week

days = design.groupby(design.requested_datetime.dt.date).size()
print(f"per day: mean {days.mean():.0f}, lowest {days.min()} ({days.idxmin()}), highest {days.max()} ({days.idxmax()})")

per day: mean 2432, lowest 1375 (2025-07-04), highest 3750 (2025-07-23)

The notebook also averages the requests for each hour of the day, separately for weekdays and weekends:

• Weekdays
• Weekends

San Francisco 311, January to September 2025: the average number of requests in each hour of the day. Weekdays peak at about 239 an hour at 9 in the morning; nights fall to about 9.

Text description of the chart

A line chart of two lines over the 24 hours of the day. On weekdays, requests rise from about 9 an hour at 3 in the morning to a peak of about 239 an hour at 9 o’clock. They stay above 150 an hour until about 4 in the afternoon. By 10 at night they fall to about 40 an hour. On weekends, the line has the same shape but is lower: its peak is about 188 an hour at 10 in the morning.

What this tells us. A busy weekday morning hour holds about 25 times the requests of a night hour. So the average, about 1.7 requests a minute, describes no real minute.

Three time scales

A design point is the load that you plan for. The average is the wrong one, because it hides the peaks, as Latency and capacity showed for request times. But which peak, and at which time scale? Leo counts the requests in every hour, every minute and every 10 seconds of the design months, empty windows included:

def per_window(seconds, width):
    """How many requests arrived in each window of `width` seconds (empty windows count too)."""
    start = np.floor(seconds.min() / 86400) * 86400
    return np.bincount(((seconds - start) // width).astype(int))

rows = []
for name, width in [("per hour", 3600), ("per minute", 60), ("per 10 seconds", 10)]:
    counts = per_window(sec, width)
    rows.append([name, counts.mean(), *np.percentile(counts, [50, 99, 99.9]), counts.max()])
print(pd.DataFrame(rows, columns=["window", "mean", "p50", "p99", "p99.9", "max"]).round(2).to_string(index=False))

The output:

        window   mean  p50   p99  p99.9  max
      per hour 101.33 94.0 310.0 397.84  572
    per minute   1.69  1.0   8.0  12.00   21
per 10 seconds   0.28  0.0   3.0   5.00   13

What this tells us. Each time scale tells a different story. The busiest hour had 572 requests, about 5.6 times the mean hour. The busiest minute had 21, about 12 times the mean minute. And the busiest 10 seconds had 13, about 46 times the mean. The shorter the window, the further the peak is from the mean. So which window matters? The one that matches the work and the target. A job takes about 2.3 seconds and the target is 10 seconds, so a queue fills and empties in seconds. 10-second windows are the right lens. An hourly plan would never see the problem.

Bursts inside the minute

If residents acted alone, each second of the minute would get about 1/60 of the requests: 1.7%. Leo checks each channel:

second = design.requested_datetime.dt.second
for source in ["Phone", "Mobile", "Web"]:
    counts = second[design.source == source].value_counts()
    print(f"{source:6}: second :{counts.idxmax():02d} has {counts.max() / counts.sum():.1%} of the requests")
same = design.groupby("requested_datetime").size()
print("seconds with 5 or more requests:", (same >= 5).sum(), "holding", same[same >= 5].sum(), "requests")
print("the largest group:", same.max(), "requests at", same.idxmax())
print(design[design.requested_datetime.isin(same[same >= 5].index)].source.value_counts().to_dict())

The output:

Phone : second :31 has 1.8% of the requests
Mobile: second :45 has 4.0% of the requests
Web   : second :47 has 38.6% of the requests
seconds with 5 or more requests: 1095 holding 6379 requests
the largest group: 10 requests at 2025-07-15 10:51:47
{'Web': 6305, 'Mobile': 61, 'Phone': 13}

• Web
• Mobile
• Phone

San Francisco 311, all of 2025: the share of each channel's requests recorded at each second of the minute. Phone is flat; Web has a tall spike at second 47.

Text description of the chart

A line chart with three lines over the seconds 0 to 59. The phone line is flat at about 1.7% for every second. The mobile line is almost flat, with a small rise to about 3.5% between seconds 44 and 49. The web line is about 1% for most seconds and jumps to about 30% at second 47 over the whole year, with smaller rises at seconds 46, 48 and 51.

What this tells us. Phone requests are spread evenly: people call when they want. Web requests are not. In the design months, 38.6% of them carry second 47 of their minute. The city’s system seems to record many web requests once a minute, in a batch. What matters is the effect: a third of the web requests arrive together, up to 10 in the same second. For the AI queue, the batch is the arrival. The jobs start when the system records the requests, not when the residents pressed “send”.

A baseline: the arithmetic of the average

The simplest baseline is arithmetic: requests per second, times seconds per job, gives the jobs in flight on average. One job is three calls in a row (classify, draft a reply, embed). Each time is drawn from the course’s 100 recorded times of that call, plus 17 ms of database work:

lat = {task: np.array(ms) / 1000 for task, ms in calibration["latency_ms"].items()}
rng = np.random.default_rng(21)
job = sum(rng.choice(lat[task], 100_000) for task in ["classify", "draft_reply", "embed"]) + 0.017
print(f"one job: median {np.median(job):.2f} s, p95 {np.percentile(job, 95):.2f} s, max {job.max():.2f} s, mean {job.mean():.2f} s")
per_minute = per_window(sec, 60)
for name, rate in [("mean minute", per_minute.mean()), ("p99 minute", np.percentile(per_minute, 99)),
                   ("busiest minute", per_minute.max())]:
    print(f"{name:15} {rate:5.2f} requests -> {rate / 60 * job.mean():.2f} job slots busy on average")

The output:

one job: median 2.25 s, p95 3.08 s, max 4.63 s, mean 2.32 s
mean minute      1.69 requests -> 0.07 job slots busy on average
p99 minute       8.00 requests -> 0.31 job slots busy on average
busiest minute  21.00 requests -> 0.81 job slots busy on average

What this tells us. A job slot is one job in flight. In the course project, the slots are the workers times WORKER_CONCURRENCY: one worker with the default of 4 has 4 slots. The arithmetic says that even the busiest minute keeps less than one slot busy. Tomás would say: “One worker is plenty, maybe too many.” That is the baseline. The next sections test it against the real arrivals.

The queue model

A counter at the bakery

A bakery has one person at the counter, and each order takes about two minutes. If thirty customers come in an hour, one every two minutes, nobody waits. If the same thirty come as a school class at 8:00, the last child waits an hour. The number per hour is the same; the wait is not. A wait depends on how the arrivals bunch together, and on how long each one takes.

The arithmetic of the average cannot see bunching. So Leo uses a queue simulation: a short program that replays each arrival in time order. It gives each one to a free slot or puts it in the queue, and records how long it waited. It is a model of the course’s queue, with the four parts in the diagram.

The capacity model of this case study: real arrivals, a queue with an optional bound, a number of job slots, and the provider with its quota. The job times are the course's recordings.

Text description of the diagram

The model has four parts in a row. Arrivals come from the real trace, one ticket per request. They enter the queue, which keeps them in order. If the queue has a bound and is full, the ticket is refused with 503 queue_full. A free job slot takes the oldest ticket. The slots are workers times WORKER_CONCURRENCY. Each job calls the provider three times in a row, classify, draft_reply and embed, with times drawn from the course’s 100 recorded calls of each task. The two chat calls count against the quota of 100 chat calls per minute; when it is used up, the call waits. During an outage, no new call starts until the outage ends. The model records each ticket’s wait, the jobs waiting, and the tickets refused.

The function simulate() in the notebook is about 70 lines of Python. It keeps a clock and a list of the next events: “a ticket arrives”, “a call ends”, “the quota has room again”. It always handles the earliest event first. Its quota is a token bucket, as in the course’s simulated provider. The bucket holds 100 chat calls and refills at 100 per minute, and each chat call takes one token.

Two simplifications make the model a little optimistic. When the bucket is empty, the call waits for a token. The real worker gets a 429 first, which costs about 0.13 seconds and a backoff. And the model has no database or network delay apart from the 17 ms. The replay on your computer measures what these leave out.

A tiny example

Leo tries the model on cases that he can check by hand. First, four tickets 30 seconds apart. Then ten tickets in the same second, with one slot and with two:

calm = [0, 30, 60, 90]
burst = [0] * 10
print("calm, 1 slot :", np.round(simulate(calm, 1, calibration)["wait"], 1))
for slots in [1, 2]:
    print(f"burst, {slots} slot(s):", np.round(simulate(burst, slots, calibration)["wait"], 1))

The output:

calm, 1 slot : [0. 0. 0. 0.]
burst, 1 slot(s): [ 0.   2.3  5.6  7.7 10.  12.6 14.9 17.2 19.7 22.4]
burst, 2 slot(s): [ 0.   0.   2.3  3.3  4.4  5.6  7.   7.8  9.4 10.3]

What this tells us. The calm tickets never wait: each job ends long before the next ticket arrives. In the burst with one slot, each ticket waits for all the jobs before it, about 2.3 seconds each, so the tenth waits 22.4 seconds. Two slots halve that. You can check it by hand: 9 jobs ahead × about 2.3 s ÷ 2 slots is about 10 s. Ten tickets in one second fit easily into an average minute, and they still miss the target.

Three workload models, one queue, one metric

Now the real question: which description of the arrivals gives the right answer? Leo compares three workload models. All three go through the same queue, with the same job times and the same seed. One metric judges them: the longest wait in the design months, and the count of tickets over 10 seconds.

• Even. The same 663,926 requests, evenly spaced. This is the arithmetic of the average, as a simulation.
• Random. Random arrivals, with the design months’ average for each hour of the week, so a Monday at 9:00 has its own average. Arrivals that are random and independent like this are a Poisson process, the textbook model of a queue.
• Trace. The real times, batches and all.

TARGET = 10  # seconds: the longest a ticket may wait before a slot takes it

def summary(result):
    wait = result["wait"][~result["refused"]]
    return {"p99": np.percentile(wait, 99), "max": wait.max(), "over": int((wait > TARGET).sum())}

models = {"even": np.linspace(sec.min(), sec.max(), len(sec)),
          "random": random_arrivals(T0, 273, seed=21), "trace": sec}
rows = [{"model": name, "slots": slots, **summary(simulate(arrivals, slots, calibration))}
        for slots in [1, 2, 3, 4] for name, arrivals in models.items()]

The notebook defines random_arrivals() just before this cell, and prints two tables:

longest wait (s):
slots      1     2    3    4
model
even     0.0   0.0  0.0  0.0
random  10.0   3.2  2.3  0.6
trace   25.1  11.3  6.9  4.9
tickets that waited more than 10 s:
slots      1   2  3  4
model
even       0   0  0  0
random     1   0  0  0
trace   2893  14  0  0

What this tells us. The three models give three different plans:

• Even says one slot, and no ticket ever waits. It cannot see a peak, so it is no plan at all.
• Random says one slot almost works (one ticket over 10 seconds in nine months), and two are plenty. It knows the daily shape. But it assumes that residents act alone, so it never makes a batch of 10 in one second.
• Trace says one slot fails 2,893 times, and two slots still miss 14 times, with a longest wait of 11.3 seconds. Three slots meet the target.

The difference between random and trace is the batches. The textbook model is a good start, but here it is too kind: its longest waits are about half of the real ones or less. Use the real arrivals when you have them.

The operating point

One burst, second by second

The longest waits come from the batches. Leo replays the largest one, 10 requests at 10:51:47 on 15 July, with the two minutes before and after it, and plots the oldest job’s age:

• 1 slot
• 2 slots
• 4 slots

The queue model, replaying the largest batch of the design months (10 requests at 10:51:47 on 15 July 2025) with 1, 2 and 4 job slots (seed 21). The oldest job's age rises while the batch waits, and falls when the last job of the batch starts. The target is 10 seconds.

Text description of the chart

A line chart from 10 seconds before the batch to 60 seconds after it. Before the batch, nothing waits and every line is at 0. At second 0, ten tickets arrive. With 1 slot, the oldest job’s age climbs to about 21.2 seconds before the last ticket starts, about 21 seconds after the batch. With 2 slots, it peaks at about 10.1 seconds, right at the target. With 4 slots it peaks at 4.5 seconds. (With 3 slots, not drawn, the peak is 6.5 seconds.) All lines are back at 0 within about 25 seconds.

What this tells us. One slot keeps the batch waiting for about 21 seconds. Two slots sit right at the target; three and four stay clear.

Five passes, not one

The job times are random draws, so one run is one sample, as one load-test pass is. Leo runs each choice five times with different seeds and keeps the median, as Compare configurations does:

for slots in [2, 3, 4]:
    print(f"design months, {slots} slots:", passes(sec, slots))

design months, 2 slots: {'median max': 11.3, 'max range': (10.5, 11.6), 'median over': 12}
design months, 3 slots: {'median max': 7.0, 'max range': (6.8, 7.4), 'median over': 0}
design months, 4 slots: {'median max': 4.9, 'max range': (4.7, 5.1), 'median over': 0}

What this tells us. The answer is stable: two slots miss in every pass, three meet the target in every pass with about 3 seconds to spare, and four leave about 5 seconds.

The choice

Three slots are the smallest number that meets the target. Leo recommends four, for two reasons:

• A slot is cheap here. A worker mostly waits for the provider. Four slots hold about four database connections. They cost nothing extra at the provider, because the same requests make the same calls.
• Nine months are a sample, not the future. The design months contain the bursts that happened, not every burst that can happen. A margin is cheap insurance.

In the project, four slots can be one worker with the default WORKER_CONCURRENCY=4, or two workers with WORKER_CONCURRENCY=2. Leo prefers two workers. When one is replaced, as in Graceful shutdown, the other keeps two slots running. Two slots missed the target 12 times in nine months. That is acceptable for the few minutes of a replacement.

The final check, once

Now Leo opens October to December for the first time and runs the plan, once:

check_sec = check.sec.to_numpy()
check_ten = per_window(check_sec, 10)
print("check months: busiest 10 seconds", check_ten.max(), "requests (design months: 13)")
for slots in [2, 3, 4]:
    print(f"check months, {slots} slots:", passes(check_sec, slots))

The output:

check months: busiest 10 seconds 22 requests (design months: 13)
check months, 2 slots: {'median max': 16.8, 'max range': (15.6, 16.9), 'median over': 10}
check months, 3 slots: {'median max': 9.0, 'max range': (8.4, 9.6), 'median over': 0}
check months, 4 slots: {'median max': 5.5, 'max range': (5.2, 5.7), 'median over': 0}

• Design months (January to September)
• Check months (October to December)

The queue model with 2, 3 and 4 job slots: the longest wait in the design months and in the check months. The target is 10 seconds. Three slots only just held in the check months; four kept a margin.

Text description of the chart

A column chart with six columns, three for the design months and three for the check months. Design months: 2 slots 11.3 seconds, 3 slots 7 seconds, 4 slots 4.9 seconds. Check months: 2 slots 16.8 seconds, 3 slots 9 seconds, 4 slots 5.5 seconds. Every column of the check months is higher than the same column of the design months.

What this tells us. The check months held a burst bigger than any in the design months: 22 requests in 10 seconds, against 13. It came on Saturday 6 December at 14:54, and it was not a web batch. In 9 seconds, 22 General Request reports came from the app. The data cannot say whether that was one person, a group, or an app that sent again. With it, three slots reached 9.0 seconds, just under the target, and two slots failed badly. Four slots, chosen with a margin, kept 4.5 seconds to spare. This is why the final check runs once, on months that the plan never saw. Had Leo tuned the plan on all twelve months, he would have no honest test left.

Check the model on your computer

A model should meet a real queue at least once. This optional part is the only one that runs the course project. It replays four minutes of the trace, around the largest batch, against your local ticket-api, and compares the measured waits with the model’s. The provider is the course’s simulated provider, quota on. Run it only against the services on your own computer.

Caution

The replay sends 44 tickets in 4 minutes, and each pass takes about 5 minutes. Do not run it while another load test runs on your computer. Stop the worker when you finish.

1. Download replay_trace.py from the case study’s folder in the labs repository. Save it in your project’s scripts/ folder.

2. In the project folder, activate the virtual environment. Start PostgreSQL, Azurite and Valkey if they do not run:

   docker compose up -d db azurite cache

3. In a second terminal, start the simulated provider:

   python -m simulator

4. In a third terminal, start the API:

   fastapi dev

5. In a fourth terminal, start one worker with one slot:

   python -m ticket_api.worker --concurrency 1

   You should see Worker …: up to 1 job at a time.

6. Empty the cache, so that no embedding comes from an earlier run:

   python -m scripts.cache flush

7. Replay the four minutes. The first run downloads the trace (about 4 MB):

   python -m scripts.replay_trace --start "2025-07-15 10:49:47" --minutes 4

The author’s run, pass 1 of 3 (your numbers will differ):

Replaying 44 requests from 2025-07-15 10:49:47 for 4 minutes (largest same-second group: 10 at 10:51:47) ...
Sent: status codes {202: 44}
Jobs: {'succeeded': 44}
Wait before a worker took the job (s): median 7.5, p95 19.1, max 22.2
Most jobs waiting at once: 10

Then stop the worker (Ctrl+C), start it again with --concurrency 2, empty the cache, and replay the same four minutes. The author ran each setting three times, and compared the medians of the passes with the model’s medians of five seeds:

Waits of the same 44 tickets	1 slot: measured	1 slot: model	2 slots: measured	2 slots: model
Median	7.9 s	7.8 s	2.8 s	2.5 s
p95	19 s	17.9 s	8.9 s	8.5 s
Longest	21 s	20 s	9.5 s	9.2 s

What this tells us. The model and the real queue agree within about a second. The real queue is a little slower every time: its median, p95 and longest wait are 0.1 to 1.1 seconds above the model’s. That fits what the model leaves out. An idle worker looks for new jobs every 0.5 seconds (WORKER_POLL_SECONDS), and each job also spends time in the API and the database. So the model is slightly optimistic, and the margin of the fourth slot covers it. Note one limit of this check: the real queue used the simulated provider, so it confirms the queue, not the provider.

The quota, an outage and a queue bound

The quota. The busiest minute of the design months needs 2 × 21 = 42 chat calls, and the account allows 100. So on a normal day the quota never limits the plan. The headroom is 100 ÷ 42, about 2.4 times the busiest minute ever seen. That number also marks where this plan stops.

An outage. For Nadia’s second question, Leo assumes an outage of 30 minutes from the start of the busiest hour (27 August, 9:00, 572 requests). This is a stated scenario, not a measurement. While the provider is down, jobs wait; the course’s worker gives them back without spending an attempt. Afterwards, the queue empties as fast as the quota allows: 100 chat calls a minute, or 50 tickets.

for name, slots, kw in [("4 slots", 4, {}), ("8 slots", 8, {}), ("4 slots, quota 200", 4, {"rpm": 200}),
                        ("4 slots, bound 500", 4, {"bound": 500}), ("4 slots, bound 100", 4, {"bound": 100})]:
    result = simulate(win, slots, calibration, outage=outage, **kw)
    ...

The output:

busiest hour: 2025-08-27 09:00:00 with 572 requests
4 slots             peak 273 waiting, refused   0, longest wait 29.2 min, empty 5.5 min after the outage
8 slots             peak 269 waiting, refused   0, longest wait 29.2 min, empty 5.5 min after the outage
4 slots, quota 200  peak 273 waiting, refused   0, longest wait 29.2 min, empty 3.0 min after the outage
4 slots, bound 500  peak 273 waiting, refused   0, longest wait 29.2 min, empty 5.5 min after the outage
4 slots, bound 100  peak 100 waiting, refused 177, longest wait 29.2 min, empty 1.5 min after the outage

• Quota 100
• Quota 200
• Bound

The queue model with 4 job slots during a 30-minute outage from 9:00 on 27 August 2025, the busiest hour (a stated scenario). The queue grows while the provider is down; after it, the quota sets how fast it empties. A queue bound of 100 (Bound) holds the queue flat and refuses the rest.

Text description of the chart

A line chart from 5 minutes before the hour to 50 minutes after it. Before the outage, nothing waits. From minute 0 to 30, the queue grows steadily as requests arrive, to about 273 jobs at minute 30, with both quotas. After the outage, with quota 100, it falls to 0 about 5.5 minutes later. With quota 200 it falls faster and is empty about 3 minutes after the outage. With a bound of 100, the line climbs to 100 and stays flat until the outage ends, while 177 requests are refused. It is empty about 1.5 minutes after the outage.

What this tells us.

• More workers do not help after an outage. Four slots and eight slots empty the queue in the same 5.5 minutes, because the quota is the limit, not the slots. This is the lesson of Scaling signals again: add what the bottleneck needs. Here that is a larger quota: with 200 calls a minute, the queue is empty in 3.0 minutes.
• A tight bound turns waiting into refusing. The course’s default QUEUE_MAX of 500 refuses nothing: the queue peaks at 273. A bound of 100 keeps the queue short, but it refuses 177 requests. In the course’s design, a 503 queue_full means that the ticket is not saved.

That last point matters more for a city than for a shop. A refused 311 request is a resident’s report of a problem, lost because the AI work was behind. The Backpressure lesson named the alternative: save the ticket, and skip or delay its AI work when the queue is full. For a public service, that is the design to choose. The AI work helps the agents; it must never be the reason that a report is lost.

The plan on new input, end to end

A plan must work on data that it has not seen, and say when it cannot. The function plan() in the notebook runs the whole method on a new file: the checks, the three time scales, the quota, five passes with the chosen slots, and a verdict. It refuses to extrapolate. The plan was checked on bursts of up to 22 requests in 10 seconds and 22 in a minute. If the new trace has a bigger burst, or needs more chat calls than the quota, plan() says “measure again” and gives no number.

Leo runs it on June 2026, a month that came after the whole analysis:

june = plan("sf311_2026_06_arrivals.csv.gz")

The output:

73719 requests, 2026-06-01 to 2026-06-30
per minute: mean 1.71, p99 8, max 21 (checked up to 22)
per 10 seconds: max 12 (checked up to 22)
busiest minute: 42 chat calls (quota 100)
4 slots: {'median max': 4.0, 'max range': (3.5, 4.2), 'median over': 0}
Verdict: 4 slots meets the 10 s target on this input.

What this tells us. June 2026 looks like 2025, with bursts inside the checked range, and four slots keep the longest wait at about 4 seconds. One detail changed: the web batch now arrives at second 53, not 47. A plan built on “second 47” would have broken; a plan built on the measured arrivals did not.

The capacity recommendation

Leo writes it for Nadia, in the form of Capacity recommendation:

• Recommendation. Run two workers with WORKER_CONCURRENCY=2: four job slots. In the model, with San Francisco’s arrivals, the longest wait was 4.9 seconds in the design months. It was 5.5 seconds in the held-out months and 4.0 seconds in June 2026 (medians of five passes).
• What it assumes.
  • The arrivals are like San Francisco’s in 2025 and June 2026: up to 22 requests in 10 seconds and 22 in a minute.
  • The job is the course’s: three calls in a row, with the recorded times (median 2.25 s, p95 3.08 s). The real provider got slower under load (the course measured +7% at 16 calls in flight). At four slots that is small, and the model does not include it.
  • The quota is 100 chat calls per minute, and nothing else uses the same account.
  • The city’s system keeps its batches at about 10 requests or fewer.
• What it does not cover. Bursts larger than 22 in 10 seconds; more than 50 requests in a minute (the quota); another city’s traffic; a provider outage, which no number of workers fixes.
• The outage. A 30-minute outage at the busiest hour leaves about 273 jobs waiting, and the quota empties them in about 5.5 minutes. Keep QUEUE_MAX at 500 or more, and change the full-queue answer to “save the request, skip the AI work”. To recover faster, ask for a larger quota, not more workers.
• What to measure next. Run this notebook’s plan() on the arrivals of Nadia’s own city. Then load-test the real service with a replay of the worst burst, as in Load-test design. In production, watch the oldest job’s age as an alarm.

Responsible use

• The data is about real residents. The copy has no names, addresses or descriptions, and the method needs none. Do not join it with the full dataset to study individual people. The 22 app reports of 6 December may come from one person: treat them as a load pattern, not as someone to investigate.
• A model of San Francisco is not a model of another city. Reuse the method elsewhere, not the numbers.
• Never let the AI queue lose a report. A public service must save every request, even when the AI work cannot keep up.
• Say what is simulated. The provider in the replay is simulated, and the outage is a scenario. Label both.

Limits and next steps

• One city, one year and one month. Seasons, storms and events of other years may bring bursts that 2025 did not have. Run plan() on each new month, and alarm on the oldest job’s age.
• The timestamps are the system’s, not the residents’. The batches show that the recorded time is sometimes later than the moment of sending. For the queue, that is the right time. For a study of residents’ behaviour, it is not.
• The model leaves things out: the 429 and the backoff when the quota binds, the worker’s poll interval, the database, and a provider that slows under load. The local replay measures some of them; a load test of the real service measures the rest.
• The outage is assumed. Use the provider’s real incident history if you have it.

Make it a portfolio piece

This case study shows an employer that you can turn a vague question (“how many workers?”) into a number that you can defend. It also shows that you know where the number stops. Make it yours with one change: another city’s open 311 data (check its licence first) or another target.

What to put in the repository:

• README.md (template below).
• The notebook, run from top to bottom, with its outputs and charts.
• CAPACITY.md: the recommendation, with every assumption and what it does not cover.
• replay_trace.py and your replay results, if you ran the local check.
• The data source and licence. PDDL needs no credit, but credit DataSF anyway, and give the checksum of your copy.

A README template:

# Capacity plan for AI triage of 311 requests

## Question
How many job slots keep every request's wait before its AI work under 10 s,
and what happens when the AI provider is down?

## Data and licence
San Francisco 311 Cases (DataSF, vw6y-z8j6), 2025 and June 2026: time, channel,
type. ODC PDDL 1.0. Test rows removed; local time.

## Approach
Arrivals at three time scales; bursts inside the minute (web batches at one
second of each minute). A queue simulation with recorded AI job times
(median 2.25 s) and a 100-calls-per-minute quota. Three workload models
(even, random, trace) compared on Jan-Sep; final check once on Oct-Dec.

## Results
Trace model: 3 slots meet 10 s on Jan-Sep (longest 7.0 s); the check months
had a bigger burst (22 in 10 s): 3 slots 9.0 s, 4 slots 5.5 s. Recommended:
4 slots. A 30-minute outage: 273 waiting, empty 5.5 min after (quota-bound).

## Limits
One city; simulated provider; outage assumed; no extrapolation beyond
22 requests in 10 s or 50 a minute.

How to describe it in an interview, in two sentences: “I planned the AI workers for a 311 service from a year of real request times: the average said one job slot, but the real bursts needed three, and I recommended four. A held-out quarter with a bigger burst showed why the margin mattered, and an outage scenario showed that the quota, not the workers, sets the recovery time.”

Go further:

• Replay the 6 December burst on your computer with four slots, and compare it with the model.
• Add batches to the random model (a “compound” Poisson process), and see whether it predicts the check months better.

Knowledge check

Check your understanding

The busiest minute of the design months had 21 requests, which keeps less than one job slot busy on average. Why did one slot still fail the 10-second target 2,893 times?

Show the answer

The average over a minute hides how the requests bunch inside the minute. Up to 10 requests arrived in the same second, because the city’s system records many web requests in a batch once a minute. With one slot, the tenth job of a batch waits for nine jobs of about 2.3 seconds each: about 21 seconds. The time scale of the design point must match the work and the target. A job takes seconds and the target is 10 seconds, so the right lens is the count per 10 seconds, or a replay of the real arrivals.

Check your understanding

Three slots met the target in the design months. Why did Leo recommend four, and what did the final check show?

Show the answer

Nine months of data are a sample of the bursts that happened, not every burst that can happen. A slot costs little here, because a worker mostly waits for the provider. So he chose a margin. The check months, opened once, had a bigger burst than any in the design months: 22 requests in 10 seconds, against 13. Three slots reached 9.0 seconds, just under the target; four slots reached 5.5 seconds. Without the margin, the plan would have held only by luck.

Check your understanding

After a 30-minute outage, Nadia wants to double the workers so that the queue empties faster. What do you answer?

Show the answer

It will not help. In the model, 4 slots and 8 slots both empty the queue 5.5 minutes after the outage. The limit is the provider’s quota of 100 chat calls per minute (50 tickets), not the workers. A larger quota helps: with 200 calls a minute, the queue is empty in 3.0 minutes. Also keep the queue bound high enough: 500 refuses nothing here, and 100 would refuse 177 requests. And save every request, even when its AI work must wait.

Recap

• You checked a real trace: test rows out, missing channels kept, local time with its clock changes, and same-second groups that turned out to be batches.
• You measured the arrivals at three time scales and chose the one that matches the job and the target: 10 seconds, not minutes or hours.
• You compared three workload models through one queue model. The average said one slot, the textbook random model said two, and the real arrivals said three.
• You chose four slots with a margin, and the one final check on held-out months showed why it was needed.
• After an outage, the quota sets the recovery, and a tight queue bound refuses residents’ reports. plan() refuses to extrapolate beyond what the plan was checked on.

Next, the other case study: an open-data API under load, with a cache and a background export job.

Next: Queue and cache for an open-data API