Workload model

What you will be able to do

Turn Omar’s “ten times the usual tickets” into numbers that a test can use. How fast do tickets arrive, how big are the bursts, how long and how large is each one, and what do customers expect?

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• The AI work per ticket, the simulated provider and your first Locust run, from Run the AI workload on your computer.
• The large practice data of the databases course (300,000 tickets), from Access-pattern inventory.

What you need on your computer

• Your project ~/projects/ticket-api from the last lesson: python -m pytest prints 123 passed. PostgreSQL and Azurite run (docker compose up -d db azurite).
• About 1 GB of free disk space for the large data. The first load downloads its vectors (15 MB).

A bakery before a holiday

A bakery owner hears that the town festival will bring “ten times more customers” on Saturday. How many loaves should she bake? How many people should work?

“Ten times” is not enough to plan with. She needs to know more:

• When do they come? Spread over ten hours, or all at once when the parade ends at noon?
• What do they order? A loaf takes ten seconds to sell. A wedding cake takes twenty minutes.
• How long does each order take at the counter and in the oven?
• How long will a customer wait before leaving the line?

With those answers, she can plan. She can even practise: ask friends to arrive at noon, all at once, and time the line.

The problem: “ten times” is not a test

Omar says the help desk will get about ten times the usual tickets on Black Friday. Tomás has a quick plan: “Let’s send 1,000 tickets to the API and see what happens.”

But 1,000 tickets over what time? Compare two readings of “ten times”, both from the same data. Each ticket needs two chat calls (classify and draft a reply), and the quota is 100 chat calls per minute:

• The busiest hour, ten times: 1,010 tickets in one hour. That is 17 tickets per minute, so 34 chat calls per minute. It fits in the quota, with room to spare.
• The busiest minute, ten times: 90 tickets in one minute, so 180 chat calls. That is almost twice the quota.

Both are “ten times”. One says that the service is fine; the other says that about half of the tickets fail. A test without the time scale has no answer. And if Tomás picks the scale by chance, nobody can repeat his test next week.

The idea: a workload model

A workload model is a short written description of the requests that a service must handle. It has five parts:

1. Arrival rate: how many requests arrive per second, on average, in the period that you plan for.
2. Burst size: the most requests in a short window, such as one minute. Real traffic is never smooth.
3. Task duration: how long one request needs, when nothing else is waiting.
4. Payload size: how big each request is. A longer text costs more tokens and time.
5. What users expect: the waiting time that users accept, in their words.

It is called a model because it is a simplified copy of the real traffic: it keeps what matters for the test and drops the rest. And every part is a number, or an assumption that someone wrote down and can check later.

A tiny example: the busiest hour, ten times

The large practice data has 300,000 made-up tickets over one year. The busiest hour of the year had 101 tickets. Work through Omar’s “ten times” for that hour, one step at a time:

Step	Calculation	Result
Tickets in the busiest hour	from the data	101
Ten times (Omar’s estimate)	101 × 10	1,010 tickets
Arrival rate per second	1,010 ÷ 3,600 s	0.28 tickets per second
Chat calls per minute	1,010 × 2 ÷ 60	34 per minute
Compared with the quota	34 ÷ 100	34% of the quota

Now the burst. The busiest single minute of the year had 9 tickets:

Step	Calculation	Result
Ten times	9 × 10	90 tickets in one minute
Arrival rate per second	90 ÷ 60 s	1.5 tickets per second
Chat calls in that minute	90 × 2	180
Compared with the quota	180 ÷ 100	180% of the quota

The average hour is safe. The burst is not. A workload model needs both numbers, because they lead to different decisions.

Chat calls per minute that the help desk needs, from the large practice data. The busiest hour fits in the quota even at ten times. The busiest minute at ten times needs almost twice the quota.

Text description of the chart

Four columns, in chat calls per minute. The busiest hour of a normal year: 3.4. The busiest hour at ten times: 34. The busiest minute at ten times: 180. The provider’s quota: 100. Only the burst goes over the quota.

Why bursts happen even in a calm hour

A rate of 101 tickets per hour sounds smooth: about 1.7 per minute. It is not. Customers act on their own, so some minutes get none and some get several. The chart counts the 60 minutes of the busiest hour by how many tickets arrived in each.

The busiest hour of the large data (16 September 2026, 19:00 to 20:00 UTC): 101 tickets, but not 1.7 in every minute. Ten minutes had none, and four minutes had four.

Text description of the chart

For each number of tickets per minute, from 0 to 4, the number of minutes in the busiest hour: 0 tickets in 10 minutes, 1 ticket in 17, 2 in 19, 3 in 10, and 4 in 4 minutes. The average is 1.7 tickets per minute.

The busiest minute of this hour had four tickets, more than twice the average. Over the whole year, the busiest minute had nine. That is why the burst is a separate part of the model. A test that sends a smooth average never meets the minute that breaks the quota.

The day itself is not smooth either. On 16 September, the day of the busiest hour, tickets came from 07:00 to 20:59 UTC. The hours were very different:

Tickets per hour on 16 September 2026, the day with the busiest hour of the large data (times in UTC). The 19:00 hour had 101 tickets; quiet hours had about 31.

Text description of the chart

Tickets per hour on 16 September 2026, from 07:00 to 20:00 UTC: 67, 72, 69, 33, 76, 75, 31, 74, 68, 72, 37, 73, 101 and 35. The 19:00 hour is the busiest of the whole year.

The other three parts

Task duration. You measured it in the last lesson: one ticket takes about 2.2 seconds alone, almost all of it waiting for the provider. Write down where the number came from: the recorded provider times, one call at a time.

Payload size. scripts.workload measures the ticket texts: median 125 characters, about 31 tokens. The longest is 196 characters. Small texts mean that the size of a request is not a problem here. In another service, such as one that summarises long documents, the payload size could be the main part of the model.

What users expect. This one comes from people, not from data. Amira asks Omar. He thinks for a moment: “A customer must see within 5 seconds that the ticket was saved. The help desk can wait a few minutes for the draft reply.” Write his words into the model as they are. They make the promise of this course: a new ticket is saved within 5 seconds. An answer that comes later is not useful work. Later lessons test against this promise, and Module 4 changes the design because of it.

Larkfield’s workload in numbers

Now run the measurement yourself.

1. In ~/projects/ticket-api, with the virtual environment active, copy this lesson’s two files from the snapshot end-of-m01:

   macOS / Linux

   S=../nextia-learning-labs/C21/snapshots/end-of-m01
   cp "$S/scripts/workload.py" scripts/
   cp "$S/docs/workload.md" docs/

   Windows (PowerShell)

   $S = "..\nextia-learning-labs\C21\snapshots\end-of-m01"
   Copy-Item "$S\scripts\workload.py" scripts\
   Copy-Item "$S\docs\workload.md" docs\

2. Run the script on the small data, which is loaded now:

   python -m scripts.workload

   Tickets: 200 from 2026-09-25 to 2026-10-08 (14 days)
   Per day: 14 on average, 23 on the busiest day
   Per hour with tickets: 1.7 on average, 5 in the busiest hour
   Busiest minute: 2 tickets
   Ticket text (subject + body): median 126 characters (about 32 tokens), p95 164, longest 183
   Busiest hour: 5 tickets = 0.00 per second on average
   Chat calls to the AI provider in that hour: 0 per minute (the quota is 100 per minute)

   Two weeks of 200 tickets cannot describe Black Friday. You need a year of data.

3. Load the large data. It takes about half a minute, plus the download of its vectors the first time:

   python -m scripts.load --size large --reset

   The last lines list the counts, starting with customers 20,000 and tickets 300,000.

4. Describe the large data, first as it is, then ten times bigger:

   python -m scripts.workload
   python -m scripts.workload --factor 10

   Tickets: 300,000 from 2025-10-09 to 2026-10-08 (365 days)
   Per day: 822 on average, 899 on the busiest day
   Per hour with tickets: 58.7 on average, 101 in the busiest hour
   Busiest minute: 9 tickets
   Ticket text (subject + body): median 125 characters (about 31 tokens), p95 169, longest 196
   Busiest hour x 10: 1,010 tickets = 0.28 per second on average
   Chat calls to the AI provider in that hour: 34 per minute (the quota is 100 per minute)

   The output above is the one with --factor 10. Without it, the last two lines say 101 tickets = 0.03 per second and 3 per minute.

5. Open docs/workload.md. It is the worked example of this lesson as a table. Each row has a value for a normal day, a value for Black Friday, and its source. Find the row “What users expect”. It holds Omar’s words and the promise of 5 seconds. Compare it with your own model.

6. Load the small data again. The load tests of this module use it:

   python -m scripts.load --reset

The script counts only the practice tickets. The tickets from your load tests start at T-500001, and the script leaves them out, so a test never changes the model.

From a model to a repeatable workload

A model on paper is not yet a test. A repeatable workload is a script that sends the same requests, in the same way, every time you run it. Then two runs differ only in what you changed.

loadtest/locustfile.py, which you ran in the last lesson, fixes these parts of the test:

Part	How the locustfile fixes it
The request	POST /v1/tickets with a subject and a body
The payload	the subjects and bodies of the 200 practice tickets, picked with the fixed seed 21
The senders	-u users; each one sends, waits for the answer, then sends again
The length	-t 60s, and the first seconds left out as a warm-up (next lesson)

One part is not fixed by the locustfile: the arrival rate. Each Locust user waits for its answer before it sends the next ticket. So with 4 users, at most 4 tickets are ever in progress. This kind of test is a closed loop: the answers control when the next requests come.

Real customers are an open stream. They do not wait for each other. On Black Friday, 90 customers in a minute send 90 tickets, whether the API has answered the first ones or not.

A closed loop waits for answers; an open stream does not. Module 1's tests are closed loops.

Text description of the diagram

In the closed loop on the left, four users each wait for their answer, about 2.2 seconds, before they send again. A slow API slows the senders down, so the test never sends more than four tickets at once. In the open stream on the right, customers arrive at their own rate: 90 in the busiest minute is one every 0.7 seconds. They keep coming, whether or not the API has answered.

Sam asks, “So is the closed loop wrong?” No. A closed loop is the right tool to measure how fast the service works at a fixed number of requests in flight. That is the job of the next two lessons. But a closed loop hides one danger: when the API slows down, the test slows down too, so the queue never grows. Module 4 sends an open stream at a fixed rate, and you will see what happens there.

When to use a workload model

Write a workload model before every load test, and before every capacity decision. It costs an hour, and it prevents the most common mistake in performance work: a test that answers a question that nobody asked.

Keep each model small. Five numbers and their sources are enough. A model with fifty numbers looks precise, but nobody checks it.

Know its limits. The ×10 is Omar’s estimate, not data. The model says so, in the column “where it comes from”. After Black Friday, compare the real day with the model, and correct the next one. A model that is never checked is only a guess with a table around it.

Common mistakes

Common mistake

The workload says 0.00 per second

Symptom: python -m scripts.workload prints Busiest hour: 5 tickets = 0.00 per second on average and 0 per minute.

Cause: the small data is loaded: 200 tickets over 14 days. The numbers are correct for that data, but they are too small to describe a real help desk.

Diagnose: read the first line of the output. It says Tickets: 200 from 2026-09-25 to 2026-10-08 (14 days).

Fix: load the large data with python -m scripts.load --size large --reset, and run the script again. Afterwards, load the small data again with python -m scripts.load --reset.

Common mistake

A model from the average hour only

Symptom: the model says “0.28 tickets per second, 34% of the quota: no problem”. Then on the real day, many tickets fail with errors.

Cause: the model has the arrival rate but no burst size. The busiest minute at ten times needs 180 chat calls, almost twice the quota.

Fix: always put the burst next to the rate. In this data, a burst means the busiest minute. For a service whose limit counts per second, look at the busiest second too.

Your turn: fifteen times

Omar calls back: the marketing team now plans a larger campaign, and he expects fifteen times the usual tickets. Predict the arrival rate in the busiest hour and the chat calls per minute. Then predict the chat calls in the busiest minute. Check the hour with python -m scripts.workload --factor 15 on the large data.

Show the answer

The busiest hour: 101 × 15 = 1,515 tickets, so 1,515 ÷ 3,600 = 0.42 tickets per second, and 1,515 × 2 ÷ 60 = about 50 chat calls per minute. That is still inside the quota. The author’s run printed:

Busiest hour x 15: 1,515 tickets = 0.42 per second on average
Chat calls to the AI provider in that hour: 50 per minute (the quota is 100 per minute)

The busiest minute: 9 × 15 = 135 tickets, so 270 chat calls in one minute: 2.7 times the quota. The script does not print the burst at the new factor, so you compute it. The hour still fits, and the burst gets much worse.

See it end to endCapacity plan from a public traffic traceCase study

The traffic-trace case study compares three workload models (even, random and the real trace) on a year of San Francisco’s 311 requests. The even and random models say that one or two job slots are enough; the real trace, with its bursts, needs three.

Knowledge check

Check your understanding

Omar's model says 1,010 tickets in the busiest hour. Tomás says that this is only 34 chat calls per minute, so the quota of 100 is no problem. What is missing from his reasoning? Use the numbers of this lesson.

Show the answer

The burst. Tickets do not arrive evenly: the busiest minute of the year had 9 tickets, against an average of 1.7 per minute in the busiest hour. At ten times, that minute has 90 tickets and needs 180 chat calls, almost twice the quota of 100. So the hourly average fits, but the burst does not. A workload model needs the burst size next to the arrival rate.

Check your understanding

A Locust test with 4 users never sends more than 4 tickets at once. Explain why, and say why this can hide a problem that real customers would cause.

Show the answer

Each Locust user sends a ticket, waits for the answer, and only then sends the next one. This is a closed loop, so at most 4 tickets are in progress. Real customers are an open stream: they send tickets at their own rate, whether the API has answered or not. If the API slows down, a closed loop sends less, and the queue never grows. An open stream keeps sending, and the queue grows.

Check your understanding

Practical check: write the five parts of the workload model for Larkfield's Black Friday, each with its value and its source.

Show the answer

Arrival rate: 1,010 tickets in the busiest hour = 0.28 per second (scripts.workload --factor 10; the ×10 is Omar’s estimate, an assumption). Burst: 90 tickets in the busiest minute = 180 chat calls, against a quota of 100 (9 × 10, from the same script). Task duration: about 2.2 s per ticket alone (the recorded provider times). Payload: median 125 characters, about 31 tokens, longest 196 (scripts.workload). Users expect: the ticket saved within 5 seconds, the draft reply within a few minutes (Omar). Your docs/workload.md has the same values.

Recap

• A workload model has five parts: arrival rate, burst size, task duration, payload size and what users expect, each with its source.
• Larkfield’s Black Friday at ten times: 0.28 tickets per second in the busiest hour (34 chat calls per minute), but 90 tickets in the busiest minute (180 chat calls, against a quota of 100).
• Traffic is not smooth: even the busiest hour had empty minutes and minutes with four tickets. The burst decides whether the quota holds.
• A repeatable workload sends the same requests the same way every time. The locustfile is a closed loop: users wait for answers. Real customers are an open stream.
• The ×10 is an assumption. Write it down, and check it after the day.

Next, you measure the service under this workload: averages, percentiles and the first baseline.

Next: Latency and capacity