Demand assumptions

Lesson 4
55 min
Module 1: Start with requirements

Lesson 4 of 19 · AI System Design and Cost EngineeringDemand assumptions

Course overview

1. Start with requirements

2. Design the end-to-end architecture

3. Estimate resources and cost

4. Optimize with measured trade-offs

5. Design for failure and growth

6. Communicate the proposal

7. Case studies

What you will be able to do

Estimate how much the assistant will be used: shops, people, questions, documents, the busiest minute and growth. Nobody knows these numbers yet, so write each one as a range with its evidence, and mark which ones you checked.

Before you startWhat to know and to have readyAbout 55 minutes

What you need to know

The three kinds of numbers and the design pack, from Set up the design pack.
The five journeys, from Users and workflows, and the targets Q1 to Q8, from Quality attributes.
A workload model: arrivals per second, the busiest hour and bursts, from Workload model in Scaling APIs and AI Workloads.

What you need on your computer

Your folder design-pack with its active virtual environment, and python -m pytest -q printing 6 passed.
The labs folder nextia-learning-labs next to it, from the first lesson. You copy two new files of the calculator from it.

Tested with Python 3.12 on macOS (Apple silicon). The Windows (PowerShell) and Linux commands are written, not tested.

Food for a wedding

A caterer plans the food for a wedding. The couple does not know yet how many guests will come. Some have answered the invitation; most have not. So the caterer asks for three numbers: “What is the fewest, the most likely and the most?” The couple says 80, 100 and 120. Then the caterer asks a second question: “Which of these do you know?” Sixty guests have answered yes. The other 40 to 60 are a guess.

The caterer also asks when people eat. A wedding does not spread its eating over the day. Almost everyone wants dinner at seven. So the kitchen must cook 100 dinners in about an hour, not 100 dinners over twelve hours. The busiest hour decides the kitchen, the ovens and the staff.

Three habits make the caterer’s plan safe. She uses a range instead of one number, she notes what is known, and she plans for the busiest moment. This lesson gives the assistant the same three habits.

The problem: one number, or a million

Tomás has a confident suggestion: “Let’s design for a million users. Then we never have to worry.” A million users who each ask 1.5 questions would ask 1.5 million questions a day. Kwame’s base case, which you build in this lesson, is 1,465 a day. Tomás’s design is for about a thousand times that. Every part would cost more and do almost nothing.

The opposite mistake is just as common: one number, with no range. Suppose Kwame designs for “800 customers ask questions each day”. Then Omar puts the assistant on the home page, and twice as many customers ask, three questions each. In this lesson’s high case, the busiest hour has 1,116 questions instead of 180: about six times more. A design sized for one number cannot say what happens then.

Both mistakes have the same cause: the demand is unknown, and the design hides that.

The idea: demand as ranges, with evidence

A demand assumption is an estimate of how much people will use the system. It covers shops, people, questions and documents, and how fast they grow. Write each one as an assumption range of three values:

low: the fewest that you can reasonably expect;
base: the most likely value, the one you plan around;
high: the most that you can reasonably expect, not the most you can imagine.

Why three values? Because a design is judged at all three. The base case sets the cost that Omar quotes. The high case shows what breaks first. The low case shows the fixed costs that you pay even if few people come.

Each value also needs its evidence, which says where it comes from. It starts with one of two words:

verified: you checked it, and the text says how. “Grace’s help-desk rota has 10 agents.”
assumed: your best guess, and the text says why. “Omar’s sales plan: 2% more questions every month.”

A verified value is still an estimate of the future, but it rests on something that you can check. The assumed values are where the proposal is most likely to be wrong, so the team checks them first.

Plan for the busiest minute

Demand per day is not enough, as the caterer knows. The system must have the capacity for its busiest moment. Capacity is how much work a part of the system can do in a fixed time, such as questions per second. Module 3 computes the capacity that the assistant needs. This lesson gives it the demand at the peak, with two numbers:

The busiest hour’s share: the share of an average day’s questions that arrive in its busiest hour.
The busiest minute factor: how much faster questions arrive in the busiest minute than on average in that hour. Even in a calm hour, arrivals come in small bursts.

Both come from the scaling course’s ticket workload, which is synthetic data. In it, an average day had 822 tickets and the busiest hour had 101: a share of 0.123. The busiest minute ran 5.35 times faster than that hour’s average. The pack uses these values as the base, with a range around them.

Measure the share against an average day, because the pack multiplies an average day by it. The busiest day of that workload had 899 tickets. A share of the busiest day (101 ÷ 899 = 0.112) would make every peak about 9% too low.

A tiny example: one shop’s questions and its busiest minute

Here is Larkfield’s demand in the base case, with every value visible. Questions come from two kinds of people, the journeys J1 and J2:

Staff (J1): 10 agents × 20 questions per agent per day = 200 questions.
Customers (J2): 800 customers who ask × 1.5 questions each = 1,200 questions.
Larkfield, base: 200 + 1,200 = 1,400 questions a day.

The low and high cases use the same formula with the other values of each range:

Case	Staff	× questions	Customers	× questions	Questions a day
Low	8	10	300	1.0	80 + 300 = 380
Base	10	20	800	1.5	200 + 1,200 = 1,400
High	12	40	1,600	3.0	480 + 4,800 = 5,280

The high case is not 20% more than the base. It is almost four times more, because three uncertain numbers multiply. That is why a single number is dangerous: the uncertainty grows when assumptions multiply.

Now add Bramble Books’ base case: 2 staff × 10 + 30 customers × 1.5 = 65 questions a day. Together, the two shops ask 1,465 questions a day. The diagram takes this day to its busiest minute:

From a day to the busiest minute, base case. The design must handle the last box, not the average of the first.

Text description of the diagram

The chain starts with a day of 1,465 questions, the base case for both shops. The busiest hour gets 0.123 of the day: 1,465 × 0.123 = 180 questions. Spread evenly over 60 minutes, that is 3.0 questions a minute. But arrivals come in bursts. The busiest minute is 5.35 times faster than the hour’s average: 3.0 × 5.35 = 16.1 questions. That is 0.268 questions per second, in the last box.

So the base case’s busiest moment is about one question every four seconds. That is a small number, and it is the honest answer to “a million users”. Module 3 turns 0.268 per second into the number of answers in progress at the same time. Then it checks that number against the provider’s quota.

Larkfield’s demand file

Kwame writes these ranges in demand.toml, the pack’s demand file. He asks each person for the number that they know best. Grace gives the agents and their questions; Omar the customers and the growth; Ines her shop; Mei the documents. Here is Larkfield’s table from the file:

[tenants.larkfield]
name = "Larkfield"
joins_in_month = 0
staff_users = [8, 10, 12]
staff_questions_per_day = [10, 20, 40]
customers_per_day = [300, 800, 1600]
customer_questions_per_day = [1.0, 1.5, 3.0]
documents = [36, 36, 60]
words_per_document = [150, 181, 400]
document_changes_per_month = [4, 8, 20]
user_growth_per_month = [0.0, 0.02, 0.05]

Each line is a range [low, base, high]. The value joins_in_month = 0 means that the shop uses the assistant from the start. The growth is a share, not a percentage: 0.02 means 2% more questions every month. The documents and their length matter for storage and ingestion in Module 3.

Under the table, every value has its evidence:

[tenants.larkfield.evidence]
staff_users = "verified: Grace's help-desk rota has 10 agents; 8 to 12 with holidays and the busy season"
staff_questions_per_day = "assumed: Grace's estimate after a one-week trial; check it with the usage log in the first month"
customers_per_day = "assumed: about one asking customer per new ticket today (822 tickets a day); 0.4 to 2 times that"
customer_questions_per_day = "assumed: no data yet; a customer usually asks one question and a follow-up"
documents = "verified: 36 documents in the collection today; 60 if the product guides are added next year"
words_per_document = "verified: 181 words on average in today's collection (the RAG course's documents)"
document_changes_per_month = "assumed: Mei's estimate from last year's policy updates"
user_growth_per_month = "assumed: Omar's sales plan (2% a month); 5% if the assistant goes on the home page"

Read the evidence, not only the numbers. The three verified values are things that somebody counted. The five assumed values are about behaviour that has not happened yet. The evidence of staff_questions_per_day already says how to check it: “with the usage log in the first month”. A good assumption carries its own test.

Bramble Books and two future shops follow in the same form. The third shop joins in month 4 and the fourth in month 9. No shop has signed yet, so all their values are assumed. In the whole file, 7 values are verified and 27 are assumed. That ratio is normal before a launch, as long as the 27 are visible, with reasons.

Guided practice: fill in demand.toml

You add the calculator’s demand code from the snapshot, write Larkfield’s ranges yourself, and then take the other shops from the snapshot.

In the terminal, in your folder design-pack, with the environment active, copy the two new files of the calculator from the snapshot end-of-m01:
- macOS / Linux
- Windows (PowerShell)
Terminal window
S=../nextia-learning-labs/C26/snapshots/end-of-m01 cp "$S/costmodel/demand.py" costmodel/ cp "$S/tests/test_demand.py" tests/
Terminal window
$S = "..\nextia-learning-labs\C26\snapshots\end-of-m01" Copy-Item "$S\costmodel\demand.py" costmodel\ Copy-Item "$S\tests\test_demand.py" tests\
demand.py reads demand.toml, checks every range and its evidence, and prints the demand table.
Run the demand command. Your demand.toml is still the template, so it fails with a clear message:
Terminal window
```
python -m costmodel demand
```
```
Input problem: demand.toml has no tenants yet
```
Open demand.toml. Below the [settings] tables, delete the commented example. Write the table [tenants.larkfield] and its [tenants.larkfield.evidence] from the section above. Type the evidence in your own words, but start each line with verified: or assumed:.

Run the demand command again:

python -m costmodel demand

Demand in month 0 (questions per day; low / base / high)
  Larkfield             380    1,400    5,280
  All tenants           380    1,400    5,280
  Busiest hour           38      157    1,056
Evidence: 5 values verified, 5 assumed (check the assumed ones first)

The three Larkfield numbers are the ones that you computed by hand in the tiny example. The 5 verified values are Larkfield’s 3 and the 2 settings.

Run the tests: python -m pytest -q. One test fails: 1 failed, 11 passed. The failing test, test_the_pack_demand_counts_verified_and_assumed, expects both shops that use the assistant today, Larkfield and Bramble Books.
Copy the snapshot’s demand.toml over yours. It has your Larkfield table, Bramble Books and the two future shops. If you opened a new terminal, set S again as in step 1. Then run the tests again:
- macOS / Linux
- Windows (PowerShell)
Terminal window
cp demand.toml demand-mine.toml cp "$S/demand.toml" demand.toml python -m pytest -q
Terminal window
Copy-Item demand.toml demand-mine.toml Copy-Item "$S\demand.toml" demand.toml python -m pytest -q
```
............                                                             [100%]
12 passed in 0.01s
```
Compare demand-mine.toml with the new file. The Larkfield numbers must be the same; the evidence words can differ. Then delete demand-mine.toml.

Print the demand for both shops:

python -m costmodel demand

Demand in month 0 (questions per day; low / base / high)
  Larkfield             380    1,400    5,280
  Bramble Books          15       65      300
  All tenants           395    1,465    5,580
  Busiest hour           40      180    1,116
Evidence: 7 values verified, 27 assumed (check the assumed ones first)

Print the demand in a year, in month 12:

python -m costmodel demand --month 12

Demand in month 12 (questions per day; low / base / high)
  Larkfield             380    1,776    9,482
  Bramble Books          15       93      755
  Third shop             25      228    2,039
  Fourth shop            25      207    1,598
  All tenants           445    2,304   13,874
  Busiest hour           44      283    2,775
Evidence: 7 values verified, 27 assumed (check the assumed ones first)

Check Larkfield’s month 12 yourself. The base growth is 2% a month for 12 months: 1,400 × 1.02¹² = 1,400 × 1.268 = 1,776. The low case does not grow, because its growth is 0. The high case grows by 5% a month: 5,280 × 1.05¹² = 9,482.

Read the ranges

The demand table is easier to judge as a picture. The chart compares the three cases now and in month 12, for all shops together:

Now (month 0, two shops)
Month 12 (four shops)

Demand for all shops, low, base and high, now and in month 12. The base case grows from 1,465 to 2,304 questions a day; the high case is about 3.8 times the base now and 6 times in month 12.

Text description of the chart

A column chart with six columns: low, base and high now, in one colour, then low, base and high in month 12, in another colour. Now: low 395, base 1,465, high 5,580 questions a day. In month 12, with four shops: low 445, base 2,304, high 13,874. The low columns hardly change; the base column grows by about 57%; the high column grows to more than twice its height now.

The gap between base and high grows with time, because the high growth rate compounds for longer, and the new shops add their own high cases. The further ahead you look, the less you know, and the range shows it.

The second chart shows the same three cases month by month:

High
Base
Low

Questions a day for all shops, month by month. The steps at months 4 and 9 are the new shops; between them, the base and high cases grow a little every month.

Text description of the chart

A line chart from month 0 to month 12 with three lines. The low line starts at 395 and steps up to 420 in month 4 and 445 in month 9. It is flat in between, because the low case has no growth. The base line starts at 1,465 and grows slowly. It jumps from 1,557 to 1,784 in month 4, when the third shop joins. It jumps from 1,934 to 2,168 in month 9, when the fourth shop joins, and ends at 2,304. The high line starts at 5,580, grows faster, jumps in months 4 and 9, and ends at 13,874.

Each new shop adds about 195 questions a day in the base case, about as much as half a year of growth. So the dates when the shops join matter as much as the growth rate. Both dates are assumed: no shop has signed yet.

Finish the requirements

The last section of requirements.md, Verified and assumed, summarises the evidence for readers who do not open demand.toml. Kwame writes three points:

Verified: Larkfield’s 10 agents and 36 documents; Bramble Books’ 3 documents today. The first answer-quality and latency numbers come from real runs (measured.toml).
Assumed (check these first): the customers who ask, their questions, the growth, the new shops’ dates, document changes, and repeated questions. Each one is a range with its reason.
Open question for Omar: will customers see the assistant on the home page? That changes customer questions by a factor of about 2: the high scenario.

Write this section in your own words, and compare the whole file with the snapshot’s requirements.md. Your pack is now the snapshot end-of-m01, except for your wording.

When to use ranges

Use them for every number about the future: users, questions, documents, growth, dates. Use them also when someone gives you a single number with confidence. Ask “what is the fewest, and what is the most?”, as the caterer did.

Do not use a range for a value that you can count today. Larkfield’s documents, [36, 36, 60], are a range only because next year may add more. And do not make the high case the worst that you can imagine. The high case is the most that you can reasonably expect.

Ranges have a cost: every later number becomes three numbers, and the proposal must explain all three. Omar needs that, to know how wrong the base case could be.

Common mistakes

Common mistake

Growth written as a percentage

Symptom: you write user_growth_per_month = [0, 2, 5], meaning 2% and 5%. The calculator stops:

Input problem: larkfield.user_growth_per_month is a share: 0.02 means 2%

Cause: the formula multiplies by (1 + growth) for every month. With 2 instead of 0.02, the questions would triple every month: 1,400 × 3¹² is about 744 million questions a day. The calculator refuses a growth above 1 to catch this.

Fix: write shares: [0.0, 0.02, 0.05].

Common mistake

A value without evidence, or a range in the wrong order

Symptom: the calculator stops with one of these messages:

Input problem: [tenants.larkfield]: 'documents' needs evidence that starts with 'verified:' or 'assumed:'

Input problem: 'larkfield.customers_per_day': low <= base <= high is not true for [800, 300, 1600]

Cause: in the first case, the evidence line says “36 documents …” without verified: or assumed:. The pack does not accept a number unless it says what kind it is. In the second case, the low and base values are swapped.

Fix: start each evidence line with verified: or assumed:. Write each range as [low, base, high], from small to large.

Reset. If your pack breaks, copy the snapshot end-of-m01 as the snapshots README shows. It has everything from this module.

Your turn: the home page

Change one assumption. Omar decides to put the assistant on Larkfield’s home page. He now expects 1,600 asking customers a day in the base case, not 800. Before you run anything, predict Larkfield’s base demand, the total for both shops, and the busiest hour. Then change the base value of Larkfield’s customers_per_day to 1600, run python -m costmodel demand, and change it back.

Show the answer

Larkfield: 10 × 20 + 1,600 × 1.5 = 200 + 2,400 = 2,600 questions a day. Both shops: 2,600 + 65 = 2,665. The busiest hour: 2,665 × 0.123 = 328.

Demand in month 0 (questions per day; low / base / high)
  Larkfield             380    2,600    5,280
  Bramble Books          15       65      300
  All tenants           395    2,665    5,580
  Busiest hour           40      328    1,116
Evidence: 7 values verified, 27 assumed (check the assumed ones first)

The base demand grows by 82%, from 1,465 to 2,665, but it stays inside the range: the high case was already 5,580. That is the value of a range. Omar’s decision moves the base case. It does not take the design outside what Kwame planned for. Write the change in the evidence too: “assumed: home page from month 0, Omar’s decision”. The tests still pass, because they check the file’s form, not Omar’s numbers. Change the value back to 800, because the pack keeps Omar’s question open.

Module practice: a system-design brief

You now have everything for a system-design brief. It is one page, for a reader who is not an engineer. It says what the system must do, for whom, how well and how much, and what is still a guess.

A worked example, small. Here is a brief for a much smaller system: the assistant for Bramble Books alone, with the numbers from this module.

Problem. Ines and one helper answer every customer question at Bramble Books. Customers ask about supplier terms and returns before they write. We want an assistant that answers from Bramble Books’ own documents, with citations. Journeys. J1: Ines or her helper asks while helping a customer. J2: a customer asks on the website and gets public documents only, or “the documents do not say”. J3: Ines uploads a changed document; it is searchable in minutes. Out of scope. Refunds and order changes; documents from other shops; a mobile app. Targets. 95% of answers in 6 s or less, every week (first estimate 4.6 s). At least 85% correct on Ines’s evaluation set. No other shop ever sees Bramble Books’ documents. Demand. 65 questions a day (low 15, high 300); 25 documents (low 3, high 60). Busiest hour: 65 × 0.123 = 8 questions. Verified: 2 people; 3 documents today. Check first: customers per day (Ines’s guess from web traffic), and how many documents she really adds.

The brief computes the busiest hour in front of the reader. Each target names its period, and the last line says which numbers to check first.

Your task. Write the brief for the whole shared assistant, for Omar, on one page. Use your requirements.md and python -m costmodel demand. Include these parts:

the problem, the journeys and what is out of scope;
the four targets that matter most to Omar;
the demand now and in month 12, as ranges;
the three assumptions to check first.

Show a worked solution

Problem. Larkfield’s assistant answers questions from Larkfield’s documents with citations. Bramble Books wants the same, and two more shops may join next year. We propose one shared assistant. Each shop sees only its own documents, and the cost per shop can go into a price. Journeys. J1 agents ask while helping a customer. J2 customers ask on the website: public documents only, or “the documents do not say”. J3 editors upload documents; searchable in minutes, old versions kept. J4 owners see every answer with its citations and versions. J5 a new shop needs no code change. Out of scope. Internet answers; actions such as refunds; sharing documents between shops; languages other than English and French; a mobile app. Targets. Q1: 95% of answers in 6 s or less, every week (first estimate 4.6 s, an upper estimate). Q2: 99.5% of questions answered in shop hours, each month. Q3: no shop ever sees another’s documents (tests on every change). Q8: at most US$150 a month in the first year. Demand. Now: 1,465 questions a day (low 395, high 5,580). Busiest hour: 180 (40 to 1,116). Busiest minute, base case: about 0.27 questions per second. Month 12, with four shops: 2,304 a day (low 445, high 13,874). Evidence. 7 values verified, 27 assumed. Check first: (1) customers per day: they ask most of the questions. (2) When the new shops sign: each one adds about half a year of growth. (3) Omar’s home-page decision: it moves the base case by about 80%.

Your brief may choose other targets, for example Q4 instead of Q2. Check three things. Each target names a number, a period and how it is measured. The demand is a range. And the reader can see which numbers are verified.

Knowledge check

Check your understanding

Module completion check: for each statement, say whether it is verified or an assumption that needs validation, and how you would check the assumptions. (a) Larkfield has 10 help-desk agents. (b) 800 customers a day will ask a question. (c) The third shop joins in month 4. (d) The collection has 36 documents of 181 words on average. (e) Each agent asks 20 questions a day. (f) The busiest hour has 0.123 of an average day's questions.

Show the answer

(a) Verified: Grace’s rota. Nothing to check now; the range 8 to 12 covers holidays.
(b) Assumed: based on 822 tickets a day, but nobody has asked the assistant yet. Check it in the first month with the usage log: count customers with at least one question per day.
(c) Assumed: Omar’s sales plan; no shop has signed. Check it with the sales pipeline; a signed contract replaces it.
(d) Verified: counted in the collection.
(e) Assumed: Grace’s estimate after a one-week trial. Check it with the usage log, per agent per day.
(f) Verified, with a limit: it matches the scaling course’s ticket workload, which is synthetic: 101 tickets in the busiest hour of an average day of 822. It is the best shape available, but real questions may peak differently, for example at lunch time. Check it with the first month’s logs; the high case already allows 0.20.

The assumed values need a check with a date and an owner. The verified ones need only a reason. Statement (f) shows that “verified” means “checked against something”, and the proposal should say against what.

Check your understanding

Predict and explain: why is Larkfield's high case (5,280 questions a day) almost four times its base case (1,400), when no single value in its ranges is more than twice its base?

Show the answer

Because the high case multiplies several high values. Customers go from 800 to 1,600 (×2) and their questions from 1.5 to 3.0 (×2), so customer questions grow ×4: 1,200 → 4,800. Staff questions grow ×2.4 (200 → 480), but they are a small part. So the total grows from 1,400 to 5,280, about ×3.8. When uncertain numbers multiply, their uncertainty multiplies too. A single “safe margin” of 20% would badly underestimate the high case.

Check your understanding

Tomás says: 'Design for a million users, so we never worry.' Answer him with this module's numbers, in two or three sentences.

Show the answer

The base case is 1,465 questions a day for both shops, and about 0.27 questions per second in the busiest minute. Even the high case in month 12 is 13,874 a day. A million users at 1.5 questions each ask 1.5 million a day: a hundred times the high case. We would pay for capacity that nobody uses, and we still would not know what limits the assistant. So we design for the range, and the growth plan in Module 5 says what to change when demand reaches a measured signal.

Recap

A demand assumption is an estimate of use: shops, people, questions, documents, growth. Write each one as an assumption range, low, base and high, with evidence that starts with verified: or assumed:.
Larkfield’s base case: 10 agents × 20 + 800 customers × 1.5 = 1,400 questions a day. With Bramble Books: 1,465 (low 395, high 5,580). In month 12, with four shops: 2,304 (low 445, high 13,874).
Plan for the busiest minute, not the day. The busiest hour has 1,465 × 0.123 = 180 questions (the share is measured against an average day). The busiest minute has 180 ÷ 60 × 5.35 = 16.1, or 0.268 questions per second.
In demand.toml, 7 values are verified and 27 assumed. The proposal lists the assumptions to check first: customers per day, the new shops’ dates, and the home page.
python -m pytest -q prints 12 passed. Your pack is now the snapshot end-of-m01.

Module 1 in one paragraph. You set up the design pack, where every number is measured, a price or an assumption, and says which. You wrote what the assistant must do as five journeys, and what it will not do. You turned “fast”, “always on”, “private” and “cheap” into eight targets with a way to measure each. For example, 95% of answers in 6 seconds, with a first estimate of 4.6. And you estimated demand as ranges: 1,465 questions a day in the base case, 0.268 per second at the busiest minute. The 27 assumptions are written down for the team to check.

Next, Module 2 draws the architecture that serves these journeys and targets, one part per job.

Next: Service boundaries