Stakeholder presentation
- Lesson 18
- 50 min
- Module 6: Communicate the proposal
Lesson 18 of 19 · AI System Design and Cost EngineeringStakeholder presentation
1. Start with requirements
2. Design the end-to-end architecture
3. Estimate resources and cost
4. Optimize with measured trade-offs
5. Design for failure and growth
6. Communicate the proposal
What you will be able to do
Explain the proposal in ten minutes to the people who decide and the people who use it. Say what it does for them, what it costs, what we assumed, what can go wrong, and what they must decide. Every number says what kind of number it is.
Before you startWhat to know and to have ready
What you need to know
- The decision records, from Architecture decisions.
- The scenarios and the sensitivity table, from Unit economics: Read the scenarios.
- The container diagram, from Service boundaries: Larkfield’s parts, and the busiest minute in flight, from Workload arithmetic.
What you need on your computer
- Your folder
design-packfrom the last lesson, withpython -m costmodel checkprinting7 of 9 passed.
Tested with Python 3.12 on macOS (Apple silicon). The Windows (PowerShell) and Linux commands are written, not tested.
The mechanic’s five minutes
Section titled “The mechanic’s five minutes”You take your car to a garage because the brakes make a noise. The mechanic does not show you the repair manual. She says five things.
“The front brake pads are worn; new pads stop the noise and make the car safe. It costs about US$180. The discs look fine, but I can only see one side of the left one. If it is worn too, add about US$120. The risk of waiting is a longer stop on a wet road. Do you want me to open the left wheel and check today?”
In under a minute, you know the benefit and the price with its range. You know what she cannot see yet, the risk, and the one decision that is yours. You did not need to know how a brake works. And you trust her more because she said what she could not see.
The problem: the right numbers, the wrong message
Section titled “The problem: the right numbers, the wrong message”Kwame’s first draft of the presentation has 31 slides. It shows the cost table with 14 drivers, Little’s law, the routing results to three decimals, and every row of demand.toml. He rehearses it with Grace.
The next day, Grace tells Omar: “The calculator says a question costs US$0.000157. That is almost nothing.” Omar writes to the owner of a third shop: “About 1,950 questions a month would cost you about 31 cents.”
The number was correct and the message was wrong. US$0.000157 is the variable cost per question only. 87% of the monthly cost is fixed. The always-on API replica costs US$27.43, the database server US$13.50 and its storage US$4.05. Those costs are there with zero questions. A fair price for a shop today is closer to US$26 a month, about 84 times the promise. The deck had the right table; it did not say which number answers Omar’s question, and what kind of number it is.
The idea: a walkthrough built on the listeners’ questions
Section titled “The idea: a walkthrough built on the listeners’ questions”A stakeholder is a person who decides about the system, pays for it, or works with it. Larkfield’s stakeholders in this meeting are Omar, who pays; Grace, whose agents use the assistant; and Ines, who owns Bramble Books.
A walkthrough is a short spoken tour of the design for these people, with a few pictures. It is not the design pack read aloud. The pack stays the source of every number, and the walkthrough chooses the few that answer the listeners’ questions. Four rules make it work:
- Start from their questions. Omar asks “what will it cost per shop?” Grace asks “is it fast and correct?” Ines asks “can another shop see my supplier terms?” Each part of the talk answers one of these.
- Say the kind of every number, rounded, with its range. “About US$52 a month, from US$41 to US$76; a calculation from measured times, list prices and assumed demand.” Two significant digits are enough to decide.
- Show what you do not know. The assumptions that move the answer most, and the risks, each with what limits the damage.
- End with an ask. One decision that the listeners make today, often an unresolved assumption: a guess that the team cannot check alone.
The third and fourth rules are the ones that people skip, and they are the ones that build trust. The mechanic who says “I cannot see the left disc” is believed about the pads.
A tiny example: one output line, translated
Section titled “A tiny example: one output line, translated”Take five lines of the pack’s output. For each one, write the sentence that you would say aloud, and its kind:
| The pack says | You say | Kind |
|---|---|---|
Total 51.90 (scenario base, month 0) |
“About US$52 a month for both shops. It could be US$41 to US$76.” | calculated: measured times × list prices × assumed demand |
per tenant 25.95, month 12: 13.97 |
“About US$26 per shop now. With four shops next year, about US$14 each, because they share the fixed costs.” | calculated; the new shops are assumed |
usage x 2 58.82 +13% |
“If people ask twice as many questions, the bill rises about 13%, not 100%.” | calculated |
chat-strong only 85.8%, chat-small only 86.6% |
“The strong model costs 19 times more per question, and it was not better on our 67 questions.” | measured |
tokens/quota 114% (high, month 0) |
“If the assistant goes on the home page, the busiest minute needs more than the model provider allows us today.” | calculated from a measured quota and assumed demand |
Notice what is gone: the third decimal, the units of tokens, the name of the law. Notice what stays: the range, the kind, and the comparison that makes the number mean something.
Larkfield: the ten-minute walkthrough
Section titled “Larkfield: the ten-minute walkthrough”Kwame writes the talk as eight parts. Each part has one message, its minutes, and the picture or record that supports it.
Text description of the diagram
The talk has eight parts in ten minutes. First, the problem in one sentence, one minute. Second, what people will do, the five journeys, one minute. Third, the design on one page, the container diagram, two minutes. Fourth, what it costs: about US$52 a month, about US$26 per shop, two minutes. Fifth, what we tested: the strong model costs 19 times more per question and was not better, one minute. Sixth, the assumptions we did not verify, one minute. Seventh, the risks: one database, and the provider’s quota, one minute. Eighth, the roadmap and the one question to decide today, one minute.
The cost part needs one chart, and it is not the 14-driver table. Omar’s question is “per shop”, so the chart shows the cost per shop, in the three scenarios, now and with four shops:
- now, two shops
- month 12, four shops
Text description of the chart
A column chart with three pairs of columns, low, base and high, in US dollars per shop per month. In each pair, the first column is now, with two shops, and the second is month 12, with four shops. Low: 20.27, then 10.20. Base: 25.95, then 13.97. High: 37.90, then 26.79. In every scenario, the cost per shop in month 12 is lower than now.
Read the chart the way Omar will. The base case is about US$26 per shop now and about US$14 with four shops. Even the high case stays under US$40 per shop. The total for all shops stays under his budget of US$150 in every scenario, as Growth plan showed.
The assumptions part needs a different chart. Not every assumption matters equally. The sensitivity table shows how much each one moves the total:
Text description of the chart
A column chart with four columns, in percent change of the monthly total. Usage, twice the questions: plus 13. Length, answers twice as long: plus 6. Model, chat-strong for every answer: plus 235. Prices, all 20 percent higher: plus 20. The last column is about ten times taller than the others.
So the message for part 6 is short. “Our demand numbers are guesses, but a guess that is twice too low adds only 13% to the bill. The decision that could triple the bill is the model, and we measured it.” That is the “designed for a million users” fear, answered with one picture.
Here is the outline file that Kwame keeps in the pack, decisions/presentation-outline.md:
# Ten-minute walkthrough (for Omar, Grace and Ines)
1. **The problem in one sentence (1 min).** Several shops want the assistant; each must see only its own documents; Omar needs a price per shop.2. **What people will do (1 min).** The journeys J1 to J5 and what is out of scope.3. **The design on one page (2 min).** The container diagram: one application, one database, one model provider. Why not more services yet (ADR-0001).4. **What it costs (2 min).** Base scenario: about US$52 a month for two shops, about US$26 per shop; low to high: US$41 to US$76. Most of it is fixed (the API replica and the database), so the cost per shop falls as shops join: about US$14 per shop with four shops in month 12. If usage doubles, the total rises about 13%, not 100%.5. **What we tested (1 min).** The strong model costs 19 times more per question and was not better on our 67 questions (ADR-0003).6. **Assumptions (1 min).** The numbers we did not verify: customers per day, questions per customer, growth, the cache hit share. The high scenario hits the provider quota in the busiest minute from the first month: the first thing to measure.7. **Risks and failures (1 min).** The database is a single point of failure; the restore plan (ADR-0005). The provider quota (F1).8. **The roadmap and what we ask for (1 min).** Release 1 with two shops, the validation work, the signals for each later step. One unresolved assumption to decide today: will customers see the assistant on the home page?
Speak plainly: say "measured", "price" or "assumption" for every number you show.Look at part 8. “Will customers see the assistant on the home page?” is not a question that Kwame can answer with a measurement. It is Omar’s business choice. And it decides between the base case and the high case: the high case needs a higher provider quota before launch. So the walkthrough ends with the one decision that only the listeners can make.
Guided practice: the presentation in the pack
Section titled “Guided practice: the presentation in the pack”-
In your folder
design-pack, print the numbers for part 4. They are the course-end scenarios:Terminal window python -m costmodel scenariosScenarios (US$ per month)low base highmonth 0 40.54 51.90 75.81 (2 tenants)per tenant 20.27 25.95 37.90month 12 40.79 55.86 107.17 (4 tenants)per tenant 10.20 13.97 26.79 -
Print the numbers for part 6:
Terminal window python -m costmodel sensitivitySensitivity (base scenario, month 0)what if US$/month change per question per tenantbase 51.90 +0% 0.001181 25.95usage x 2 58.82 +13% 0.000669 29.41answers 2x longer 54.98 +6% 0.001251 27.49chat-strong for every answer 173.99 +235% 0.003959 86.99all prices +20% 62.28 +20% 0.001417 31.14Look at the column “per question”. When usage doubles, the all-in cost per question falls: from US$0.001181 to US$0.000669. That is why a cost per question means nothing without its volume.
-
Copy the outline from the snapshot
end-of-m06.Terminal window S=../nextia-learning-labs/C26/snapshots/end-of-m06cp "$S/decisions/presentation-outline.md" decisions/Terminal window $S = "..\nextia-learning-labs\C26\snapshots\end-of-m06"Copy-Item "$S\decisions\presentation-outline.md" decisions\ -
Check every number in the outline against the two outputs. Round each one in your head. 51.90 is “about 52” and 25.95 is “about 26”. 40.54 to 75.81 is “41 to 76”, and 13.97 is “about 14”.
-
Run the check:
Terminal window python -m costmodel checkPASS requirements: journeys, out of scope, measurable quality attributesPASS demand: every value has evidence (verified or assumed)PASS prices: every price has its sourcePASS measured: every value names its runPASS architecture: every component has a job, an owner and a place on the diagramPASS decisions: every ADR has status, context, options, decision, consequences, revisitPASS failures: at least 5, each with owner, mitigation and signalFAIL roadmap: every ADR appears in the roadmap: decisions/roadmap.md does not exist yetPASS presentation: assumptions, risks and an unresolved assumptionDesign pack check: 8 of 9 passedThe presentation check looks for three words in the outline: “Assumptions”, “Risks” and “unresolved”. It cannot tell if your talk is good. It only makes sure that the two honest parts and the ask are there.
Questions you will get
Section titled “Questions you will get”A walkthrough is half talk and half questions. Kwame prepares the three questions that Tomás asked during the course, because Omar will ask them too. Each answer is one number and one sentence:
| Question | Answer | From |
|---|---|---|
| “Shouldn’t we design for a million users?” | “At the busiest minute of the base case, 0.58 requests are in flight. The first limit is the provider’s quota, which a bigger design does not change.” | workload, growth plan |
| “Isn’t it about a cent per question?” | “Only the model’s part is per question: US$0.000157. All-in, it is about US$0.0012 today. It falls as usage grows, because 87% is fixed.” | units |
| “Don’t we need Kubernetes and a vector database service?” | “Not yet. One application and PostgreSQL serve this size; each record says the signal that would change it.” | ADR-0001, ADR-0002 |
And one question that only the team can answer honestly: “How sure are you?” Kwame’s answer: “The model times, quality and token counts are measured. The prices are list prices; prices change. The demand is our estimate, and we show it as a range. The restore time is an estimate until the first test.”
When to give a walkthrough like this, and what it costs
Section titled “When to give a walkthrough like this, and what it costs”Give one before any decision that needs money or people from outside the team: a launch, a new shop, a new model. Ten minutes and one outline file are cheap compared with a week of work on the wrong design.
Change it for each audience. A talk for developers can show Little’s law and the cost drivers. A talk for a shop owner needs her price, her privacy and her quality. The pack is the same; the questions are not.
What it costs: the talk hides detail on purpose, so a listener cannot check every number in the room. That is why each number in the outline points to the pack or a record. After the meeting, anyone can open decisions/ and check the reasons.
Common mistakes
Section titled “Common mistakes”Common mistake
No unresolved assumption
Symptom: you replaced the last sentence of part 8 with “Next steps: start building.” The check fails:
FAIL presentation: assumptions, risks and an unresolved assumption: presentation-outline.md does not mention 'unresolved'Design pack check: 7 of 9 passedCause: the talk now ends as if every input were known. The listeners leave without the one decision that changes the design, and the home page question returns after launch, as a quota problem.
Fix: end with the open question and the reason that it matters. For example: “One unresolved assumption to decide today: will customers see the assistant on the home page? If yes, we need a higher provider quota before launch.”
Common mistake
Precision that sounds like certainty
Symptom: a slide says “Monthly cost: US$51.90”. Omar puts US$51.90 in the budget. After the first month, the bill is US$58, and he asks what went wrong.
Cause: two decimals look like a measurement. The number is a calculation on assumed demand, and its own range is US$40.54 to US$75.81.
Fix: round it and give the range and the kind: “about US$52, from US$41 to US$76, calculated with assumed demand”. Then a bill of US$58 is inside the plan, and the usage log shows which assumption moved.
Your turn: three minutes for Ines
Section titled “Your turn: three minutes for Ines”Change one condition: the audience. Ines, the owner of Bramble Books, cannot come to the meeting. She gives you three minutes on a call. She asks three things: “What will it cost me? Can Larkfield or another shop see my supplier terms? Will it answer my staff correctly?”
Write a three-part outline for her, with one number per part and its kind. Use the pack. python -m costmodel units gives Bramble Books’ share of the cost. End with one thing that is still unknown for her shop.
Show a worked solution
python -m costmodel units prints, in the base case, month 0:
Unit economics, scenario base, month 0 total US$ 51.90 per month (fixed 87%, variable 13%) per question (variable only) US$ 0.000157 per question (everything) US$ 0.001181 per document change US$ 0.000169 (embed + summary + worker) Larkfield US$ 29.10 per month Bramble Books US$ 22.80 per month- Your cost (1 min). “About US$23 a month for Bramble Books today. That is half of the fixed costs plus your own questions. The calculation uses measured times, list prices and our estimate of your 65 questions a day. When two more shops join, your share falls, because the fixed costs are shared by four.”
- Your privacy (1 min). “Another shop cannot see your documents. Every row in the database carries your shop, and the database shows each request only the rows of its own shop. The authentication course measured this check at about 0.1 ms per request. An access test runs on every change, and a release stops if it fails.”
- Your quality (1 min). “On Larkfield’s 67 questions, 86.6% of the answers were correct, measured. We have not measured your questions yet. Priya will build an evaluation set for your supplier terms during the first release. Until then, we do not know how well it answers your staff. That is the unresolved part for your shop.”
The privacy answer uses no price and no model. It answers her real worry with the mechanism and a test. And the last part says plainly what is unknown, with the work that will measure it.
Knowledge check
Section titled “Knowledge check”Check your understanding
Why should the walkthrough say 'about US$52 a month, from US$41 to US$76' instead of 'US$51.90'? Give two reasons.
Show the answer
First, US$51.90 is a calculation on assumed demand, not a measurement. Two decimals make it sound exact, and listeners will plan with it. Second, the range tells the listener how much the answer can move: from the low to the high scenario. A later bill inside the range is then expected, not a failure. The rounded number is also easier to remember and to repeat correctly.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Predict: in month 12 of the base case, with four shops, does the all-in cost per question go up or down from US$0.001181? Explain with the pack's numbers.
Show the answer
It goes down, to about US$0.000808. The total rises only from US$51.90 to US$55.86, because 87% of the cost is fixed. The questions rise from 1,465 to 2,304 a day. The fixed costs are now shared by more questions, so each question carries less of them.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Omar says: 'Skip the assumptions slide. It makes us look unsure.' What do you answer, in three or four sentences?
Show the answer
Without the assumptions, the listeners take every number as certain, and the first bill that differs looks like a mistake. With them, they know which inputs to watch, and a bill inside the range is expected. The sensitivity chart also shows that the demand guesses matter little: twice the usage adds only 13%. And the one assumption that matters, the home page, is a decision that only Omar can make. Leaving it out would hide his own choice from him.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- A stakeholder decides about the system, pays for it or uses it. A walkthrough answers their questions in about ten minutes, with a few pictures; the pack stays the source of the numbers.
- Four rules: start from their questions; round every number, give its range and its kind; show what you do not know; end with an ask.
- A variable cost per question alone misleads: US$0.000157 against about US$26 per shop, because 87% of the cost is fixed.
- Larkfield’s walkthrough has eight parts. It shows the cost per shop: about US$26 now, about US$14 with four shops. It also shows the sensitivity chart: usage ×2 adds 13%, chat-strong 235%.
- It ends with an unresolved assumption: will customers see the assistant on the home page?
python -m costmodel checknow prints8 of 9 passed.
Next: Implementation roadmap