Set up the design pack

What you will be able to do

Get the course’s project, the design pack, onto your computer and run its cost calculator for the first time. Then learn the rule that the whole course follows: every number is measured, a price or an assumption, and it says which.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• The document assistant that answers questions from a shop’s documents, with citations, from RAG: Building AI That Uses Your Data. Its 36 documents and Grace’s 67 test questions come back in this course.
• Tenants, and why one shop must never see another shop’s data, from Authentication, Permissions and Multi-Tenant Applications.
• The provider’s quota, latency percentiles and a workload model, from Scaling APIs and AI Workloads.
• What a database stores and how a backup is restored, from Databases and Storage for AI Applications.
• A terminal, a virtual environment and pip, from Python for Practical AI Engineering.

What you need on your computer

• Python 3.12 or newer, VS Code (or any text editor) and a terminal.
• About 40 MB of free disk space for the virtual environment.
• No account, no key and no money. The calculator makes no network calls.

Tested with Python 3.12 on macOS (Apple silicon). The Windows (PowerShell) and Linux commands are written, not tested.

A quote for a new kitchen floor

Think of two builders who give you a quote for a new kitchen floor. The first builder says: “About 5,000 dollars.” The second builder gives you one page:

• “The floor is 4.2 m by 3.5 m: I measured it. That is 14.7 square metres.”
• “The tiles cost 38 dollars per square metre in the shop’s catalogue.”
• “I assume two days of work, three if the old floor is damaged.”

The first quote is shorter. But you cannot check it, and you cannot change it. If you choose cheaper tiles, what is the new price? If the old floor is damaged, how much more? Only the builder knows.

The second quote answers those questions for you. Each number says where it comes from. Some numbers are measured (the room). Some are a price that somebody published (the tiles). Some are an assumption (two days of work), and the builder says so and gives a range. You can change one number and do the sum again yourself.

The problem: a number nobody can check

Larkfield’s help desk has a document assistant. It answers questions from Larkfield’s own documents, with citations. Bramble Books, a small book shop on Larkfield’s help-desk platform, wants the same assistant. Omar, the product manager, hopes that a third and a fourth shop join next year. So Omar asks one question: “What will it cost per shop?”

Tomás, an AI engineer, answers at once: “About a cent per question.” It sounds useful, so test it. Omar does not know how many questions there will be. If the shops ask 400 questions a day, a cent each is 120 dollars a month. If they ask 5,000 a day, it is 1,500 dollars a month. A cent from where? Is it the model alone, or the database and the servers too? Does it change if usage doubles?

Tomás’s number has the same problem as the first builder’s quote. It has no volume (how many questions), no source (where the cent comes from) and no assumptions that someone could check. Nobody can say if it is right. Nobody can change one input and see the new result. And when the bill arrives, nobody can say which part was wrong.

Kwame, the solution architect, owns the proposal for the shared assistant. He wants a proposal like the second builder’s page. That is what this course builds, one module at a time.

The idea: a design pack with three kinds of numbers

A design pack is a folder that holds a system design and everything you need to check it. Kwame’s pack holds the requirements, the diagrams, the decisions and a small cost calculator. The calculator reads its numbers from text files. It does not hide any number in its code.

Every number in the pack is one of three kinds:

• Measured. A run of committed code produced it, and the pack names that run. For example: the small chat model answered in 1.52 seconds (median) over 67 questions, two passes, one question at a time. Anybody with the code and the data can run it again.
• Price. A published list price, and the pack names the price page. For example: the small chat model costs US$0.10 per million input tokens. Prices change, so you check the current page before a real decision.
• Assumption. A value that you chose, because nobody can measure it yet. The pack writes it as a range of three values (low, base and high) and gives the reason. For example: 800 Larkfield customers ask a question each day, between 300 and 1,600.

The diagram shows how you decide which kind a number is.

Two questions sort every number into one of three kinds. Each kind has its own duty: name the run, name the page, or give a range and a reason.

Text description of the diagram

The flow starts with one number from the proposal. The first question is: did a run of committed code produce it? If yes, the number is measured, and you name the run: what was measured, how many times, and on which machine. If no, the second question is: does a published price page list it? If yes, the number is a price, and you name the page and check it again before a decision, because prices change. If no, the number is an assumption. You give it a range of three values, low, base and high, and you write the reason.

Why does the kind matter? Because each kind fails in a different way. A measured number can come from a different machine or a different day: times change, but the shape of the result lasts. A price can be out of date. An assumption can be wrong, and nothing in the pack shows it until real use begins. So the pack keeps the three kinds apart, and the proposal says which assumptions to check first.

An assumption is not a mistake. A hidden assumption is. “A cent per question” hides at least three assumptions. Kwame’s pack makes them visible.

A tiny example: the model cost of one answer

Here is one sum from the pack, with every value visible. When the assistant answers a question, it sends the question and five passages from the shop’s documents to the small chat model. The model reads input tokens and writes output tokens. A token is a piece of a word; the provider charges per million tokens.

Value	Number	Kind	Where it comes from
Input tokens per answer	946.5	measured	the course team’s run: 67 questions, 2 passes, mean
Output tokens per answer	152.75	measured	the same run
Price of input tokens	US$0.10 per 1M	price	the provider’s price page
Price of output tokens	US$0.50 per 1M	price	the same page

The sum, one step at a time:

• Input: 946.5 tokens × US$0.10 ÷ 1,000,000 = US$0.0000947.
• Output: 152.75 tokens × US$0.50 ÷ 1,000,000 = US$0.0000764.
• One answer: US$0.0000947 + US$0.0000764 = US$0.000171.

That is less than two hundredths of a cent for the model’s part of one answer. Now add an assumption. Suppose the shops ask 1,000 questions a day. That is a made-up number for this example only. Then the model costs 1,000 × US$0.000171 = US$0.171 a day, or about US$5.13 in a month of 30 days.

Look at what changed. The first three steps used only measured numbers and prices. Only the last step needed an assumption, and you can see it. If Omar says “more like 2,000 a day”, you change one number. You also see what this sum leaves out: the database, the servers, the logs. Tomás’s “a cent” could not tell you any of this. Module 3 adds the other parts, one cost driver at a time.

Larkfield’s design pack

Kwame does not work alone. Omar brings the business goals and the shops’ sizes. Amira, the developer, knows what the code does today and what a change costs. Mei, the data engineer, owns storage, backups and restores. Priya, the data scientist, owns the quality evaluation. Grace leads Larkfield’s support team, and Ines owns Bramble Books. Grace and Ines say what “good” and “fast enough” mean for the people who use the assistant.

The proposal is built one module at a time, and each module adds to the pack:

Module	What it adds to the pack
Start with requirements	requirements.md (users, journeys, quality targets), demand.toml (demand as ranges)
Design the end-to-end architecture	architecture/: the container diagram, the request flow, the ingestion sequence
Estimate resources and cost	design.toml, the workload arithmetic, the cost by driver, the cost per shop
Optimize with measured trade-offs	model routing, caching and batching, build versus buy
Design for failure and growth	the failure modes, recovery targets and a growth plan
Communicate the proposal	decision records, a roadmap and a ten-minute presentation

The design pack. The calculator has no numbers of its own: it reads them from three kinds of input file, and the tests check it.

Text description of the diagram

On the left are the input files, one colour per kind of number. The file prices.toml holds 35 list prices, and each one names its price page. measured.toml holds 39 measured values, and each one names the run it comes from. demand.toml, which you fill in during this module, and design.toml, from Module 3, hold the assumptions: each value is a range with a reason. All three files feed the calculator, python -m costmodel, which uses only the Python standard library. The tests check the calculator. The calculator prints tables in the terminal and can write costs.csv for a spreadsheet. Those results go into Kwame’s proposal, next to the requirements, the diagrams and the decisions.

The prices and the measured values are given: the course team collected them. The team made 659 real calls to a hosted model provider and read the provider’s public price list. You do not repeat those runs, and you need no account. Your work in this course is the design: the requirements, the assumptions, the diagrams and the decisions.

Set up the pack

You copy the snapshot start of the pack into a new folder, make a virtual environment, and run the tests. A snapshot is a copy of the pack at one point of the course. If your pack breaks later, you copy the snapshot of your last finished module and continue.

1. Open a terminal and go to your projects folder, for example ~/projects. If you have Git, clone the labs repository once. If you have the clone already, run git -C nextia-learning-labs pull instead.

   cd ~/projects
   git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.git

   Without Git, download the ZIP file of the repository, and unzip it in ~/projects. Its folder is then nextia-learning-labs-main. Use that name in the next steps.

2. Copy the snapshot start into a new folder design-pack, and go into it:

   macOS / Linux

   cp -R nextia-learning-labs/C26/snapshots/start design-pack
   cd design-pack

   Windows (PowerShell)

   Copy-Item -Recurse nextia-learning-labs\C26\snapshots\start design-pack
   cd design-pack

3. Open the folder design-pack in VS Code. All the commands in this course run in this folder.

4. Make the virtual environment, activate it, and install the two tools for the tests: pytest and ruff.

   macOS / Linux

   python3 -m venv .venv
   source .venv/bin/activate
   python -m pip install -r requirements-dev.txt

   Windows (PowerShell)

   python -m venv .venv
   .venv\Scripts\Activate.ps1
   python -m pip install -r requirements-dev.txt

   The prompt now starts with (.venv). The calculator itself needs nothing from pip: it uses only the standard library.

5. Run the tests:

   python -m pytest -q

   ......                                                                   [100%]
   6 passed in 0.01s

   Six tests check that every price names a page, every measured value names a run, and bad input is refused. Your time can differ.

6. Run the calculator without a command. It tells you what it can calculate now:

   python -m costmodel

   Prices:           35 (each with its source page)
   Measured values:  39 (each with the run it comes from)
   Demand values:     0 (fill in demand.toml in Module 1)
   
   Next: describe the users and fill in demand.toml (Module 1).

You now have a working pack. It has 35 prices and 39 measured values, and no assumptions about demand. The calculator cannot say anything about cost yet, because nobody has said who uses the assistant, or how much. That is the job of this module.

Try a command from a later lesson, for example python -m costmodel demand. The calculator does not crash. It says when the command arrives:

This command arrives in Module 1. Download the end-of-module-1 snapshot or keep going.

What is in the pack

File or folder	What it is
prices.toml	35 list prices. Each has a value, a unit and a source that starts with price: or price page:.
measured.toml	39 measured values. Each source starts with measured: and names the run.
demand.toml	A template. You fill it in during Demand assumptions.
requirements.md	A template. You fill it in during the next two lessons.
costmodel/	The calculator. inputs.py reads the files and refuses a number that does not say what it is.
tests/	The calculator’s tests.
README.md	Every command, and the reset path.

Read the numbers

Open prices.toml in VS Code. Each price is a small table. Here is the first one:

[chat_small_input]
value = 0.1
unit = "US$ per 1M tokens"
source = "price: https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/ (Azure Retail Prices API (prices.azure.com/api/retail/prices): 6-luna ShortCo Inp Std Gl 1M Tokens)"
note = "chat-small (gpt-6-luna), Global Standard, input tokens"

The source names the public price page and the line of the provider’s price list. The course uses the Azure Retail Prices API, a public list that needs no account. “chat-small” is the course’s name for the small chat model; “chat-strong” is the strong one. Prices change: check the current page before you use a price in a real proposal. The comment lines at the top of the file say when the price list was read. Do the same in your own proposals: write the date you checked the prices.

Some prices are not an amount per hour or per gigabyte. They are a rule on the price page, such as “the first 5 GB of logs each month are free”. Their source starts with price page:, because the page states them in words.

Now open measured.toml. Here is one value:

[answer_small_p50_s]
value = 1.5165
unit = "seconds (median)"
source = "measured: the course team's run, the RAG course's 67 questions with its best search results, 2 passes, one question at a time, chat-small"

The source says what was measured, on which questions, how many times and how. That is enough to judge the number. These times come from one afternoon, one call at a time, from the course team’s computer. Your network gives other times, but the shape of the result lasts: the strong model is slower than the small one.

Some measured values come from the earlier courses, and the source says so. The reranker’s time comes from the RAG course; the provider’s quota comes from the scaling course. The pack does not measure again what an earlier course measured.

Guided practice: sort six numbers

Sort each number into measured, price or assumption. Use the two questions of the diagram. Then open the answer.

1. “PostgreSQL B1ms server: US$0.0185 per hour.”
2. “The question embedding took 0.34 s (median) over 67 questions.”
3. “Bramble Books will have 25 documents next year.”
4. “The provider allows 100 chat requests per minute on the course team’s account.”
5. “The first 100 GB of data sent out to the internet each month are free.”
6. “A customer usually asks one question and one follow-up.”

Show the answer

1. Price. pg_b1ms_hour in prices.toml, from the PostgreSQL price page.
2. Measured. question_embed_p50_s in measured.toml: a run of 67 calls.
3. Assumption. Ines plans it, but nobody can measure next year. It becomes a range in demand.toml.
4. Measured. provider_quota_requests_per_minute: the scaling course read it from the provider’s rate-limit headers. It is true for that account; another account can have another quota.
5. Price. A rule on the price page (egress_free_gb_month, source price page:).
6. Assumption. There is no data yet. In demand.toml it becomes customer_questions_per_day = [1.0, 1.5, 3.0].

Number 4 is the hard one. It looks like a fact about the provider, but it is a measurement of one account. If your account has another quota, you measure it again.

When to use a design pack

Use one whenever a decision depends on numbers that other people must trust: a cost per customer, a server size, a choice between two services. Write every number with its kind and its source, and keep the calculation in a file, not in your head. Then anybody can check it, change one assumption, and get the new result in a second.

Do not build a calculator for a decision that one sum on paper answers. If the whole question is “is this US$5 or US$500 a month?”, one line with three numbers is enough. The pack pays off when there are many parts, several people and several scenarios.

It has a cost. Every number needs a source, and somebody must check the prices and the assumptions again before the decision. Kwame accepts that cost, because Omar will put the result in a price for the shops.

Common mistakes

Common mistake

A price without a source

Symptom: you add or edit a price and forget its source line. The calculator stops at once:

Input problem: prices.toml: price 'chat_small_output' has no source

The tests fail too: 1 failed, 5 passed, in test_the_pack_prices_all_have_a_source.

Cause: the calculator refuses a number that does not say what it is. This is on purpose. A price with no page is a guess that looks like a fact.

Fix: add the source line, starting with price: and the page. If you cannot find a page, the number is not a price. It is an assumption, and it belongs in demand.toml or design.toml, with a range.

Common mistake

No module named pytest, or No module named costmodel

Symptom: a command prints one of these lines and stops. The paths are shortened here; yours show your own folders.

…/bin/python3.12: No module named pytest

…/design-pack/.venv/bin/python: No module named costmodel

Cause: “No module named pytest” means that the virtual environment is not active, so Python does not find the tools that you installed. “No module named costmodel” means that the terminal is not in the folder design-pack.

Fix: go to the folder with cd ~/projects/design-pack. Activate the environment: source .venv/bin/activate on macOS and Linux, .venv\Scripts\Activate.ps1 on Windows. The prompt starts with (.venv) again.

Reset. If your pack breaks, rename the folder, copy the snapshot start again and make the environment again, as in the steps above. The snapshots README shows the same steps for every later snapshot.

Your turn: a price that is text

Change one thing: write a price as text. In prices.toml, change the first price’s line value = 0.1 to value = "0.10 US$". Before you run anything, predict: does the calculator accept it, and if not, what does it say? Then run python -m costmodel, and change the line back.

Show the answer

The calculator refuses it:

Input problem: prices.toml: price 'chat_small_input' must be a number

A price is a number, with its unit in a line of its own (unit = "US$ per 1M tokens"). Text such as “0.10 US$” mixes the two, and a sum cannot use it. The calculator checks this when it reads the file, before any sum, so a wrong input never becomes a wrong cost. Change the line back to value = 0.1, and python -m pytest -q prints 6 passed again.

Knowledge check

Check your understanding

Tomás says the assistant costs 'about a cent per question'. Name three things that are missing from his number, and say why each one matters to Omar.

Show the answer

• Volume. The number of questions a month. Without it, Omar cannot get a monthly cost. A cent each is 120 dollars a month at 400 questions a day, and 1,500 at 5,000 a day.
• A source. Where the cent comes from: a measured run, a price page, or a guess. Without it, nobody can check it or update it when prices change.
• The assumptions and what is included. Is it the model alone, or the servers and the database too? What happens if usage doubles? Without stated assumptions, Omar cannot change one input and see the new cost, and nobody knows which part was wrong when the bill arrives.

Check your understanding

Predict: the provider halves the price of chat-small's output tokens, to US$0.25 per million. Use the tiny example (946.5 tokens in at US$0.10, 152.75 tokens out) to compute the new model cost of one answer. Which kind of number did you change?

Show the answer

Input: 946.5 × 0.10 ÷ 1,000,000 = US$0.0000947, unchanged. Output: 152.75 × 0.25 ÷ 1,000,000 = US$0.0000382, half of US$0.0000764. One answer: US$0.0000947 + US$0.0000382 = about US$0.000133, instead of US$0.000171. That is about 22% less, not 50%, because input tokens cost the same. You changed a price. The measured token counts stay the same, and so do the assumptions about volume.

Check your understanding

Practical check: show that your pack works. What do the tests and the calculator print, and what does each `source` in the two input files start with?

Show the answer

The tests print 6 passed. The calculator prints Prices: 35, Measured values: 39, Demand values: 0 (fill in demand.toml in Module 1) and the next step. In prices.toml, every source starts with price: or price page:. The first names a page and a line of the price list; the second names a rule stated in words on the page. In measured.toml, every source starts with measured: and names the run.

Case studies in this course

At the end of the course, you use what you learn on a realistic project, from the data to the recommendation. Complete at least one for the certificate; the others are optional.

• Cost and capacity plan from real demand dataTurn a public demand series into peaks, growth and a cost range for a public question-answering assistant, with the course's calculator.
• Design review of an open document-question serviceSize storage and ingestion for a real open document collection, choose routing and caching with measured numbers, and write the decision records and the failure table.

Recap

• A number without a volume, a source or its assumptions cannot be checked or changed. “About a cent per question” is such a number.
• Every number in the design pack is one of three kinds. Measured: a run of committed code, named. Price: a published list price, with its page; prices change. Assumption: a range of low, base and high, with the reason.
• The model cost of one answer is 946.5 tokens × US$0.10 per million + 152.75 tokens × US$0.50 per million = US$0.000171. Only the volume needs an assumption.
• Your pack works: python -m pytest -q prints 6 passed, and python -m costmodel counts 35 prices, 39 measured values and 0 demand values.
• The calculator refuses a price without a source, or a price that is text.

Next, before any number about demand, you describe who uses the assistant and what each person needs from it.

Next: Users and workflows