Implementation roadmap

What you will be able to do

Turn the decisions into an order of work. First a small release that people can use, then the work that replaces assumptions with measurements, then later steps that each wait for a signal. Then deliver the whole pack and its walkthrough.

Before you startWhat to know and to have readyAbout 55 minutes

What you need to know

• The decision log, from Architecture decisions: The decision log, and the unresolved assumption, from Stakeholder presentation.
• Signals, lines and steps, from Growth plan.

What you need on your computer

• Your folder design-pack from the last lesson, with python -m costmodel check printing 8 of 9 passed.

Tested with Python 3.12 on macOS (Apple silicon). The Windows (PowerShell) and Linux commands are written, not tested.

Moving into a new flat

You move into a new flat. On the first night, you need a bed, water, light and a lock on the door. You do not need the bookshelf or the second sofa yet. So the first day’s work is small, but the flat works: you can sleep there.

Then you live in it for two weeks and learn things that no plan told you. The kitchen light is too dim. The sofa that you wanted would block the window. The neighbour’s noise is fine. Each of these facts changes what you buy next.

Later purchases wait for a reason. “Buy a lamp when cooking in the dark becomes a problem.” You did not buy the lamp on day one, and you did not buy the sofa at all.

The problem: a roadmap by layers and dates

Tomás writes a first roadmap:

• Months 1 and 2: the platform. Kubernetes, a vector database service, a separate cache.
• Months 3 and 4: the search and the ingestion services.
• Months 5 and 6: the answer service and the help-desk integration.
• Month 7: the first shops use the assistant.

In that plan, nobody asks a question for six months. The first real measurement of demand, latency and quality arrives in month 7. Suppose Omar then puts the assistant on the home page. In the high case, the busiest minute needs 114% of the provider’s token quota. That problem arrives on launch day, after six months of work on parts that the base case does not need.

And the dates do their own damage. “Month 2: Kubernetes” happens because the date arrives, not because a signal fired. At the busiest minute of the base case, 0.58 requests are in flight. Each extra always-on part costs about US$35 a month in the rough model of Simplicity and evolution, for capacity that nobody uses.

The idea: an incremental roadmap

An implementation roadmap is the order in which the team builds the design. An incremental roadmap delivers it in small steps that people can use, and it learns from each step. It has three kinds of items:

1. A first usable release. The smallest version that real people use for real work. Here, it is one question from one shop, answered with a citation, through every part of the design. It is a thin slice through all the parts, not one finished layer. It has a done-when line: measurable targets that say it works.
2. Validation work. Tasks that replace an assumption with a measurement. Order them by how much the answer could change a decision.
3. Later capabilities. Each one waits for a signal and names the decision record that it would change. No dates.

Why a slice and not a layer? A slice reaches users in weeks, so the first measurements arrive early. A finished database layer with no API teaches nothing about demand. And the slice is cheap here, because the design is one application with one database.

Two ways to order the same work. Layer by layer, the first answer and the first measurement come last. A thin slice reaches users first and lets the measurements decide the next steps.

Text description of the diagram

On the top row, the work goes one layer at a time. The database comes in month 1, the search in month 2 and the API in month 3. The first answer to a person comes only in month 4. On the bottom row, release 1 answers one shop’s question with a citation through every part. Then the validation work measures the targets Q1 to Q8 and replaces assumptions. Release 2 adds shops with the same parts. Later steps happen only when a signal fires.

A tiny example: which assumption to check first

Validation work has an order too. Rank each assumption by how much it moves the cost or a decision. Every value below comes from the pack’s outputs in earlier lessons:

Assumption	Range in the pack	Effect on the monthly total	Does it change a decision?
Customers see the assistant on the home page	base or high scenario	51.90 → 75.81: +46%	yes: the high case needs a higher provider quota before launch
Restore time (people and a new server)	about 70 minutes, assumed	none	maybe: ADR-0005 holds only if the real RTO is under 240 minutes
Questions per person	usage ×2	51.90 → 58.82: +13%	no: still far under the budget and the quota
Cache hit share	0% to 30%	52.67 to 50.37: about ±3%	no: the cache is cheap either way

The order follows from the last column. The home page comes first: it moves the bill most, and it changes what the team must do before launch. The restore test comes second: it can break a decision record. Usage and the cache share come from the usage log, which runs anyway. No extra work is needed for them, only a look after a few weeks.

Notice that the cheapest assumption to check is the most important one. The home page is a single answer from Omar. That is why the walkthrough ends with it.

Larkfield: the roadmap

Kwame writes the roadmap as a file in the pack, decisions/roadmap.md. Every step names the decisions that it builds on or would change:

# Implementation roadmap

Each step starts when its signal says so, not on a date. Each step names the decisions it builds on.

## Release 1: the first usable assistant (Larkfield and Bramble Books)

- One modular application with the API and the ingestion worker (ADR-0001).
- PostgreSQL for records, vectors, the queue and the answer cache, with row-level security (ADR-0002).
- chat-small answers every question (ADR-0003).
- Managed PostgreSQL (ADR-0004) with point-in-time restore and the first restore test (ADR-0005).
- Done when: Q1 to Q8 are measured for two weeks and each one meets its target.

## Validation work (during release 1)

- Replace the assumed demand with the usage log: questions per user, customers per day, repeats (the cache hit share).
- Build an evaluation set for Bramble Books (Priya) and run it before every model or prompt change.
- Run the first restore test and write the real RTO into ADR-0005.
- Decide with Omar whether customers see the assistant on the home page (the high scenario). If yes, ask the provider for a higher quota before launch: the high scenario needs more than the quota in its busiest minute (ADR-0003).

## Release 2: the third and fourth shops

- Add a tenant with the tenant checklist (J5), no code change.
- Start when: a third shop signs; the cost model says the cost per tenant falls (fixed costs are shared).

## Later, only when a signal fires

The first three signals are printed by `python -m costmodel growth`; the others come from the logs, the evaluation set and the team list.

| Signal | Step | Decision to revisit |
|---|---|---|
| the busiest minute uses 70% of the provider quota | ask for a higher quota; queue answers for customers | ADR-0003 |
| the database uses 80% of its storage | a bigger storage size (a setting) | ADR-0004 |
| the monthly cost passes the budget (US$150) | find the biggest driver in the cost table first (today: the API replica) | the decision behind that driver |
| search p95 > 100 ms or 5 million chunks | tune the index; then consider a separate vector store | ADR-0002 |
| a new question set shows chat-strong 5 points better | switch routing on | ADR-0003 |
| more than 4 API replicas at the busiest minute, or ingestion needs a GPU, or a second team | split that part into its own service | ADR-0001 |

Three things make this a roadmap and not a wish list. Release 1 has a done-when line with the targets of Module 1. The validation work includes the home page and the restore test, the top of the ranking. And every later step has a signal and a record.

The picture below is the module’s main visual: the decision log connected to the roadmap.

The decision log connected to the roadmap. Release 1 builds all five records. The validation work and the later signals each point to the record that they could change.

Text description of the diagram

The roadmap has four stages. Release 1 serves Larkfield and Bramble Books. Then come the validation work, release 2 with the third and fourth shops, and later steps on signals. Release 1 builds all five decision records. The validation work can change two of them. The first restore test gives ADR-0005 a real RTO. Bramble Books’ evaluation set and the home page decision test ADR-0003. The later signals point back to their records. More than 4 replicas points to ADR-0001, a slow search to ADR-0002, and storage at 80% to ADR-0004.

Why does each later step wait? Because none of them is free. The chart shows the base case’s monthly total, today and with each later step added alone:

The base case's monthly total in US$, today and with one later step added: routing switched on (measured cost per question), one more always-on service (the pack's rough model), and high availability (HA) for the database (list prices). Omar's budget is US$150 a month.

Text description of the chart

A column chart in US dollars per month. Today: 51.90. With routing switched on: 77.89. With one more always-on service: 86.89. With high availability for the database: 326.90. The budget: 150. High availability alone would take the total to more than twice the budget.

Each step can become right later, when its signal fires. On day one, each one is a cost without a reason: routing alone adds 50% for no measured gain.

Guided practice: finish the pack

1. In your folder design-pack, copy the roadmap and the last test file from the snapshot end-of-m06.

   macOS / Linux

   S=../nextia-learning-labs/C26/snapshots/end-of-m06
   cp "$S/decisions/roadmap.md" decisions/
   cp "$S/tests/test_packcheck.py" tests/

   Windows (PowerShell)

   $S = "..\nextia-learning-labs\C26\snapshots\end-of-m06"
   Copy-Item "$S\decisions\roadmap.md" decisions\
   Copy-Item "$S\tests\test_packcheck.py" tests\

2. Run the whole check:

   python -m costmodel check

     PASS  requirements: journeys, out of scope, measurable quality attributes
     PASS  demand: every value has evidence (verified or assumed)
     PASS  prices: every price has its source
     PASS  measured: every value names its run
     PASS  architecture: every component has a job, an owner and a place on the diagram
     PASS  decisions: every ADR has status, context, options, decision, consequences, revisit
     PASS  failures: at least 5, each with owner, mitigation and signal
     PASS  roadmap: every ADR appears in the roadmap
     PASS  presentation: assumptions, risks and an unresolved assumption
   Design pack check: 9 of 9 passed

3. Run the tests and the style check:

   python -m pytest -q
   ruff check .

   .................................................                        [100%]
   49 passed in 0.11s
   All checks passed!

   The three new tests check that the pack passes, that a record without ## Revisit when fails, and that a failure row without an owner fails.

4. Your pack is now the snapshot end-of-m06, the course-end design pack. On macOS or Linux, you can confirm it. No output means no difference:

   diff -r -x .venv -x __pycache__ -x .pytest_cache -x .ruff_cache -x costs.csv -x explorer.json . "$S"

   The command skips costs.csv and explorer.json, which the export command of Module 3 wrote, and which the snapshot leaves out. If you kept your own ADR-0001, diff shows only the lines where your words differ.

When to plan this way, and what it costs

Plan incrementally when the demand, the quality or the costs are partly assumed. Before launch, that is almost always true. The first release produces the measurements that the later decisions need.

A date still has a place. Omar needs to tell shops when they can start. Give release 1 a target date, because it is known work. Do not put dates on later steps; give them signals.

What it costs: the first release looks small, and someone will call it unfinished. Make the done-when line visible: the targets Q1 to Q8, measured for two weeks. And the signals need someone to read them every month. The growth command prints three of them in a second.

Common mistakes

Common mistake

The roadmap forgets a decision

Symptom: you shortened the release 1 bullet about the database and removed “(ADR-0005)” from it and from the validation work. The check fails:

  FAIL  roadmap: every ADR appears in the roadmap: roadmap.md does not link ADR-0005
Design pack check: 8 of 9 passed

Cause: a decision record that no roadmap step builds or revisits is a decision that nobody will build or test. Here, the restore test lost its place in the plan.

Fix: put the link back where the work happens: “Managed PostgreSQL (ADR-0004) with point-in-time restore and the first restore test (ADR-0005)”.

Common mistake

Later steps with dates

Symptom: the roadmap says “Month 6: switch routing on”. In month 6, Amira switches it on. The bill rises from US$51.90 to US$77.89, and Priya’s evaluation set shows no change in quality.

Cause: the date replaced the signal. Nothing had measured a reason for the step.

Fix: replace the date with the signal from the decision record: “when a new question set shows chat-strong at least 5 points better”. Then check the signal every month.

Module practice: deliver the pack and the walkthrough

The outline’s task for this module is to deliver the architecture pack and a ten-minute walkthrough outline. Your pack is complete and the check passes. Now you practise the delivery with a change.

A worked example, small: trace one roadmap row. Take the first later step. Follow it from the signal to the record, and read today’s value:

1. The signal: the busiest minute uses 70% of the provider quota.
2. Where to read it today: python -m costmodel growth, the column “tokens/quota”.
3. Today’s value: 16% in the base case, month 0. The high case: 114%.
4. The record it revisits: ADR-0003, the model and its provider deployment.
5. The step: ask for a higher quota; queue customers’ questions at the busiest minutes.

In the base case, the row waits. In the high case, it has already fired before the first release. One answer from Omar decides which of the two is true.

Your task. Change one condition. Omar answers the unresolved assumption: “Yes, customers will see the assistant on the home page.” Kwame now plans with the high scenario. Update the pack for this answer:

1. Change decisions/roadmap.md: what moves into release 1, and what is the new done-when?
2. Change parts 4, 6 and 8 of decisions/presentation-outline.md: the new cost, the new first assumption to measure, and a new unresolved assumption.
3. Run python -m costmodel check. It must still print 9 of 9 passed.
4. Write one paragraph to defend one trade-off of the design under the new condition, and name the assumption that is still unresolved.

Use the pack’s outputs: python -m costmodel scenarios, units --scenario high and growth --scenario high. Keep a copy of the two files first, so that you can go back to the snapshot.

Show a worked solution

The outputs that the solution uses:

Unit economics, scenario high, month 0
  total                        US$      75.81 per month (fixed 72%, variable 28%)
  per question (variable only) US$   0.000126
  per question (everything)    US$   0.000453
  per document change          US$   0.000220 (embed + summary + worker)
  Larkfield                    US$      47.32 per month
  Bramble Books                US$      28.49 per month

python -m costmodel growth --scenario high starts with 0 2 5,580 104% 114% 0% 75.81 provider quota and ends with First signal: provider quota in month 0. The month 12 row is 12 4 13,874 259% 285% 6% 107.17 provider quota.

1. The roadmap. The quota request moves from “later” into release 1, before launch:

## Release 1: the first usable assistant (Larkfield and Bramble Books)

- ... (the five decisions as before)
- Before launch: a higher provider quota for chat-small (ADR-0003). Month 0 of the high
  scenario needs 104% of the calls and 114% of the tokens in the busiest minute. To stay
  under the 70% line in month 12, ask for at least 370 requests and about 407,000 tokens
  a minute.
- Until the quota is raised: customers' questions wait in a queue at the busiest minutes;
  agents go first.
- Done when: Q1 to Q8 are measured for two weeks and each one meets its target, and the
  share of 429 answers from the provider stays under 1% in the busiest hour.

In the validation work, the home page item changes. It becomes: “Measure the real customer questions a day from the home page; replace the high scenario’s assumption.”

2. The walkthrough. Part 4: “About US$76 a month for two shops, about US$38 per shop now. With four shops next year, about US$27 per shop; the total is about US$107, still under the budget of US$150. Calculated with measured times, list prices and the high scenario’s demand.” Part 6: “The first thing to measure is the provider quota: our busiest minute needs more than the provider allows us today.” Part 8: “One unresolved assumption: how long the provider takes to raise the quota. We have asked; until it answers, customers may wait at the busiest minutes.”

3. The check. Both files keep every record link and the words “Assumptions”, “Risks” and “unresolved”, so the check still prints Design pack check: 9 of 9 passed.

4. Defend one trade-off. “We keep one modular application and one PostgreSQL server, even in the high case. The busiest minute’s limit is the provider’s quota, not our servers. More services or more replicas share the same quota, and each adds about US$35 a month. Storage reaches only 6% of the minimum size in month 12. The total stays at US$107, against a budget of US$150. The trade-off is a single database server. We accept it, with a tested point-in-time restore, because high availability would cost about US$275 a month more. Still unresolved: the provider’s answer to the quota request, and the real number of questions from the home page.”

Check your solution against the outline’s completion check. You defended a trade-off with numbers, and you said plainly what is not yet known.

See it end to endDesign review of an open document-question serviceCase study

The case study writes the decision records, the failure table and a roadmap for a real open document collection, with the same pack check.

Knowledge check

Check your understanding

Why does the validation work happen during release 1, and not after it? Give one example from Larkfield's roadmap.

Show the answer

Because the validation work needs real use, and release 1 is the first real use. The usage log can only replace the assumed demand once shops ask real questions. And some results can change what release 1 must contain. The home page answer decides whether the high scenario’s quota request must happen before launch. The first restore test decides whether ADR-0005 meets the RTO of 240 minutes. Doing this work after release 1 would discover these problems in production.

Check your understanding

Predict: you keep the ADR-0006 from the first lesson's exercise in the folder decisions, and you do not change the roadmap. What does python -m costmodel check print for the roadmap, and why is that useful?

Show the answer

FAIL roadmap: every ADR appears in the roadmap: roadmap.md does not link ADR-0006, and Design pack check: 8 of 9 passed. It is useful because a new decision with no place in the roadmap would never be built or tested. The fix is a step, probably in the validation work: “Measure routing for Bramble Books on its evaluation set (ADR-0006).”

Check your understanding

Module completion check: defend one trade-off of Larkfield's course-end design with numbers, and acknowledge one unresolved assumption explicitly.

Show the answer

A model answer: “We answer every question with chat-small and no routing (ADR-0003). On Grace’s 67 questions, chat-small was 86.6% correct and chat-strong 85.8%, measured. The strong model costs 19 times more per question. Routing would raise the base total from about US$52 to about US$78 a month. The trade-off: a harder question set could need the strong model. So we keep the routing code, switched off, and revisit when a new set shows chat-strong 5 points better. Unresolved: we have not measured Bramble Books’ questions yet. Priya builds that evaluation set during release 1, and until then its quality is an assumption.”

A complete answer names the trade-off and gives at least one measured number and one cost. It says what would change the decision. And it states one open assumption, with the work that will close it.

Recap

• An implementation roadmap orders the work. An incremental one has a first usable release with a done-when line, validation work, and later steps that each wait for a signal.
• A thin slice through every part reaches users first, so the first measurements arrive early. A layer-by-layer plan measures nothing until the end.
• Rank the validation work by how much each answer moves the cost or a decision. At Larkfield: the home page first (+46%, and the quota), then the restore test, then usage (+13%) and the cache share (about 3%).
• Every later step names its signal and its record. Each one costs money: routing +50%, one more service about US$35 a month, high availability US$275 more.
• python -m costmodel check prints 9 of 9 passed, and python -m pytest -q prints 49 passed. Your pack is now the snapshot end-of-m06.

Module 6 in one paragraph. You turned the design into something that other people can check, understand and build. Five decision records keep each big choice with its options, its numbers and the signal to revisit it. One record, the restore plan, still rests on an estimate. A ten-minute walkthrough answers Omar’s, Grace’s and Ines’s questions with rounded numbers, their ranges and their kinds. It shows about US$52 a month and about US$26 per shop. It ends with the one decision that only Omar can make: the home page. And a roadmap orders the work. A thin first release comes first, then the validation that replaces assumptions, then later steps that each wait for a signal.

The course in one paragraph. You started from people and their journeys, not from services. Then you drew one modular application and one database. You turned measured times, list prices and assumed demand into a cost: about US$52 a month for two shops, 87% fixed. You measured options before choosing: the strong model was not better, and a cache buys quota headroom more than money. You planned for failure and growth with signals, and found that the first limit is the provider’s quota. And you wrote it down so that others can decide. Next, the case studies apply the same pack to real open data, and the final assignment asks for your own proposal.

Next: Cost and capacity plan from real demand data