Simplicity and evolution

What you will be able to do

Decide whether Larkfield’s parts run as one application or as separate services, with numbers instead of fashion. Then write down the measured signals that would make a split worth its cost, so the design can change later on purpose.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• The eight parts and the two processes, the API and the worker, from Service boundaries.
• The tenant and the versions in every flow, from Data and identity flows.
• Replicas: more copies of the same application behind one address, from Horizontal scaling in Scaling APIs and AI Workloads.

What you need on your computer

• Your pack from the last lesson: python -m pytest -q prints 1 failed, 15 passed. You write one more file in the module practice at the end.

A bakery that grows

A family opens a bakery. One building holds the shop counter, the kitchen and the storeroom. Each room has a door and a clear job. But one key opens the building, one alarm protects it, and one person locks up at night.

Years later, wedding cakes become a big business. They need a large oven, a separate team and their own opening hours. Now the family rents a second building for cakes. The reason is concrete: the oven does not fit, and the cake team works on Sundays when the shop is closed.

A family that rents two buildings on the first day pays two rents and two alarms. Every order walks across the street. The second building is not wrong. It is early, and nobody can say when it will pay for itself.

The problem: services before need

Tomás proposes four separate services: an answer service, a search service, an ingestion service and a tenant service. Each would have its own deployment, and they would talk over the network. He wants Kubernetes to run them. “Then each part can grow on its own,” he says.

Look at what the parts must do today. In the base case, the busiest minute brings 0.268 questions per second. Each answer spends about 2.18 s in the system. So about 0.268 × 2.18 = 0.58 requests are in flight at the busiest moment: started and not yet finished. Module 3 explains this multiplication. Less than one. One small copy of the application waits most of the day.

Now the cost. A service that must answer at any time needs at least one copy that runs all month. We ran the pack’s own calculator, from Module 3, with one, two and three always-on copies of the API’s size. Each copy has 0.5 vCPU and 1 GiB. The table shows the base case, month 0. After Module 3, you can repeat it: change min_replicas in design.toml.

Design	Always-on copies	Compute, US$ per month	Total, US$ per month
One modular application: API always on, worker only while it works	1	27.43	52.67
Answer and search as separate services, plus the worker	2	62.42	87.66
Answer, search and tenant services, plus the worker	3	97.42	122.65

This is a rough model, and it is kind to the split. It leaves out the cluster, the network between the parts, more logs and more people time. Even so, three always-on services use 82% of Omar’s budget of US$150 a month (Q8), before any growth.

And none of it makes an answer faster. The model writes for 1.52 s of the 2.18 s. The database adds about 0.1 ms for row-level security and 0.22 ms for a vector search. A split moves function calls onto the network, so it adds time and new ways to fail.

The idea: a modular application, with split triggers

A modular application is one code base and one deployment, with clear modules inside. Larkfield’s modules are answers, search, ingestion and tenants. Each module has one job and a small set of functions that others may call. Modules call each other as functions in the same program, not over the network. The same code can still run as more than one process: the API, always on, and the worker, only while there are jobs.

Separate services (often called microservices) put each module in its own deployment. They talk over the network. Each can be released, scaled and run on its own hardware. That freedom is real, and so is its price: more deployments, more network calls that can fail, and more places for the tenant rule.

A split trigger is the bridge between the two. It names a measured signal, a threshold and the change it justifies: “if ingestion needs a GPU, move the worker to its own service”. You write the triggers today, while the design is simple. Then the design can evolve: it changes in small steps, each one paid for by a measured need. You do not guess the final architecture on day one.

The same four modules, packed two ways. On the left, the arrows are function calls; on the right, each arrow is a network call that can be slow or fail.

Text description of the diagram

On the left, one modular application: one code base and one deployment. Inside it, the answers module calls search and tenants, and the ingestion module calls search. The code runs as two processes, the API, always on, and the worker, on demand. On the right, the same modules as separate services, in orange. The answer service calls the search service and the tenant service over the network, and the ingestion service calls the search service. That makes four deployments and four release pipelines, with three services always on. On a phone, the diagram shows two modules on each side.

A tiny example: what one more always-on part costs

Here are the three totals from the table as a chart, next to Omar’s budget. Before you read on, look at the compute column. Does the second copy cost more or less than the first one? Why?

The pack's calculator, base case, month 0, with one, two and three always-on copies of the API's size. A rough model: it leaves out the cluster, the network and people time, so a real split costs more.

Text description of the chart

Four columns, in US dollars per month, labelled by the number of always-on parts. One always-on part: 52.67. Two: 87.66. Three: 122.65. Omar’s budget for all tenants together: 150. Each extra always-on part adds about 35 dollars a month.

The second copy costs more than the first. The compute rises by 62.42 − 27.43 = US$34.99, while the whole first copy cost US$27.43. The reason is in the price list. The cost model runs the application on Azure Container Apps, whose price page gives a free monthly amount of compute. The first copy already uses it. Every copy after that pays the full price. The third copy adds US$35.00 more.

So the cost of a split is not “a little more of the same”. The first part is cheap, and each extra always-on part pays full price, every month, even when nobody asks a question.

Larkfield’s choice

Kwame compares the two designs on the things that matter for this workload, with Amira:

Question	One modular application	Separate services
Deployments and release pipelines	1	4
Network calls inside one answer	0	at least 2 (search, tenant)
Places where the tenant rule must hold	1 database, 1 code base	1 database, 4 code bases
Ingestion away from answers	yes: the worker is its own process	yes
Worker cost when there are no uploads	0: it scales to zero	0
Compute, base case (rough)	US$27.43 a month	US$97.42 a month
Who maintains it (Q6: one developer)	Amira alone	needs more people

Notice the fourth and fifth rows. The main benefit of a split, keeping slow ingestion away from answers, is already in the modular design. The worker is a separate process that reads the job queue, and it scales to zero. What the split adds is independent releases, separate hardware and separate scaling for search and tenants. Today nothing needs those.

So the decision is one modular application with two processes. In Module 6 it becomes the first architecture decision record. And because the modules already have clear boundaries, a later split is a move of code, not a rewrite.

The split triggers

A decision without its “revisit when” is a guess that nobody checks again. Kwame writes three triggers. Each has a signal that someone can measure, and a change that follows.

Three triggers, each with a signal and the one change it justifies. Until a signal crosses its line, the design stays simple.

Text description of the diagram

Today’s design is one application with the API and the worker. Three triggers can change it. If ingestion needs a GPU or its own release rhythm, the worker becomes its own service. If the API needs more than 4 replicas at the busiest minute, the part that limits the API is split out. If a second team owns part of the code, the split follows that team’s modules. Each trigger changes one part, not the whole design.

Trigger	How it is measured	Today
Ingestion needs a different machine (a GPU) or a different release rhythm	the model choice (Module 4 compares a self-hosted model); the release log	the worker calls the model provider; no GPU
The API needs more than 4 replicas at the busiest minute	requests in flight at the busiest minute, from the logs; the growth plan of Module 5	0.58 in flight, 1 replica
A second team owns part of the code	the team list	one developer, Amira

Each trigger is concrete. “When we get big” is not a trigger, because nobody can say when it is true. “More than 4 replicas at the busiest minute” is a number that the logs show every day. Module 5 turns these signals into a growth plan, and Module 6 writes them into the decision record.

Tomás asks: “And if we are wrong, and the shops grow fast?” Kwame answers with the high case of Module 1: 13,874 questions a day in month 12, with four shops. Module 3 works out what that needs, and Module 5 watches for it. If a trigger fires, the team splits one part, with the measurement in hand. That is cheaper than four services today, paid every month on a guess.

When to choose which, and what each costs

Start with one modular application when one team builds the product, the demand is small or uncertain, and the parts share one database. That describes most new products, and it describes Larkfield. Keep the modules strict. Each module calls only the functions that another module offers, and never reads another module’s tables directly. That rule makes a later split cheap.

Choose separate services when a trigger has fired. One part needs different hardware, or a different release rhythm, or it scales very differently from the rest. Or several teams must release without waiting for each other. Then the extra deployments buy something real.

Both choices cost something. The modular application can grow into a tangle if nobody guards the module boundaries. Separate services cost always-on compute, network calls, and a tenant rule in every service. Write down what you chose, why, and when you will look again.

Common mistakes

Common mistake

A trigger that nobody can measure

Symptom: the design says “split into services when we reach scale”. A year later, Tomás and Amira argue about whether “scale” has arrived. Nobody can show a number.

Cause: the trigger names a feeling, not a signal. There is no threshold and no place to read it.

Fix: rewrite it as a signal, a threshold and a source. For example: “more than 4 API replicas at the busiest minute, from the request logs”. Check that someone already collects that signal today. If nobody does, the trigger needs a log line first.

Common mistake

Services that share one table

Symptom: after a split, a change to the chunks table in the ingestion service breaks the search service. Two teams must now release together.

Cause: the services were split, but the data was not. Both read and write the same table, so they are still one system, with a network in the middle.

Fix: before any split, give each module its own tables, and let others use them only through the module’s functions. Then a split moves code and tables together. In the modular application, this rule costs nothing and keeps the option open.

Your turn: two more shops

Change one condition. Omar signs the third and fourth shops. In month 12, the base case has four tenants and 2,304 questions a day. Does this fire a split trigger? Which journey and which rule answer the question?

Show the answer

No trigger fires. A new shop is data, not code: journey J5 says “a new tenant without a code change”. The new shops add rows in PostgreSQL, a folder in file storage, and their users in the identity provider. The tenant rule already holds for them.

The demand grows from 1,465 to 2,304 questions a day in the base case. That is still far below the second trigger: the busiest minute stays a fraction of one request in flight, with one replica. The third trigger does not fire either, because the team is the same. Module 3 checks the cost and Module 5 the growth, with the same triggers.

Module practice: the ingestion sequence

You now have the container diagram, the data and identity flow and the request flow. One picture is missing: what happens when a shop uploads a document. That is the module’s independent task.

A worked example, small. The last lesson’s request flow followed one question. Here is one upload instead, as a table of states. Ines uploads version 5 of Bramble Books’ returns policy; version 4 is active. Watch the states change, step by step. The numbers are constructed for the example.

After the step	File of version 5	Version 5	Version 4	Job 812	Documents version
The API saves the file	saved	not yet	active	not yet	7
The API writes one transaction, then replies 202 Accepted	saved	processing	active	queued	7
The worker takes the job	saved	processing	active	running	7
The worker embeds and summarizes	saved	processing	active	running	7
The worker writes one transaction	saved	active	replaced	succeeded	8

Two things stand out. Until the last row, every question still uses version 4, and customers get correct answers from it. And the change happens in one transaction: the new chunks, the summary, both version states, the job and the documents version together. If the worker stops after it embeds, nothing is half-saved. The job’s lease (its time limit) ends. Then a worker takes the job again from the start, as in Job lifecycle in the scaling course.

Your task. Write architecture/ingestion-sequence.md with a Mermaid sequenceDiagram of this upload. Use six participants: Ines (the editor), the Assistant API, file storage, PostgreSQL, the Ingestion worker and the model provider. Show where the tenant comes from, where the request ends for Ines, and which step is the one transaction. Add a note about a worker that stops halfway. Then run the tests.

Show a worked solution

The ingestion sequence of journey J3. Ines's request ends at step 4; the slow work runs in the background and ends in one transaction.

Text description of the diagram

Ines uploads version 5 of the returns policy. The API saves the file in Bramble Books’ folder and writes one transaction: version 5 with the status “processing”, and a job row with the tenant from her token. It replies 202 Accepted at once. The worker takes the next job with SKIP LOCKED, so two workers never take the same job. It reads the file and parses and chunks it, in about 0.1 ms. It embeds all chunks in one call, in about 0.4 s, and summarizes in about 1.9 s. Then one transaction saves the chunks, the vectors and the summary, and marks version 5 active, version 4 replaced and the job done. If the worker stops halfway, the job runs again, and nothing is half-saved. The times are measured medians from the course team’s calls.

The file in the snapshot end-of-m02 is one possible text for this picture. Compare it with yours, then run the tests:

python -m pytest -q

................                                                         [100%]
16 passed in 0.02s

The last test checks that both sequences name the parts of the container diagram. The ingestion sequence must also name the Ingestion worker and file storage. Your pack is now the snapshot end-of-m02.

Check your sequence against four points. The tenant comes from the upload’s token, through the job row, and never from the file name. Ines’s request ends with 202, before any model call. The last write is one transaction. And the worker can stop at any step without leaving half a document.

Knowledge check

Check your understanding

Tomás says: 'Separate services are the modern way, and each part can grow on its own.' Give Kwame's answer with this lesson's numbers, in three or four sentences.

Show the answer

At the busiest minute of the base case, about 0.58 requests are in flight, so one copy of the application is enough. The part that must grow on its own, ingestion, already runs as its own process and scales to zero. Three always-on services would raise the rough monthly total from US$52.67 to US$122.65, before the cluster and people time. And no answer would get faster: the model takes 1.52 s of 2.18 s. We keep one modular application and split a part when a written trigger fires.

Check your understanding

Predict: why does the second always-on part add US$34.99 a month, when the whole first part costs only US$27.43?

Show the answer

Because the price list has a free monthly amount of compute, and the first always-on part already uses it. Every extra part pays the full price for every second of the month, even when it is idle. So the first part is the cheapest one, and each part after it costs about US$35 a month in this rough model.

Check your understanding

Module completion check: explain the responsibility of every part on the container diagram, and why each one exists as its own part.

Show the answer

• Help-desk tool (Amira): shows the assistant to agents and customers. It exists because the people use it; it is where the journeys start.
• Identity provider (Kwame): signs people in and issues a token with user, tenant and role. It is outside: we use it, we do not build it, and every flow starts from its token.
• Assistant API (Amira): answers one tenant’s question: checks the token, searches, calls the model, saves the answer. It exists because a person waits for it.
• Ingestion worker (Mei): turns an uploaded document version into chunks, vectors and a summary. It exists because nobody waits for it, and it must never block an answer.
• PostgreSQL (Mei): keeps every record, with row-level security. It is one store for records, vectors, the queue and the cache, so there is one backup and one tenant rule.
• File storage (Mei): keeps the original file of every version. Files are large and rarely read, so they live outside the database, in one folder per tenant.
• Model provider (Kwame): runs the chat and embedding models. It is outside, with a quota and outages we plan for.
• Logs and alerts (Amira): collects logs and alerts a person. It exists so that the targets of Module 1 can be measured and the split triggers can be read.

Retrieval and model access are modules inside the application. They have boundaries, but no box of their own, because nothing measured today needs them to run apart.

Recap

• A modular application is one code base and one deployment, with strict modules inside. It can still run as two processes: Larkfield’s API and worker.
• Separate services buy independent releases, hardware and scaling, at the cost of always-on compute, network calls and more places for the tenant rule.
• At the busiest base minute, 0.58 requests are in flight. In the pack’s rough model, the total goes from US$52.67 (one always-on part) to US$87.66 (two) and US$122.65 (three). The free compute grant covers only the first part.
• A split trigger names a signal, a threshold and one change. Larkfield’s three: ingestion needs a GPU or its own releases; more than 4 API replicas at the busiest minute; a second team.
• python -m pytest -q prints 16 passed. Your pack is now the snapshot end-of-m02.

Module 2 in one paragraph. You split the assistant into eight parts, each with one job and one owner. You asked who waits, which data each part keeps, and whether it is ours. An answer and an ingestion both take about two seconds, but a person waits only for the answer, so ingestion runs in a background worker. You traced the tenant, the audience and the document versions through every request and every job. The token, the transaction, the cache key, the job row and the file path each carry the tenant, and every answer keeps its versions. And you chose one modular application over separate services, with three measured triggers that would justify a split.

Next, Module 3 turns this architecture and the demand of Module 1 into resources and a monthly cost.

Next: Workload arithmetic