Package and deploy an adapter

What you will be able to do

An adapter is a small file that works only on top of one exact base model. You package it with everything it depends on and load it in two ways. You check it with fixed tickets, and you know what to switch back to.

Before you startWhat to know and to have readyAbout 40 minutes

What you need to know

• What an adapter trains, from Full and parameter-efficient tuning.
• The result memo and why it says “do not ship”, from Regressions and operational value.

What you need on your computer

• ~/projects/ticket-adapter at the end of Module 5, with .venv active: python -m pytest ends with 34 passed, 2 deselected. If it does not, copy the snapshot end-of-m05 as Use a snapshot explains.
• The base model’s weights, from python -m adapt download in Plan and configure the run.
• The labs repository in ~/projects/nextia-learning-labs, up to date (git -C ../nextia-learning-labs pull).

No account and no training. Each check run loads the model once: about 1.3 GB of memory, a few seconds. Tested with Python 3.12.13 on macOS (Apple silicon). The Windows commands are written, not tested.

A lens that fits one frame

An optician cuts a lens for one frame. The box says “for the Aria frame”. Next year, the maker changes the Aria a little. The lens still clicks into the new frame, but the picture is blurred. Nothing broke loudly. The box named the frame, but not which year.

The problem: the adapter loads on the wrong model

The course’s adapter has the same weakness. Its file adapter_config.json names its base model, but not the version:

"base_model_name_or_path": "HuggingFaceTB/SmolLM2-135M-Instruct",
"revision": null,

So nothing stops a tool from loading it on another model with the same shapes. The course attached the adapter with peft to SmolLM2-135M, the base model before its instruction training: same layers, other weights. Then it asked the 8 check tickets of this lesson (a one-off run; you do not need to repeat it):

adapter_config.json names: HuggingFaceTB/SmolLM2-135M-Instruct (revision: None)
model given to peft:       models--HuggingFaceTB--SmolLM2-135M @ 93efa2f097d5
warnings while loading: 0
  FAIL  C-01  expected delivery/false  got 'I am getting a message from the tracking system that the par' (not JSON)
  FAIL  C-02  expected delivery/false  got 'The garden bench arrived today with a broken leg. The box wa' (not JSON)
  FAIL  C-03  expected returns/false  got 'I would like to send back the plant pots. I changed my mind ' (not JSON)
  FAIL  C-04  expected payment/false  got 'You paid for the lawn mower order. Reply with JSON only, in ' (not JSON)
  FAIL  C-05  expected warranty/false  got 'After three months of use, the hose reel started to leak at ' (not JSON)
  FAIL  C-06  expected account/false  got 'I cannot sign in. The reset email never arrives.\nwarranty\nwa' (not JSON)
  FAIL  C-07  expected warranty/true  got 'The electric heater sparked and the plug got very hot. I am ' (not JSON)
  FAIL  C-08  expected warranty/true  got 'The kettle was on the table, and the kettle was on the floor' (not JSON)
0 of 8 checks passed

peft gave no error and no warning, and not one reply is JSON. The adapter’s changes were trained for the instruction model’s weights; on other weights they are wrong. A deployment without a check would ship this silently.

The idea: a bundle and a check before every load

A bundle is a folder that holds the adapter and the exact names of everything it depends on. A compatibility check compares those names with what is on the computer, and refuses to load if one differs. A fixed check (a smoke test) then asks a few tickets with known answers. It catches a broken load, not a weak model: 8 tickets cannot measure quality.

The bundle also carries the evidence: the settings, the data manifest, the evaluation and a model card. A model card is a short page: what the model does, how the team trained and evaluated it, and its limits.

The compatibility map: what the adapter names, what the bundle adds, and what each way of loading it gives.

Text description of the diagram

The map has three steps.

1. The adapter folder from training. Its adapter_config.json names the base model, HuggingFaceTB/SmolLM2-135M-Instruct, with revision: null. The weights file is 1.86 MB.
2. The bundle adds bundle.json: the base model’s revision 12fd25f77366, the SHA-256 of config.json and of the tokenizer, the run settings, the data manifest and the evaluation. It also holds the model card and 8 fixed checks.
3. Verify compares the bundle with the computer. If the base model, its revision, its config or the tokenizer differs, it prints INCOMPATIBLE and loads nothing. Otherwise it loads the model and runs the checks. Loaded at runtime: 6 of 8 pass; C-07 and C-08, the two safety tickets, fail. Merged: 6 of 8, the same replies. The base model alone: 0 of 8, because it never answers in JSON.

The rollback for this task is the hosted assistant, kept configured. On the holdout cases it sent all 24 tickets that need a person to one.

Make the bundle

1. In the project folder, with .venv active, point a variable at the snapshot:

   macOS and Linux

   S=../nextia-learning-labs/C12/snapshots/end-of-m06

   Windows

   $S = "..\nextia-learning-labs\C12\snapshots\end-of-m06"

   If you open a new terminal, set it again.

2. Copy the bundle code, the new commands, the checks and their test:

   macOS and Linux

   cp $S/adapt/bundle.py $S/adapt/__main__.py adapt/
   cp $S/data/checks.jsonl $S/data/checks.md data/
   cp $S/tests/test_bundle.py tests/

   Windows

   Copy-Item $S\adapt\bundle.py, $S\adapt\__main__.py adapt\
   Copy-Item $S\data\checks.jsonl, $S\data\checks.md data\
   Copy-Item $S\tests\test_bundle.py tests\

3. Run python -m pytest. The last line starts with 36 passed, 3 deselected.

4. Package the course’s adapter, the 6-epoch run:

   python -m adapt bundle lora-r8-long

   Wrote bundle:
     MODEL_CARD.md                           1,725 bytes
     adapter/adapter_config.json             1,113 bytes
     adapter/adapter_model.safetensors   1,858,776 bytes
     bundle.json                             2,593 bytes
     checks.jsonl                            1,221 bytes

5. Open bundle/bundle.json. The part that the check reads:

   "base_model": {
    "id": "HuggingFaceTB/SmolLM2-135M-Instruct",
    "revision": "12fd25f77366fa6b3b4b768ec3050bf629380bac",
    "config_sha256": "8eb740e8bbe4cff95ea7b4588d17a2432deb16e8075bc5828ff7ba9be94d982a"
   },
   "tokenizer_sha256": "9ca9acddb6525a194ec8ac7a87f24fbba7232a9a15ffa1af0c1224fcd888e47c",

   Further down: the run’s settings, the data manifest (train 73f29c94b022) and the evaluation.

6. Open bundle/MODEL_CARD.md. Its evaluation lines:

   - dev: answers 102/102 | team 89/100 | needs a person: caught 0/23, false alarms 0 | decisions 74/102
   - holdout: answers 98/98 | team 87/97 | needs a person: caught 0/24, false alarms 0 | decisions 65/98

   The card tells the reader to read “caught” first: the bad news travels with the file.

Two ways to load: at runtime or merged

You can load the base model and put the adapter beside it at runtime. Or you can merge: add the adapter’s change into the base weights once, and keep one model. A tiny example with made-up numbers shows that both give the same answer. The scale is alpha ÷ rank = 16 ÷ 8 = 2, as in the course’s run.

Merging, with made-up 2 × 2 numbers: two paths added at runtime give the same output as one changed matrix.

Text description of the diagram

Loaded at runtime, the input x = [1, 1] goes through two paths. The frozen base gives W·x = [1, 1]. The adapter gives A·x = [3], then B·[3] = [1.5, 0], then times the scale 2: [3, 0]. The sum is [4, 1].

Merged, the change is added to the matrix once: W’ = W + 2·B·A has the rows [3, 1] and [0, 1]. Then W’·x = [4, 1], the same answer, in one step.

Now load the real bundle both ways. verify checks compatibility, then asks the 8 fixed tickets of data/checks.jsonl:

python -m adapt verify bundle

lora-r8-long on HuggingFaceTB/SmolLM2-135M-Instruct@12fd25f77366: base + adapter, loaded at runtime
  pass  C-01  expected delivery/false  got '{"team": "delivery", "needs_human": false}'
  pass  C-02  expected delivery/false  got '{"team": "delivery", "needs_human": false}'
  pass  C-03  expected returns/false  got '{"team": "returns", "needs_human": false}'
  pass  C-04  expected payment/false  got '{"team": "payment", "needs_human": false}'
  pass  C-05  expected warranty/false  got '{"team": "warranty", "needs_human": false}'
  pass  C-06  expected account/false  got '{"team": "account", "needs_human": false}'
  FAIL  C-07  expected warranty/true  got '{"team": "warranty", "needs_human": false}'
  FAIL  C-08  expected warranty/true  got '{"team": "warranty", "needs_human": false}'
6 of 8 checks passed

The adapter passes every routing check and fails both safety tickets: a sparking heater and a smoking kettle base. The command ends with exit code 1, so a deployment script stops here. With --merged, the replies are the same, and again 6 of 8 checks passed.

The course measured both ways on 30 dev cases. All 30 replies were identical; the median was 0.262 s per answer loaded and 0.244 s merged. Merging itself took 0.01 s. The real difference is what you store and ship:

One adapter is less than 1% of the size of a merged model. A full fine-tune of the same model stores every parameter.

Text description of the chart

A column chart of size on disk. The adapter (weights and config): 1.86 MB. The merged model in bf16: 269.1 MB. A full fine-tune in float32, counted by python -m adapt params, not trained: 538.1 MB.

What this tells us. Keep the adapter as the thing you version and ship: it is small, and you can remove it. Merge only at the end, for one fixed deployment where the small speed gain matters.

Roll back

A rollback is a return to the last version that worked. Is the base model, without the adapter, a rollback?

python -m adapt verify bundle --base-only

lora-r8-long on HuggingFaceTB/SmolLM2-135M-Instruct@12fd25f77366: the base model only (rollback)
  FAIL  C-01  expected delivery/false  got 'You can find the tracking information for your parcel on the' (not JSON)
  FAIL  C-02  expected delivery/false  got '{"team": "delivery" or "returns" or "payment" or "warranty" ' (not JSON)
  FAIL  C-03  expected returns/false  got 'To get a return label, you can use the following steps:\n\n1. ' (not JSON)
  FAIL  C-04  expected payment/false  got 'You charged my card twice for the same lawn mower order. Ple' (not JSON)
  FAIL  C-05  expected warranty/false  got 'Based on the information provided, it is not covered by the ' (not JSON)
  FAIL  C-06  expected account/false  got "I'm sorry for the inconvenience, but as an AI, I don't have " (not JSON)
  FAIL  C-07  expected warranty/true  got 'You need to respond with JSON data, in this format:\n\n{"team"' (not JSON)
  FAIL  C-08  expected warranty/true  got 'The kettle base started smoking the first time I used it, an' (not JSON)
0 of 8 checks passed

No: it answers in prose, so every check fails. For this task, the rollback is the hosted assistant, kept configured so that a switch is a setting, not a new release.

Discuss: ship it behind a switch?

Amira, the developer who owns the assistant, has a verified bundle.

• Tomás: “Deploy it to a fifth of the tickets, merged for speed. It passes 6 of 8, it is twelve times faster, and it costs nothing per ticket. We can switch it off.”
• Kwame: “The hosted assistant costs about US$8.53 a month at 822 tickets a day (an assumption).”
• Priya, pointing at the two failures: “Look at which checks fail.”
• Grace: “What would a customer with a sparking heater get?”

Before you read on, answer these:

1. Does merging change anything in the decision?
2. Is “6 of 8” nearly a pass?
3. Could the adapter route the tickets while the hosted model keeps the safety decision?

The course’s answer. Merging changes nothing: the replies are identical. “6 of 8” is not nearly a pass, because the two failures are the checks that matter most. They match the evaluation: 0 of 24 holdout tickets that need a person were sent to one. The customer with a sparking heater gets an ordinary draft, and no person reads it. A fifth of 822 is still about 164 tickets a day. The split design is fair to ask, but the adapter also routes worse: team 87 of 97 on holdout, against 97 of 97. So the bundle stays unused.

What would change the answer: a new adapter that passes C-07 and C-08, then the targets on dev and on a fresh holdout. Saving US$8.53 a month would not.

When to merge, and what each choice costs

Choice	Use it when	Cost
Load at runtime	You version, test or swap adapters	1.86 MB per adapter; one more step at load
Merge	One fixed deployment, speed matters	269 MB per version; the adapter cannot be removed
Rollback switch	Always, before any deployment	The old system stays configured and paid for

Common mistake

A bundle for another base revision

Symptom: a colleague’s bundle was made for another revision of the base model. Here it is constructed: the revision in bundle/bundle.json was changed by hand. verify prints:

INCOMPATIBLE: base model HuggingFaceTB/SmolLM2-135M-Instruct@a3c2d1e9f0b8 is not HuggingFaceTB/SmolLM2-135M-Instruct@12fd25f77366

It exits with code 1 and loads nothing.

Cause: the adapter was trained on other weights. As the wrong-model test showed, it could load anyway and answer badly.

Fix: do not edit bundle.json to make the check pass. Download the revision named in the bundle, or train the adapter again on your revision. Then run python -m adapt bundle lora-r8-long again to restore your bundle.

Knowledge check

Check your understanding

The adapter's adapter_config.json says revision: null. What can go wrong, and which part of the bundle prevents it?

Show the answer

A tool can load the adapter on another version of the base model with the same shapes, without an error. The answers are then wrong, as on SmolLM2-135M. bundle.json records the base revision and the SHA-256 of the config and the tokenizer, and verify refuses to load when one differs.

Check your understanding

Loaded and merged give the same 6 of 8. Name one reason to keep loading at runtime, and one reason to merge.

Show the answer

Loading at runtime keeps a 1.86 MB file that you can version, remove or swap for a new adapter on the same base. Merging gives one model and one path, a little faster (0.244 s against 0.262 s per answer here). It costs 269 MB per version, and the adapter can no longer be taken out.

Check your understanding

Tomás says: 'If the adapter fails, we roll back to the base model.' Which evidence would make you disagree, and what is the rollback here?

Show the answer

verify bundle --base-only passes 0 of 8 checks: the base model answers in prose, never in the JSON format. The rollback is the hosted assistant, kept configured so that switching back is a setting. On the holdout it sent 24 of 24 tickets that need a person to one.

Recap

• An adapter works only on its exact base model. Its own config names the model but not the revision, and a wrong model can load it without an error.
• A bundle adds the revision, the SHA-256 of the config and tokenizer, the settings, the data manifest, the evaluation and a model card.
• verify refuses an incompatible bundle, then runs 8 fixed checks. Loaded or merged: 6 of 8, both safety checks failed. The base model alone: 0 of 8.
• Merging gives the same replies, a little faster; version the 1.86 MB adapter.
• The rollback is the hosted assistant, not the base model. The adapter is packaged and not deployed.

Next: Lifecycle planning