Handoff

What you will be able to do

Hand the design to another team in a form that they can check, not only read. Generate the security test report from the test results, collect the access matrix and the identity configuration, and write down what is still open. Every claim in the handoff has a command that proves it.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• The test matrix, from Negative testing, and the audit trail, from Auditability.
• The identity configuration and the real provider, from Provider configuration.
• How long access lasts after a change, from Lifecycle changes.

What you need on your computer

• The project after Auditability: python -m pytest prints 452 passed, 1 deselected.
• PostgreSQL and Azurite running, and the data loaded. The provider and the API are not needed for this lesson.

At the end of this lesson, your project equals the snapshot end-of-m06, the end of the course. Tested on macOS (Apple silicon); the Windows commands are written but not tested.

Handing over a house

You sell a house. The new owner gets the keys, but keys are not enough. A good seller adds a list. It says which key opens which door, and how to change the alarm code. It has the boiler’s last safety check, with its date. It also names the things not done yet: “the roof window leaks a little; we did not choose a repair”. Each line can be checked. The buyer tries each key, reads the certificate, and looks at the window.

A bad handover is a bag of keys and “it all works”. The first winter shows what “all” meant.

The problem: the design lives in one head

Amira built the access control of the ticket API. Larkfield’s operations team will now run it. Two short stories show what goes wrong without a handoff. Both use real results of this course.

The team moves sign-in to Keycloak, the real provider of Provider configuration. Every sign-in fails: “This is not an access token”. Keycloak’s access tokens say typ: JWT, and the API expects at+jwt unless ACCESS_TOKEN_TYPES=JWT. Amira knows. The team spends a day finding out.

Later, a developer sees that the API reads the membership table on every request. “Let us cache it for 30 seconds.” In Lifecycle changes, that setting kept a removed member reading for 30.3 s. Nobody wrote down why the cache is off, so the next person turns it on.

The same problem shows in the evidence. “We tested it” is a claim. Which tests, on which code, with which result? Kwame wants each answer from a command, not from a memory.

The idea: claims with evidence, and open questions

A handoff package is a small set of documents that answers four questions, each with a command that proves the answer:

The handoff package: four questions, four documents, and the commands that prove each claim.

Text description of the diagram

The first question, who may do what in which organization, is answered by docs/access-matrix.md. The second, how sign-in is set up, by docs/identity-config.md. The third, does the design hold, by docs/test-report.md, which a script writes from the test results. The fourth, how long access lasts and what is still open, by docs/handoff.md. Each document ends in commands: pytest, matrix_report, attempts, audit --check and lifecycle all.

You wrote two of the four documents already: the access matrix in Users and organizations, and the identity configuration in Provider configuration. This lesson adds the test report and the handoff itself.

The test report: generated, never typed

A report typed by hand can say “all passed” when they did not. So the report is written by a script, from the results of the test run. pytest writes its results as JUnit XML, a standard file that CI tools read. scripts/matrix_report.py runs the matrix with --junitxml, reads each test case’s result, and writes two files: docs/test-report.md for people and reports/access-matrix.json for programs.

1. Copy the report script and the handoff from the snapshot end-of-m06:

   macOS / Linux

   S=../nextia-learning-labs/C23/snapshots/end-of-m06
   cp "$S/scripts/matrix_report.py" scripts/
   cp "$S/docs/handoff.md" docs/

   Windows (PowerShell)

   $S = "..\nextia-learning-labs\C23\snapshots\end-of-m06"
   Copy-Item "$S\scripts\matrix_report.py" scripts\
   Copy-Item "$S\docs\handoff.md" docs\

2. Run it:

   python -m scripts.matrix_report

   213 of 213 checks passed. Wrote docs/test-report.md and reports/access-matrix.json.

   It takes some seconds, because it runs the whole matrix.

3. Open docs/test-report.md. Its start:

   docs/test-report.md (start)

   # Security test report: access control
   
   **213 of 213 checks passed.** Generated by `python -m scripts.matrix_report` from the results of `tests/test_access_matrix.py` (made-up people and data).

   Then one table row per person and organization, the four cases with the other shop’s record ID, and the bad identities:

   docs/test-report.md (end)

   ## Bad identities on every route
   
   - no identity: 11 of 11 routes answered 401
   - expired: 11 of 11 routes answered 401
   - wrong audience: 11 of 11 routes answered 401
   - ID token: 11 of 11 routes answered 401
   - no scope: 11 of 11 routes answered 403

   The table has the same rows and columns as the heat map in Negative testing.

The script exits with code 0 only when every check passed. A CI job can run it and stop a release on code 1. With the member-list bug of Negative testing, it printed 211 of 213 checks passed., exited with 1, and the table marked the cells: | Omar | larkfield | read_only | … | 403 **FAIL** | … |.

A report that lied, and its fix

While writing this lesson, the author ran the report with PostgreSQL stopped, by accident. The first version of the script printed this:

213 of 213 checks passed. Wrote docs/test-report.md and reports/access-matrix.json.

All 213 tests had been skipped, and the script counted “not failed” as “passed”. That is the worst kind of report: green, and false. The script in the snapshot now records a skipped check as SKIPPED, never as passed. With the database stopped, it prints:

0 of 213 checks passed. Wrote docs/test-report.md and reports/access-matrix.json.
213 checks did not run (skipped). Is PostgreSQL running?

and exits with code 1. Try it: stop the database (docker compose stop db), run the report, and start the database again (docker compose start db). The lesson for every report: count only what passed, and say what did not run.

The identity configuration: what was tested

docs/identity-config.md says what each side must know: the provider’s registration (client ID, redirect URI, PKCE, scopes, audience) and the API’s .env. The handoff must also say how much of it is proven. Three providers, three levels of evidence:

Three providers, three levels of evidence. Write the level next to each one: tested, tested once, or described.

Text description of the diagram

The practice provider in idp/ is tested by every test and every lesson. Keycloak 26.8.0 in Docker was tested once: it needed ACCESS_TOKEN_TYPES=JWT, an audience mapper and its basic scope. Microsoft Entra ID is described from its documentation and not tested. Its audience is an api:// address, its scopes are in scp, and the person’s ID is oid or sub. Some choices stay open for any real provider.

Provider	Evidence	What changed in the API’s settings
Practice provider (idp/)	every test; every lesson	none: the defaults
Keycloak 26.8.0	one run: sign-in 302 to /app/, Larkfield 200, Bramble 404, check_identity 0 problems	ACCESS_TOKEN_TYPES=JWT; an audience mapper and the basic scope at Keycloak; a membership row for Keycloak’s own sub, a UUID
Microsoft Entra ID	described, not tested	ACCESS_TOKEN_TYPES=JWT; issuer https://login.microsoftonline.com/<directory ID>/v2.0 with version 2 tokens; audience such as api://ticket-api

Never write “works with Entra ID” when you only read its documentation. Write “described, not tested”, as the course does. The next team then knows where the first real run must happen.

Open choices: write them down

Some questions have no answer yet, because they depend on a decision that is not Amira’s. A handoff lists them, so that nobody mistakes silence for a decision. From docs/identity-config.md:

• Which claim is the person’s stable ID. This API uses sub. Entra ID’s sub differs per application; its oid is the same in every application. If the ID changes, every membership row must move.
• Scope names. Some providers do not allow : in a scope, and some need an audience mapper before the token has aud: ticket-api at all.
• Logout. Whether the provider supports RP-initiated logout and back-channel logout. Without back-channel logout, a session at the API ends at its next refresh, within 10 minutes.
• Multi-factor sign-in and account recovery. The provider’s job, not the API’s: the API sees only the result.

docs/handoff.md adds the choices of the design itself. The membership cache is off, and the handoff says why. The audit trail needs a retention rule, and a rule for who outside an organization may read it. And somebody must decide what a deleted organization’s AI-run costs need.

Open the handoff and read its five sections: the model, the identity configuration, sessions and tokens over time, tests and evidence, and open choices. Its table of lifetimes is the short form of Lifecycle changes:

docs/handoff.md (part)

| Access token | 10 minutes (+30 s leeway) | nothing: the API cannot recall a token. Membership checks make it useless for an organization at once. |
| Refresh token | 8 hours, one use each (rotation) | logout (revoked at the provider), a disabled account, a reused old token (the whole family ends) |

Check the whole course

The outline’s completion check: pass all cross-tenant and privilege-escalation checks. Run the evidence commands of the handoff, in a fresh state:

1. Load the data again: python -m scripts.load --reset.

2. Run every test:

   python -m pytest -q

   The last line:

   452 passed, 1 deselected in 43.50s

   One test needs Azurite and is deselected by default. Run it with python -m pytest -m azurite: 1 passed, 452 deselected.

3. The cross-tenant and privilege-escalation checks, as a report: python -m scripts.matrix_report prints 213 of 213 checks passed.

4. The made-up attempts:

   python -m scripts.attempts 2>/dev/null | tail -1

   0 of 10 attempts got what they should not get.

5. The audit trail, after the attempts: python -m scripts.audit --check prints Checked 4 audit events: 0 with a token or a secret..

6. Compare your project with the snapshot. Files that are not in git (.venv, .env, .idp, .tokens, reports) and caches may differ:

   macOS / Linux

   diff -rq . ../nextia-learning-labs/C23/snapshots/end-of-m06 \
     -x .venv -x .env -x .idp -x .tokens -x reports -x __pycache__ -x .pytest_cache -x .ruff_cache

   Windows (PowerShell)

   git diff --no-index --stat . ..\nextia-learning-labs\C23\snapshots\end-of-m06

   diff prints nothing: your project is the course’s end. In PowerShell, the list also shows the ignored folders; only they may differ.

If a check fails, the course is not complete, even if every page works. A working happy path with one failing cross-tenant cell is a data leak with a nice front.

Your turn: a two-tenant demo and a test report

Prepare a 10-minute demo for Larkfield’s operations team, as if you hand the API over today. Use only made-up people and your own computer.

1. Two tenants, live. Sign in at http://127.0.0.1:8000/app/ as Camille. Show their Larkfield tickets, with a “Reply” button. Switch to Bramble Books: the list has no “Reply” button, because Camille is read-only there. Then send the reply yourself with Camille’s token and curl, as Omar did in Authorization in the API, and show the 403. Sign in as Tomás: show that he sees nothing.
2. The evidence. Run python -m scripts.matrix_report and show docs/test-report.md. Run python -m scripts.attempts. Show the denied events with python -m scripts.audit --result denied.
3. The report. Write one page, docs/demo-notes.md. Name the three questions and where the API answers each. List the test results with their commands, and the evidence level of each provider. End with your two most important open choices, and the person who must decide each.

A good demo shows refusals, not only successes. The audience should leave knowing what fails, how they know, and what is still open.

See it end to endAudit access control in an open-source multi-tenant appCase study

The case study uses the same matrix idea on an application that you did not build. You map its users, organizations and roles, and test its isolation.

Common mistakes

Common mistake

The report says 0 of 213

Symptom: python -m scripts.matrix_report prints 0 of 213 checks passed. and 213 checks did not run (skipped). Is PostgreSQL running?

Cause: the test database cannot be reached, so every test skipped itself.

Fix: start the database (docker compose up -d db azurite) and run the report again. Never commit a report with skipped checks.

Common mistake

ModuleNotFoundError: No module named 'tests'

Symptom: python -m scripts.matrix_report stops at once with this error.

Cause: the command runs outside the project folder. The script imports the routes and people from tests/test_access_matrix.py.

Fix: run it from the project folder, with the virtual environment active.

Check your understanding

Why is the test report generated from pytest's JUnit results, and not typed by hand or built from the expected values?

Show the answer

A typed report, or one built from the expected values, can say “passed” for a check that failed or never ran. The JUnit file records what each test case did. The report must also count skipped checks as not passed. The first version counted them as passed, and printed 213 of 213 with the database stopped.

Check your understanding

The handoff says Keycloak was 'tested once' and Entra ID 'described, not tested'. Give one reason why the difference matters to the next team.

Show the answer

A real run finds what documentation hides. The Keycloak run found two bugs that the practice provider had hidden: a PKCE verifier that was too short, and a unique email. Entra ID may have surprises of its own: the next team must plan its first real run, and must not trust the description as proof.

Check your understanding

Module check: for each request, give the status and say whether it makes an audit event, with or without an organization. (a) Omar adds a message at Larkfield; (b) Tomás reads Larkfield's T-30002; (c) Sam reads Bramble's T-40001 through Larkfield's path; (d) an expired token lists Larkfield's tickets.

Show the answer

(a) 403, an event with organization larkfield, reason role_cannot. (b) 404, an event without organization, reason not_a_member, the path in its details. (c) 404 “Ticket T-40001 does not exist.”, no event: Sam passes the checks at Larkfield and the lookup finds nothing; only the log has it. (d) 401, an event token.check with reason expired, no actor, no organization and no token. The test matrix checks all four statuses; test_auditability.py checks (a), (b) and (d).

Check your understanding

Practical check (completion): which four commands prove that your project passes all cross-tenant and privilege-escalation checks, and what must each print?

Show the answer

python -m pytest: 452 passed, 1 deselected. python -m scripts.matrix_report: 213 of 213 checks passed. python -m scripts.attempts: 0 of 10 attempts got what they should not get. python -m scripts.audit --check: 0 events with a token or a secret. Your project then equals the snapshot end-of-m06.

Recap

• A handoff package answers four questions with documents and commands. It has the access matrix, the identity configuration, the test report, and the handoff with lifetimes and open choices.
• The test report is generated from the test results: 213 of 213 checks passed. A skipped check is never counted as passed.
• Write the evidence level of each provider: the practice provider and Keycloak tested, Entra ID described, not tested.
• Write the open choices down, with who must decide. Examples: the stable person ID, scope names, logout, multi-factor sign-in, the membership cache, the audit’s retention.

Module recap. You tested what must fail for every person, organization and route, recorded every refusal without a secret, and handed the design over with its evidence. Your project equals end-of-m06, the end of the course.

Course recap. The API now answers three questions on every request. Who: a checked token or session. In which organization: the path and the caller’s own membership. What: the membership’s role and the token’s scopes. Every query, file link, search and cache key carries the organization, and the database checks it again.

Next: Audit access control in an open-source multi-tenant app