Auditability

What you will be able to do

Make the API remember every access decision: who acted, in which organization, what they tried, and the result. Record the refusals too, without a token or a secret in the record. Then check that the record answers two questions: who did what, and who tried.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• The audit table, support access and the platform administrator, from Privileged operations.
• The order of the checks (identity, membership, role, then the lookup), from Authorization in the API.
• The test matrix, from Negative testing.

What you need on your computer

• The project after Negative testing, with tests/test_access_matrix.py.
• PostgreSQL and Azurite running, the data loaded, and two more terminals: the provider (python -m idp) and the API (fastapi dev).

Every refused request in this lesson is a constructed case, sent to your own practice API. Tested on macOS (Apple silicon); the Windows commands are written but not tested.

The visitors’ book

An office building has a reception desk and a visitors’ book. A good book has one line per visit: the name, the company visited, the time, and who signed them in. Now imagine that the guard writes only the visitors who got in. A stranger tries the door five times at night and is sent away five times. The book shows nothing. Next week, the owner asks, “Did anybody try to get in?”, and the book cannot answer.

The book also must not hold the wrong things. A guard who copies each visitor’s door code into the book makes the book itself a key.

The problem: refusals leave no trace

Privileged operations added the table audit_events. It already records changes, such as an invitation, and every use of support access. Does it record the refusals? Try it. Load the data again, so that the table starts empty:

python -m scripts.load --reset

Then send four requests: three that must fail, and one that must work.

1. Get the tokens of three people. On Windows, type curl.exe instead of curl in each command below:

   macOS / Linux

   OMAR=$(python -m scripts.login --user usr-omar --print-access-token)
   SAM=$(python -m scripts.login --user usr-sam --print-access-token)
   GRACE=$(python -m scripts.login --user usr-grace --print-access-token)

   Windows (PowerShell)

   $OMAR = python -m scripts.login --user usr-omar --print-access-token
   $SAM = python -m scripts.login --user usr-sam --print-access-token
   $GRACE = python -m scripts.login --user usr-grace --print-access-token

2. Omar, read-only, tries to add a message. Sam tries Bramble’s ticket. Somebody sends a broken token:

   curl -X POST -H "Authorization: Bearer $OMAR" -H "Content-Type: application/json" -d '{"author": "agent", "body": "Refund approved."}' http://127.0.0.1:8000/v1/tenants/larkfield/tickets/T-30002/messages
   curl -H "Authorization: Bearer $SAM" http://127.0.0.1:8000/v1/tenants/bramble/tickets/T-40001
   curl -H "Authorization: Bearer abc.def" http://127.0.0.1:8000/v1/tenants/larkfield/tickets

   {"error":{"code":"forbidden","message":"Your role may not do this (read_only may not ticket.write).","request_id":"98e78665f77d","fields":[]}}
   {"error":{"code":"not_found","message":"Not found.","request_id":"e0fd09350d86","fields":[]}}
   {"error":{"code":"invalid_token","message":"The token is not a JWT.","request_id":"1c5c42146725","fields":[]}}

   The API refuses all three: 403, 404 and 401. Your request IDs differ.

   In PowerShell, put the JSON body in a file, msg.json, and use -d "@msg.json".

3. Grace invites a new person:

   curl -X POST -H "Authorization: Bearer $GRACE" -H "Content-Type: application/json" -d '{"email": "dana@larkfield.example", "role": "staff"}' http://127.0.0.1:8000/v1/tenants/larkfield/invitations

4. Look at the audit table in psql:

   SELECT event_id, actor_id, tenant_id, action, result, reason FROM audit_events ORDER BY event_id;

    event_id | actor_id  | tenant_id |       action       | result | reason
   ----------+-----------+-----------+--------------------+--------+--------
           1 | usr-grace | larkfield | invitation.created | done   |
   (1 row)

The table knows that Grace invited somebody. It knows nothing about Omar, Sam or the broken token. The API refused all three correctly, and then forgot them. If Omar tries every ticket, every night, nobody can see it.

The idea: an audit event for every decision

An audit event is a record of one decision: who acted (the actor), in which organization, which action, on what (the target), and the result. An audit trail is the list of these events. It answers two questions for later: “who did what?” and “who tried what?”. The second question needs the refusals.

Three rules make a trail worth having:

1. Every decision, also the refusals: a refused token, a person without membership, a role that is too small.
2. Never a secret. No token, no cookie, no client secret, no invitation code, no password and no message text. The trail must be safe to show to an owner or an auditor.
3. Append-only. Nobody changes or deletes an event, not even the API. A trail that can be edited proves nothing.

One more question decides where an event belongs. Sam tried Bramble’s ticket. Should Ines, Bramble’s owner, see Sam’s attempt in her shop’s audit? That would tell her that a Larkfield person called usr-sam exists. The outsider’s 404 hides that the record exists; the trail must not leak it either. So an outsider’s attempt is recorded without an organization: the platform team sees it, no shop does. A member’s refusal (403) belongs to that organization: the member already knows it.

Each check that refuses writes one event, and each event says only what that check knew. The lookup's 404 writes no event.

Text description of the diagram

A request passes four checks in order. If the token or session fails, the API answers 401 and records token.check as denied. The reason is the check’s name; there is no actor, no organization, and never the token. If the person is not a member, the API answers 404 and records the person, no organization, and the path in the details. If the role or scope is too small, the API answers 403 and records the person and the organization. If the record is not in the organization, the API answers 404 “does not exist”, and writes no event, only a log line. If everything passes, the API writes the change and its event in one transaction.

Look at check 4. A member who asks for another shop’s ticket ID through their own path passes the first three checks, and the lookup finds nothing. The API cannot tell a guessed ID from a mistyped one, so this course does not record it as an attempt. That is a choice with a cost, and you see it in the results below.

Record every refusal

The change is small: three places in the API call audit.record() when they refuse. In ticket_api/tenancy.py, the check of every protected route now does this:

ticket_api/tenancy.py (part)

    if not decision.allowed:
        # Module 6: who TRIED is as important as who did. The tenant is recorded only for a
        # member (403): an outsider's attempt is not written into another organization's log.
        if grant is None:
            audit.record(
                db,
                action=action,
                result="denied",
                actor_id=identity.user_id,
                actor_kind=identity.kind,
                tenant_id=tenant_id if decision.status == 403 else None,
                target=f"{request.method} {request.url.path}",
                reason=decision.code,
                request_id=getattr(request.state, "request_id", None),
                details={"rule": decision.rule, "tenant_in_path": tenant_id},
            )
        raise AccessDenied(decision, tenant_id, action)

grant is None skips a support grant: support access already records every use, allowed or denied, since Privileged operations. audit.record() writes the event in its own short transaction, because a refused request commits nothing else. If the database cannot write the event, the API still answers, and logs the failure.

In ticket_api/errors.py, a refused token becomes the event token.check, with the name of the check as the reason (expired, wrong_audience…), and no token. In ticket_api/login.py, a sign-in, a sign-out and a sign-in with an unknown state get their events too.

1. Copy the files of this lesson from the snapshot end-of-m06: the three changed modules, the audit script and its test.

   macOS / Linux

   S=../nextia-learning-labs/C23/snapshots/end-of-m06
   for f in ticket_api/tenancy.py ticket_api/errors.py ticket_api/login.py \
       scripts/audit.py tests/test_auditability.py; do
     cp "$S/$f" "$f"
   done

   Windows (PowerShell)

   $S = "..\nextia-learning-labs\C23\snapshots\end-of-m06"
   $files = 'ticket_api\tenancy.py', 'ticket_api\errors.py', 'ticket_api\login.py',
     'scripts\audit.py', 'tests\test_auditability.py'
   foreach ($f in $files) { Copy-Item "$S\$f" $f }

   fastapi dev sees the changed files and restarts the API.

2. Load the data again, and send the same four requests as before (step 2 and step 3 of the first list). The answers do not change: 403, 404, 401 and 201.

3. Read the trail with the new script:

   python -m scripts.audit

   07:12:20 usr-omar     larkfield  ticket.write             denied  role_cannot  POST /v1/tenants/larkfield/tickets/T-30002/messages
   07:12:20 usr-sam      -          ticket.read              denied  not_a_member  GET /v1/tenants/bramble/tickets/T-40001
   07:12:20 (no identity) -          token.check              denied  malformed  GET /v1/tenants/larkfield/tickets
   07:12:20 usr-grace    larkfield  invitation.created       done      dana@larkfield.example

   Each line has the time (UTC), the actor, the organization (- for none), the action, the result, the reason and the target. The table also keeps the request ID that the API sends back in every error body. When Omar reports “I got an error”, support can find his exact event.

Read the four lines with Kwame’s three questions. Omar: who is known, the organization is Larkfield, the action ticket.write was denied because the role cannot. Sam: who is known, but no organization: his attempt is not in Bramble’s trail. The broken token: no actor, because a token that fails its check proves nobody. Grace: a change, done.

Filter the trail by organization and result, as Larkfield’s owner would:

python -m scripts.audit --tenant larkfield --result denied

07:12:20 usr-omar     larkfield  ticket.write             denied  role_cannot  POST /v1/tenants/larkfield/tickets/T-30002/messages

Who tried? Run the attempts again

The ten made-up attempts of Authorization in the API still fail, and now they leave a trail:

python -m scripts.load --reset >/dev/null
python -m scripts.attempts 2>/dev/null | tail -1
python -m scripts.audit --result denied

0 of 10 attempts got what they should not get.
07:12:22 usr-sam      -          ticket.read              denied  not_a_member  GET /v1/tenants/bramble/tickets/T-40001
07:12:22 usr-omar     larkfield  ticket.write             denied  role_cannot  POST /v1/tenants/larkfield/tickets/T-30002/messages
07:12:22 usr-tomas    -          ticket.read              denied  not_a_member  GET /v1/tenants/larkfield/tickets/T-30002
07:12:22 usr-camille  bramble    ticket.write             denied  role_cannot  POST /v1/tenants/bramble/tickets/T-40001/messages

In PowerShell, write | Out-Null instead of >/dev/null, 2>$null instead of 2>/dev/null, and Select-Object -Last 1 instead of tail -1. Ten attempts, four events. Count which attempts are missing:

Attempt	Answer	In the trail?	Why
1. Sam, Bramble’s ID through Larkfield’s path	404 “does not exist”	no	check 4: the lookup found nothing
2. Sam, through Bramble’s path	404	yes, no organization	not a member
3, 6. Ines lists and searches her own shop	200	no	allowed reads are not recorded
4. Omar adds a message	403	yes, Larkfield	role too small
5. Tomás reads T-30002	404	yes, no organization	not a member
7. Sam’s forged X-Tenant-ID header	200, Larkfield only	no	the API ignores the header
8. Camille writes at Bramble Books	403	yes, Bramble	role too small
9. Sam, a Bramble attachment ID	404 “does not exist”	no	check 4
10. Omar’s search for staff documents	200, public only	no	an allowed read

The trail shows four attempts. Attempts 1 and 9 are guessed IDs of the other shop: the most typical object-level attack, and the trail cannot see them. They are only in the API’s log, as 404 lines. If you must detect ID guessing, count a person’s 404 answers in the log over time, and alert on many. This course does not build that alert.

Whose trail is it? A member's refusal goes to the organization; an outsider's attempt and a refused token go to the platform team only.

Text description of the diagram

The events of this lesson fall into three groups. Larkfield’s owner, Grace, sees Omar’s denied write and her own invitation. Bramble’s owner, Ines, sees Camille’s denied write. Some events have no organization: Sam’s and Tomás’s attempts as outsiders, and the broken token. Only the platform team sees them, with the path in the details.

Check it through the API, as the owners see it:

curl -H "Authorization: Bearer $GRACE" http://127.0.0.1:8000/v1/tenants/larkfield/audit

{"items":[{"event_id":2,"at":"2026-10-10T07:12:22.220942Z","actor_id":"usr-omar","actor_kind":"user",
  "tenant_id":"larkfield","action":"ticket.write","target":"POST /v1/tenants/larkfield/tickets/T-30002/messages",
  "result":"denied","reason":"role_cannot","request_id":"9e9a040b5bfa",
  "details":{"rule":"read_only may not ticket.write","tenant_in_path":"larkfield"}}]}

The answer is broken into lines here. Grace sees one event: Omar’s. Tomás’s attempt at Larkfield is not in her trail, because he is not a member. Now Ines reads Bramble’s trail. Get her token first, as in step 1:

INES=$(python -m scripts.login --user usr-ines --print-access-token)
curl -H "Authorization: Bearer $INES" http://127.0.0.1:8000/v1/tenants/bramble/audit

In PowerShell: $INES = python -m scripts.login --user usr-ines --print-access-token, then curl.exe as before. Her answer has one event, Camille’s denied write (event_id 4, reason role_cannot), and no line of Sam’s.

Predict before you run it: Sam, staff at Larkfield, asks for Larkfield’s trail. Which status, and which event?

curl -H "Authorization: Bearer $SAM" http://127.0.0.1:8000/v1/tenants/larkfield/audit
python -m scripts.audit --actor usr-sam

{"error":{"code":"forbidden","message":"Your role may not do this (staff may not audit.read).","request_id":"6a48c96f8169","fields":[]}}
07:12:22 usr-sam      -          ticket.read              denied  not_a_member  GET /v1/tenants/bramble/tickets/T-40001
07:12:22 usr-sam      larkfield  audit.read               denied  role_cannot  GET /v1/tenants/larkfield/audit

Only an owner may read the trail, so Sam gets 403. His new event has Larkfield as its organization, because he is a member there. His earlier attempt at Bramble Books has none.

Never a secret

Kwame checks the second rule: no secret in the trail. scripts.audit --check reads every event and looks for a JWT, the word Bearer and each secret of your .env:

python -m scripts.audit --check

Checked 5 audit events: 0 with a token or a secret.

Now a constructed mistake. Imagine a developer who records the request’s Authorization header “for debugging”. Insert such an event by hand in psql, with a made-up token:

INSERT INTO audit_events (actor_id, actor_kind, action, result, details)
VALUES ('usr-sam', 'user', 'debug.request', 'done',
        '{"header": "Bearer eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJ1c3Itc2FtIn0.c2lnbmF0dXJl"}');

python -m scripts.audit --check

  event 6 (debug.request) has a JWT, Bearer
Checked 6 audit events: 1 with a token or a secret.

The script exits with code 1, so a scheduled job or a CI step can fail on it. Now try to remove the bad event:

DELETE FROM audit_events WHERE event_id = 6;

ERROR:  audit events cannot be changed or deleted
CONTEXT:  PL/pgSQL function audit_events_append_only() line 3 at RAISE

Your event ID can differ. This is the third rule at work, and it cuts both ways. Nobody can hide an event, and nobody can easily remove a leaked secret either. A token in an append-only trail stays there. That is why audit.write(), which audit.record() also calls, cleans details before it writes: keys that name a secret are dropped, and any value that looks like a JWT becomes [removed]. The constructed event went around that code, straight into the table. On your practice database, python -m scripts.load --reset empties the table: TRUNCATE does not fire the trigger, so a database owner can still clear everything. On a real system, a leaked token must be revoked at once, because it stays in the trail.

1. Clean up: python -m scripts.load --reset, then python -m scripts.audit --check. Expect Checked 0 audit events: 0 with a token or a secret.

2. Run the new tests:

   python -m pytest tests/test_auditability.py

   tests/test_auditability.py .....                                         [100%]
   
   ============================== 5 passed in 1.11s ===============================

   The five tests check a member’s refusal, an outsider’s attempt without an organization, and a refused token without the token. One test runs a whole sign-in, invitation and sign-out. Then it looks for a token, a cookie, a client secret or an invitation code in the trail and the log, and finds none. The last one checks the append-only trigger.

3. Run every test: python -m pytest. Expect 452 passed, 1 deselected.

What a trail costs

Each refusal now writes one row, in its own transaction: one more database round trip on a path that was cheap before. A person or a script that sends many bad requests makes many rows. The trail grows forever, because nothing deletes it. A real system needs a retention rule (for example, 13 months) and a way to archive. That is an open choice for the handoff. Allowed reads are not recorded here: a “who read this ticket?” question would need many more rows. Record allowed reads only where a law or a contract asks for it, for example for support access.

Common mistakes

Common mistake

The script cannot find the trail

Symptom: python -m scripts.audit prints nothing, but you just sent refused requests.

Cause: the API and the script use different databases, or you ran python -m scripts.load --reset after the requests. Reset empties the trail.

Fix: check that .env has the same DATABASE_URL for both, and send the requests after the reset.

Your turn: the totals

Run the attempts again after a reset, then send Sam’s request for Larkfield’s trail once more. Before you run python -m scripts.audit --summary, write down the lines that you expect: each action, its result and the count.

Show the answer

  audit.read                   denied       1
  ticket.read                  denied       2
  ticket.write                 denied       2

Two outsiders read (Sam at Bramble Books, Tomás at Larkfield), two read-only members wrote (Omar, Camille), and Sam asked for the trail. The six allowed attempts and the two guessed IDs make no line.

See it end to endAudit access control in an open-source multi-tenant appCase study

The Gitea case study reads a real app’s audit log: it is off by default, and it records changes but not refusals.

Check your understanding

Sam (no membership at Bramble Books) asks for Bramble's ticket T-40001. Omar (read-only at Larkfield) tries to add a message at Larkfield. Which event has an organization, and why does the other one not?

Show the answer

Omar’s event has tenant larkfield. He is a member, the 403 tells him nothing new, and Larkfield’s owner should see what her members try. Sam’s event has no organization (NULL), and the path is in its details. In Bramble’s trail, it would tell Bramble’s owner that a Larkfield person exists, which the 404 hides. The platform team sees it.

Check your understanding

Name three things that must never be in an audit event, and say what the event keeps instead.

Show the answer

Never a token, a cookie, a client secret, an invitation code, a password or a message’s text. Instead the event keeps the actor’s user ID, the organization, the action and the target (method and path). It also keeps the result, the reason (for example expired or role_cannot) and the request ID.

Check your understanding

Predict: Sam guesses 50 Bramble ticket IDs through his own Larkfield path. How many audit events does it make, and where can you see the attempts?

Show the answer

None. Sam passes the identity, membership and role checks at Larkfield, and each lookup answers 404 “does not exist”, which is not recorded. The 50 attempts are only in the API’s log as 404 lines. To see them, count 404 answers per person in the log.

Check your understanding

Practical check: after the attempts, what do python -m scripts.audit --summary and --check print?

Show the answer

--summary prints one line per action and result: audit.read denied 1, ticket.read denied 2, ticket.write denied 2 after the attempts and Sam’s request. --check prints Checked 5 audit events: 0 with a token or a secret. and exits with 0. With a token in an event, it names the event and exits with 1.

Recap

• An audit event records one decision: actor, organization, action, target, result, reason, request ID. The audit trail answers “who did what?” and “who tried?”.
• Every refusal now leaves an event: token.check (no actor), not_a_member (no organization, the path in details), role_cannot (the organization).
• An outsider’s attempt never goes into another organization’s trail. A guessed ID through one’s own path makes no event, only a 404 log line.
• Never a secret: audit.write() cleans details, and python -m scripts.audit --check looks for tokens and .env secrets. The trail is append-only, so a leaked token stays there.
• Next: hand the design over, with the evidence.

Next: Handoff