Skip to content
Français

Make it tamper-evident and controlled

  • Lesson 21
  • 45 min
  • Module 6: Audit trails

Where we are

In the last lesson, the workflow began to write an audit event for every decision and every change, with fixed fields and no text. But the trail is a plain file: anybody who can edit it can change 9.95 into 0.95, and nobody would know. In this lesson, we make every change visible with a hash chain, send the events through the Collector to a store that only appends, and decide who may read, keep and delete them. We start with a bound notebook.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Make the audit trail tamper-evident: chain each record to the one before with a hash, keep the last hash apart, and verify. Send the audit events through the Collector to a store that only appends. Give each reader the rights that their job needs, and handle a deletion request without breaking the chain.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

What you need on your computer

  • The project as the last lesson left it: 103 tests pass.
  • otelcol-contrib 0.162.0 in the folder otelcol next to the project, as in The Collector.
  • Three terminals in observable-workflow. The one that runs the workflow has the virtual environment active.

Tested on macOS (Apple silicon) with Python 3.12.13 and otelcol-contrib 0.162.0. The Windows and Linux commands are written, not run.

A laboratory keeps its results in a bound notebook. The pages are numbered and sewn in. Nobody may tear a page out: the missing number would show it.

But a person could still change a number on page 40 with a careful pen. So the laboratory adds two habits. At the top of each new page, the scientist copies a short mark of the page before. And each evening, a colleague writes the last page number and its mark into her own diary, which the scientist never touches.

Now a changed page 40 no longer matches the mark on page 41. A torn-out last page no longer matches the colleague’s diary. Nobody can stop a change, but nobody can hide one.

The problem: a plain file forgets nothing and proves nothing

Section titled “The problem: a plain file forgets nothing and proves nothing”

The audit store is a text file, work/audit/audit.jsonl, one record per line. Look at record 673 of the replayed week, the second refund:

Terminal window
sed -n 673p data/telemetry/week/audit.jsonl | cut -c1-400
{"event":{"action":{"attempt":2,"effect":{"amount":9.95,"order_id":"LK-793996","reason_code":"RFD-LATE","tool":"request_refund"},"idempotency_key":"797fbb5bbb9f8a7e-retry1","operation_id":"797fbb5bbb9f8a7e","outcome":"done","tool":"request_refund"},"actor":{"id":"support-workflow","kind":"system","role":"executor"},"at":"2026-10-15T13:17:22+00:00","authority":{"approval_event_id":"b5188674-35aa-4e

Suppose somebody wants the double refund to look smaller. They open the file and change 9.95 into 0.95. The line is still valid JSON. Every tool still reads it. Without something more, no reader could tell that the record changed. An audit trail that anybody can change in silence proves nothing.

A hash is a short fingerprint of some data. The project uses SHA-256: from any input, it makes 64 hexadecimal characters. The same input always gives the same hash. A change of one character gives a completely different hash. And nobody can go back from a hash to the input, or make a second input with the same hash.

Try it with the two amounts:

Terminal window
python -c "import hashlib; print(hashlib.sha256(b'refund 9.95').hexdigest()); print(hashlib.sha256(b'refund 0.95').hexdigest())"
7c578ae242639f8091e878e6c58fb79d112fe5cf4f96416b5b74ef019eb96516
c3abdfc44cd6acbd25202002aae4cb349e4e4ff18b38756cb6509d35125380e8

One digit changed, and nothing in the two hashes looks alike.

A hash of each record alone is not enough: a person who changes a record can compute its new hash too. So each record also stores the hash of the record before it, as the notebook copies the mark of the page before. That is a hash chain:

  • seq is the record’s number: 1, 2, 3 …
  • prev is the hash of the record before. The first record’s prev is 64 zeros.
  • hash is SHA-256 of seq, prev and the event, written in one fixed way (record_hash() in auditstore/store.py).

The store also writes an anchor, a small file with the number of records and the last hash, after every append. That is the colleague’s diary. obs audit verify computes every hash again from the first record, checks every prev, and compares the end with the anchor.

The store has no update and no delete: append() is the only way in. A store like this is append-only. With the chain, it is tamper-evident: it cannot stop a change, but it shows one.

Worked example: three records, four ways to cheat

Section titled “Worked example: three records, four ways to cheat”

Create tiny_chain.py in the project folder. It makes a store of three records in work/tiny/, prints the chain, then changes record 2 in four ways, each on a fresh store:

The text of tiny_chain.py
"""A tiny hash chain: three audit events in a store of their own, then four ways to change record 2.
python tiny_chain.py (in the project folder, the virtual environment active)
"""
import json
from pathlib import Path
from auditstore.store import AuditStore, record_hash
path = Path("work/tiny/audit.jsonl")
def fresh() -> AuditStore:
for f in path.parent.glob("audit*"):
f.unlink() # start again each time
store = AuditStore(path)
for n, kind in enumerate(["proposal", "approval", "write"], 1):
store.append({"schema": "larkfield.audit/1", "event_id": f"e{n}", "type": kind,
"at": "2026-10-15T13:17:12+00:00", "actor": {"kind": "system", "id": "demo"},
"run_id": "R-demo", "ticket_id": "T-demo", "trace_id": "", "span_id": "",
"subject": {"order_id": "LK-793996"}, "action": {"amount": 9.95},
"authority": {}, "evidence": []})
return store
def change(store: AuditStore, rehash: str) -> None:
"""Change the amount of record 2 by hand. rehash: '' (no), 'own' (its hash), 'all' (every hash after it)."""
recs = store.records()
recs[1]["event"]["action"]["amount"] = 0.95
for i, r in enumerate(recs):
if (rehash == "own" and i == 1) or (rehash == "all" and i >= 1):
r["prev"] = recs[i - 1]["hash"]
r["hash"] = record_hash(r["seq"], r["prev"], r["event"])
path.write_text("".join(json.dumps(r, sort_keys=True, separators=(",", ":")) + "\n" for r in recs))
store = fresh()
for r in store.records():
print(f'record {r["seq"]} prev {r["prev"][:12]} amount {r["event"]["action"]["amount"]} hash {r["hash"][:12]}')
print("verify:", store.verify()["problems"] or "ok")
for label, rehash, anchor_too in [("A. change the amount only", "", False),
("B. change it and compute its own hash again", "own", False),
("C. change it and compute every hash after it again", "all", False),
("D. as C, and write the anchor again too", "all", True)]:
store = fresh()
change(store, rehash)
if anchor_too:
last = store.records()[-1]
store.anchor.write_text(json.dumps({"count": last["seq"], "head": last["hash"]}) + "\n")
print(f"{label}: {store.verify()['problems'] or 'ok'}")

Run it with python tiny_chain.py:

record 1 prev 000000000000 amount 9.95 hash ab37a2f0d1b3
record 2 prev ab37a2f0d1b3 amount 9.95 hash c4d81416b363
record 3 prev c4d81416b363 amount 9.95 hash 999b64682730
verify: ok
A. change the amount only: ['record 2: its content does not match its hash: the record was changed']
B. change it and compute its own hash again: ['record 3: its link to record 2 does not match: an earlier record was changed or removed']
C. change it and compute every hash after it again: ["the store has 3 records, as the anchor says, but its last hash is not the anchor's head: a record was changed and the hashes after it computed again"]
D. as C, and write the anchor again too: ok

Your hashes are the same as these: the records have fixed contents. Follow the chain in the first three lines: record 2’s prev is record 1’s hash, and record 3’s prev is record 2’s hash.

A table of four rows. A, change the amount only: verify says record 2: its content does not match its hash: the record was changed; caught by its own hash. B, change it and compute its own hash again: record 3: its link to record 2 does not match: an earlier record was changed or removed; caught by the next record's prev. C, change it and compute every hash after it again: the store has 3 records, as the anchor says, but its last hash is not the anchor's head: a record was changed and the hashes after it computed again; caught by the anchor's head. D, as before, and write the anchor again too: the audit trail is intact; caught by nothing in the file, only by a copy of the anchor elsewhere.
Four ways to change record 2 of a three-record chain, and what verify says. Each step is more work for the person who changes it; only the last one hides the change, and only if nobody else holds the anchor.
Text description of the diagram

A changed content breaks its own hash. A new hash breaks the next record’s prev. New hashes for every later record break the anchor’s head. Only a person who can also rewrite the anchor hides the change, so the anchor needs a copy that this person cannot reach.

In case C, the count is right, but the last hash is not the one that the anchor kept. So the verifier knows that the chain was computed again after a change.

Case D is the real lesson. A person who can write both files can rewrite the chain and the anchor, and the chain proves nothing. So in production, a copy of the anchor goes to another system with other access rights. That can be a write-once storage bucket, another team’s database, or a daily message to Mei.

Now the same on the week’s real trail. Change 9.95 into 0.95 in record 673, in a copy, and verify the copy:

Terminal window
mkdir -p work
cp data/telemetry/week/audit.jsonl work/t.jsonl
cp data/telemetry/week/audit.anchor.json work/t.anchor.json
sed -i.bak '673s/9.95/0.95/' work/t.jsonl
python -m obs audit verify --audit work/t.jsonl
The audit trail does NOT verify (673 records read):
record 673: its content does not match its hash: the record was changed

The store’s file name decides the anchor’s name: t.jsonl goes with t.anchor.json. That is why you copy both.

Four record boxes in a row, joined by arrows from each stored hash to the next record's prev. Record 671, approval by usr-sam: prev 8dd09d48, stored hash ac63a5ff. Record 672, write attempt 1, timeout: prev ac63a5ff, refund 9.95, stored hash 6fbf9d7d. Record 673, write attempt 2, done, drawn in red: prev 6fbf9d7d, refund 0.95, was 9.95, stored hash ed0673cf, computed again a5192794. Record 674, run_closed, resolved: prev ed0673cf, stored hash 9051db26. Under record 674, the anchor file: count 1331, head e378b7d0, the last hash of the whole store, kept apart. At the bottom, the command python -m obs audit verify --audit work/t.jsonl and its answer: the audit trail does NOT verify (673 records read): record 673: its content does not match its hash: the record was changed.
Records 671 to 674 of the replayed week with record 673 changed by hand: the hash computed again no longer matches the stored one, and verify names the record.
Text description of the diagram

Each stored hash is the next record’s prev. With the amount of 673 changed to 0.95, its hash computed again is a5192794, not the stored ed0673cf. The verifier stops at this first problem and names record 673. The anchor holds the count, 1331, and the store’s last hash.

A chain checks each record against the one before. Remove the last record, and every remaining link is still right. Only the anchor sees it:

Terminal window
head -n 1330 data/telemetry/week/audit.jsonl > work/t.jsonl
python -m obs audit verify --audit work/t.jsonl
The audit trail does NOT verify (1330 records read):
the store has 1330 records, the anchor says 1331: records were removed at the end or added without the store

The anchor is one line: {"count": 1331, "head": "e378b7d00139f1c356862b87968f9e407ce33a9e91e7ddfd3b0d412cbb9c0dec"}.

The chain shows changes. Access rights make changes hard in the first place:

Top row, the route: the workflow sends one audit event per decision or change with OBS_EXPORT=otlp, over OTLP on port 4318, to the Collector; its routing connector asks whether the event has audit.type; if yes it goes only to the logs/audit pipeline, with no sampling and not to Loki; its otlp_http/audit exporter sends it to the door on port 8313, which accepts only POST /v1/logs, checks the schema, and has no read, change or delete; the door appends to audit.jsonl with seq, prev, the event and its hash, plus the anchor file. Middle row, what the door answers: 200, stored in the chain; 200 and quarantine, a bad event goes to audit.rejected.jsonl for a person; 503, cannot write now, so the Collector sends it again; 400, the Collector drops the batch for good, which the first door did. Bottom, who may do what: the workflow and the Collector send events to the door and nothing else; Grace and the auditors read the store with obs audit verify, show and story; Mei, the store's owner, runs the store, the keys file and the retention, and cannot edit records; another system in production keeps a copy of the anchor that the application cannot change.
The route of an audit event through the Collector to the store's door, what the door answers, and the separate access rights around the store.
Text description of the diagram

The routing connector sends each event with audit.type to its own pipeline, with no sampling, and its exporter posts it to the door. The door only appends. It answers 200 for a stored event and 200 with a quarantine for a bad one. It answers 503 when it cannot write, so that the Collector retries. It never answers 400, because the Collector drops a batch that gets a 400. The workflow only sends; Grace and the auditors only read; Mei runs the store but cannot edit it; a copy of the anchor lives elsewhere.

The door’s answers come from a real finding. The course team’s first door answered HTTP 400 to an event that failed the schema. That is the usual answer for bad input. But the Collector’s otlp_http exporter treats a 400 as permanent and does not retry. With a test door that answers 400, the Collector logged this, and the event was gone:

error internal/queue_sender.go:57 Exporting failed. Dropping data.
"error": "not retryable error: Permanent error: rpc error: code = InvalidArgument desc = error exporting items, request to http://127.0.0.1:8313/v1/logs responded with HTTP Status Code 400
"dropped_items": 1

So the door now keeps a bad event in audit.rejected.jsonl, for a person, and answers 200. It answers 503 only when it cannot write, because the Collector retries a 503. The comments in do_POST (auditstore/server.py) explain each answer.

Guided practice: through the Collector to the store

Section titled “Guided practice: through the Collector to the store”
  1. Copy the store’s door, its test and the module’s README:

    Terminal window
    S=../nextia-learning-labs/C31/snapshots/end-of-m06
    cp $S/auditstore/server.py $S/auditstore/__main__.py auditstore/
    cp $S/tests/test_auditstore_server.py tests/
    cp $S/README.md .
  2. Run the tests: python -m pytest -q. The last line says 104 passed, 1 skipped, 1 deselected. Your project is now the snapshot end-of-m06.

  3. Delete work/, so that the new trail starts at record 1.

  4. In the first terminal, start the door:

    Terminal window
    python -m auditstore serve
    Audit store: http://127.0.0.1:8313/v1/logs -> …/observable-workflow/work/audit/audit.jsonl
  5. In the second terminal, start the Collector, as in The Collector:

    Terminal window
    mkdir -p work/collector-out
    OUT_DIR=work/collector-out AUDIT_STORE_URL=http://127.0.0.1:8313 ../otelcol/otelcol-contrib --config collector/standalone.yaml

    Wait for the line Everything is ready. Begin running and processing data.

  6. In the third terminal, replay the double refund through the Collector. In PowerShell, set $env:OBS_EXPORT = "otlp" first, and leave out OBS_EXPORT=otlp in front of the commands.

    Terminal window
    OBS_EXPORT=otlp python -m resolver run T-90104 --variant agent_structured --model chat-strong
    OBS_EXPORT=otlp python -m resolver approve T-90104-agent_structured-chat-strong-1 --as agent --fault request_refund:timeout_after_write --naive-retry
  7. Verify the trail that the door wrote, and read it:

    Terminal window
    python -m obs audit verify
    python -m obs audit story T-90104
    The audit trail is intact: 6 records, the chain and the anchor match (head d9fb6017bf615484).
    Audit story of ticket T-90104, run T-90104-agent_structured-chat-strong-1: 6 records (the trail verifies).
    1. … The AI agent support-workflow (chat-strong, agent_structured) read get_order (LK-793996): ok.
    2. … The AI agent support-workflow (chat-strong, agent_structured) proposed: resolve (rule W2): refund 9.95 (RFD-LATE) on LK-793996. It needs a support agent or above to approve. Evidence it read: order:LK-793996.
    3. … Sam (usr-sam, support agent) approved the proposal of record 2 (limit 100).
    4. … The workflow (support-workflow) sent refund 9.95 (RFD-LATE) on LK-793996: TIMEOUT (attempt 1, key 797fbb5bbb9f8a7e). Authority: the approval of record 3.
    5. … The workflow (support-workflow) sent refund 9.95 (RFD-LATE) on LK-793996: done (attempt 2, key 797fbb5bbb9f8a7e-retry1). Authority: the approval of record 3.
    6. … The ticket was closed: resolved (writes done: 1, reply: sent).
    Changes asked of the systems: 2 write call(s) after 1 approval(s).
    refund 9.95 (RFD-LATE) on LK-793996: 2 call(s) that may have changed it (timeout, done), 2 different key(s)

    All six records arrived, and the workflow did not write any of them itself: in OTLP mode, the door did. Your head differs, because your event IDs and times are new.

  8. Check that no audit event went to the logs output, Loki’s file here: grep -c larkfield.audit work/collector-out/logs.jsonl (PowerShell: (Select-String -Path work\collector-out\logs.jsonl -Pattern larkfield.audit).Count). It prints 0.

Your order may differ. In two of the course team’s three runs, the proposal arrived before the read and became record 1. The Collector sends requests in parallel, so seq is the order of arrival; at is the time of the event.

Now the traces. Wait 15 seconds, then run python -m obs runs --data work/collector-out. The course team got this:

run ticket started outcome model tools appr cost US$ secs model flags
T-90104-agent_structured-chat-strong-1 … resolved 0 0 1 0.00000 0.0 duplicate_write
1 run(s), 1 flagged

No model calls: tail sampling dropped trace 1 and kept trace 2, which has an approval and writes. The trace store has half of the story; the trail has all of it.

Do it the way a person with an editor would.

  1. Leave the door and the Collector running.
  2. Open work/audit/audit.jsonl in VS Code. Find the line with retry1: the second refund.
  3. Change "amount":9.95 into "amount":0.95 on that line, and save.
  4. Predict what verify says. Then run python -m obs audit verify.
  5. Undo the change in the editor (Ctrl+Z, or Cmd+Z on a Mac), save, and verify again.
The course team’s result
The audit trail does NOT verify (5 records read):
record 5: its content does not match its hash: the record was changed

After the undo:

The audit trail is intact: 6 records, the chain and the anchor match (head d9fb6017bf615484).

The verifier stops at the first problem, here record 5, and says how many records it read. Your record number is the line where retry1 is.

Send an event that fails the schema straight to the door. Create bad-event.json in the project folder:

{"resourceLogs":[{"scopeLogs":[{"logRecords":[{"eventName":"larkfield.audit.approval","body":{"stringValue":"{\"schema\": \"larkfield.audit/1\", \"type\": \"approval\", \"note\": \"Sam said OK on the phone\"}"}}]}]}]}

Then post it (on Windows, type curl.exe instead of curl):

Terminal window
curl -s -X POST -H "Content-Type: application/json" --data @bad-event.json http://127.0.0.1:8313/v1/logs
{"partialSuccess": {}}

The door said yes, so a Collector would not drop the batch. The first terminal says 1 event(s) kept in audit.rejected.jsonl: they do not pass the schema, and that file holds the event and the reason. verify still says 6 records. The quarantine holds the text “Sam said OK on the phone”, so it needs the store’s access rights, and a person who empties it.

Stop the door and the Collector with Ctrl+C when you finish.

Common mistake

The door is down, or the Collector is not running

Symptom 1, the door is down: the workflow runs as usual, but verify shows no new record. The Collector’s terminal shows, every few seconds:

info internal/retry_sender.go:136 Exporting failed. Will retry the request after interval. "otelcol.component.id": "otlp_http/audit", … "error": "failed to make an HTTP request: Post \"http://127.0.0.1:8313/v1/logs\": dial tcp 127.0.0.1:8313: connect: connection refused"

Recover: start the door again. In the course team’s run, the 3 records of T-80008 arrived within 10 seconds: 9 records. The exporter’s max_elapsed_time: 0 means it never gives up, but it keeps waiting events in memory. If the Collector stops too, they are lost: in production, give the audit exporter a queue on disk.

Symptom 2, no Collector at all: with OBS_EXPORT=otlp and nothing on port 4318, the workflow prints Transient error … port=4318 … lines, then Failed to export logs batch due to timeout, max retries or shutdown. four times. It still ends with Reply sent. Ticket: resolved., and the trail has no record of that run.

Cause: in OTLP mode, the workflow hands its audit events to the OpenTelemetry exporter and does not wait for the store. When the exporter gives up, the events are gone. Only file mode stops the run when an event cannot be stored.

Recover: this is a gap in the course’s code, and in many real systems. Close it with an outbox: the workflow writes each audit event to a local file first, and a sender empties the file into the store. Or stop the writes when the audit route is down. And compare counts every day: runs closed in the metrics against run_closed records. A chain cannot show an event that never arrived.

Retention. The store has no delete, and Larkfield keeps the trail 7 years, an assumption. A common design starts a new file each year, with the old file’s last hash as the new file’s first prev. Then a whole year can be deleted, and the newer chain still verifies. The course’s store has one file: this is advice, not tested here.

A deletion request. A customer asks Larkfield to delete her data. The privacy law that applies may give her that right, for example the right to erasure in Article 17 of the EU’s GDPR. But the audit trail is append-only, and a removed record breaks the chain. The AI security course handles deletion across every copy in Minimize and retain deliberately. Here is the trail’s part.

The store never holds the customer ID. It holds a pseudonym: an HMAC, a keyed hash, of the customer ID with that customer’s own key, from audit.keys.json. A deletion request deletes the key. The records stay and the chain verifies, but nobody can compute the pseudonym from the customer ID again: crypto-shredding.

The customer of the double refund is C-61975. Try it on a copy of the week:

  1. Copy the week’s store, anchor and keys: cp data/telemetry/week/audit* work/ (PowerShell: Copy-Item data\telemetry\week\audit* work\).

  2. Forget the customer:

    Terminal window
    python -m obs audit forget C-61975 --audit work/audit.jsonl
    Deleted the key of customer C-61975: their audit events stay, but they can no longer be linked to the customer.
  3. Verify: python -m obs audit verify --audit work/audit.jsonl. It says The audit trail is intact: 1331 records, the chain and the anchor match (head e378b7d00139f1c3).

  4. Count her records: grep -c 'cust:6899a74add4687df' work/audit.jsonl (PowerShell: (Select-String -Path work\audit.jsonl -Pattern 'cust:6899a74add4687df').Count). It still prints 22: the records of the five runs of T-90104 in the week. The keys file no longer has C-61975.

obs audit story R-1014-040 still tells the whole story, because the story never needed her ID. Be honest about what this does not do:

  • The records still hold order IDs, such as LK-793996, which the order system can link to her. The deletion is complete only if the order system forgets her too.
  • The traces hold her ID in clear text: larkfield.customer.id = C-61975. In a trace store, it goes when the 14 days of retention pass.
  • Whether a pseudonym without its key counts as deleted is a legal question. The GDPR’s Recital 26 treats pseudonymised data as personal data when extra information can link it back. Ask a qualified person. The keys file is a control that makes an answer possible, not the answer.

When to use each control, and what it costs

Section titled “When to use each control, and what it costs”

Use the chain and the anchor for any trail that someone may want to change later. They cost one SHA-256 and one small file write per record. That is inside the 1.9 ms per run that the course team measured for all the telemetry. Keep a copy of the anchor elsewhere when the people who run the application could be tempted, or blamed. Without it, the chain shows careless edits, not a determined insider.

Separate access rights cost the most effort. They are still cheaper than explaining to an auditor why a developer could change a refund record. A ledger database or a write-once bucket gives the same guarantees as a product, with its own price and lock-in.

Check your understanding

Someone removes the last record of the audit store. Why does the hash chain alone not see it, and what does?

Show the answer

Each record is checked against the record before it. After the last record is removed, every remaining link is still correct, so the chain verifies. The anchor sees it: it holds the count and the last hash, and verify says ‘the store has 1330 records, the anchor says 1331’. The anchor must have a copy in another system, because a person who can rewrite the store and the anchor can hide any change.

Check your understanding

The store's door answers HTTP 200 for an event that fails the schema. Why not 400, the usual answer for bad input? Where does the bad event go?

Show the answer

The Collector’s otlp_http exporter treats a 400 as a permanent error: it logs ‘Exporting failed. Dropping data.’ and drops the whole batch, so good audit events in the same batch would be lost too. The door keeps the bad event in audit.rejected.jsonl with the reason, for a person, and answers 200. It answers 503 only when it cannot write, because the Collector retries a 503.

Check your understanding

Independent task: send the double refund through the Collector to the audit store, change one record by hand, and show that verification fails. Paste the two outputs of verify, and say which record the verifier names and why.

Show the answer

Through the door, verify says that the trail is intact, with 6 records. After the change of 9.95 into 0.95 on the line with retry1, verify names record 5. Its message is ‘its content does not match its hash: the record was changed’. The verifier computes each hash again from seq, prev and the event. The changed amount gives another hash, and it stops at that first problem.

Check your understanding

A customer asks Larkfield to delete her data. What does python -m obs audit forget do, why does the chain still verify, and what does it not reach?

Show the answer

It deletes the customer’s key. Her records stay unchanged, so the chain and the anchor verify, but nobody can compute her pseudonym from her ID again (crypto-shredding). It does not reach the order IDs in the records, which the order system can link to her. It does not reach the traces, which hold her ID for 14 days. Whether this counts as deleted is a legal question.

Recap

  • Each record holds the hash of the record before it; the anchor holds the count and the last hash. verify names the first record that does not match, and the anchor catches removed records at the end.
  • A copy of the anchor must live in another system, or a person who can rewrite both files hides any change.
  • Audit events go through the Collector’s own pipeline to a door that only appends. The door never answers 400; a bad event goes to a quarantine.
  • A chain cannot show an event that never arrived. In OTLP mode, a missing Collector loses audit events: compare counts, and use an outbox in production.
  • A deletion request deletes the customer’s key, not her records; order IDs and traces need their own deletion.

Your project now: the snapshot end-of-m06, with 104 tests that pass. The trail is complete, chained, verified and controlled. But a trail is only worth its answers. Next, you take the question that started this course: why was T-90104 refunded twice? You answer it from the trail, for Grace and for an auditor.

Next: Answer an audit question