A tribunal case about a chatbot's answer
- Case study 1
- 120 min
- Optional · 1 case study needed for the certificate
Where we are
In this course, we gave Larkfield's systems owners, policies, reviews and records. So far, every case was Larkfield's own, and made up. This case study tests those tools on a real public decision: a tribunal held an airline responsible for what its website chatbot told a customer. We find what was missing, and we write the governance review that would have prevented it.
Case study 1 of 3 · AI Governance: Policies, Risk and AccountabilityA tribunal case about a chatbot's answer
1. What governance is for
2. Laws and standards in plain words
3. Inventory, risk tiers and impact assessment
4. Policies people can follow
5. Controls, documentation and evidence
What you will be able to do
Test the whole course on a real decision. You read a public tribunal decision about a chatbot’s wrong answer. You find the missing owner, policy, review and record, and write the review that would have prevented it. Then you apply it to Larkfield.
The case folder has a README, a source card and a worksheet. You read the decision itself on CanLII, not in the folder: the course links to it and does not copy it.
Before you startWhat to know and to have ready
What you need to know
This case study uses the whole course. These lessons matter most:
- The accountable owner and the approver, from Who is accountable.
- The inventory row and the risk-tier method, from The AI inventory and Risk tiers and proportionate controls.
- A control with an owner and evidence, from From policy to control.
- The five questions after an incident, from Incidents, complaints and redress.
The page teaches one new idea: one source of truth for every answer about a policy.
What you need
- A browser and a document tool (a word processor, a text editor or a spreadsheet).
- The decision on CanLII, and the README, source card and worksheet of the case folder.
- No account, no installation and no money.
The situation
Section titled “The situation”Nadia, Larkfield’s head of customer operations, read about the case in the news. She asks Farida, Larkfield’s risk and compliance lead: “Our support systems answer customers too. Could this happen to us? What would have prevented it?” Larkfield, Nadia and her question are constructed for this course. The case is real.
Your objective is a two-page governance review, in the case folder’s worksheet. It says what happened, what was missing, which three controls would have prevented it, and what Larkfield should change. This page walks through the analysis with you. You write the review at the end, in your own words.
The source and how we use it
Section titled “The source and how we use it”The “data” of this case study is one document: Moffatt v. Air Canada, 2024 BCCRT 149, a decision of the Civil Resolution Tribunal of British Columbia, issued on 14 February 2024. Read it on CanLII (official source, checked 22 October 2026), or search for “2024 BCCRT 149” on the tribunal’s decisions site. The source card says what the decision is and how you may use it.
First checks
Section titled “First checks”Before you analyse a source, check what kind of source it is. Three facts change how much weight the decision can carry:
- It is a small claim. The tribunal decided it on the documents, with no oral hearing. Moffatt represented themself, and an employee represented the airline (paras 5 and 8).
- It decides one dispute. What it means for other cases, or for your organisation, is a question for a lawyer. This page draws governance lessons, not legal ones.
- The tribunal calls the applicant “Mr. Moffatt” and uses “they”. This page says “Moffatt” and “they”.
A published case comment adds context: Isabelle St-Hilaire, in the UBC Law Review (2025 CanLIIDocs 1963). Where this page uses it, it says commentary: one author’s view, not the tribunal’s finding.
What happened
Section titled “What happened”Read the decision once, quickly, from start to end. Then read only the paragraphs below again. They hold every fact that the review needs.
Text description of the diagram
On 11 November 2022, Moffatt’s grandmother died, and Moffatt asked the chatbot about bereavement fares (paras 13–14). The chatbot said to ask for the reduced rate after travel, within 90 days (para 15). Its own link went to a page that said the opposite (paras 16–17). Moffatt booked two flights and applied within the 90 days (paras 18, 20). The airline admitted “misleading words” (para 22). The tribunal rejected its defence (paras 27–29) and ordered it to pay 812.02 dollars (para 44).
In our words: Moffatt asked the airline’s website chatbot about fares for a family death. The chatbot said they could ask for the reduced fare after travel, “within 90 days of the date your ticket was issued” (para 15). The words “bereavement fares” in that answer linked to the airline’s own policy page. That page said the policy does not apply after travel (paras 16–17). Moffatt trusted the chatbot, flew, and applied in time. The airline did not give the reduced fare. In its answer, it admitted that the chatbot had used “misleading words” (para 22).
Look at the dashed line in the diagram. One website gave two answers to one question. That is the whole case in one picture.
What the tribunal decided, and why
Section titled “What the tribunal decided, and why”The airline’s main argument was that it could not be held liable for information from its agents or representatives, including a chatbot. The tribunal called this “a remarkable submission” (para 27). In plain words, its reasons were these:
- The company answers for its whole website, chatbot included. “It makes no difference whether the information comes from a static page or a chatbot.” (para 27)
- The company did not take care. It “did not take reasonable care to ensure its chatbot was accurate” (para 28). It also did not explain why a customer should check one part of its website against another.
- Moffatt’s trust was reasonable (para 29).
- The contract terms did not help the airline. It relied on its tariff, but did not file it, so it did not prove that defence (para 31).
The tribunal then counted the money.
Text description of the chart
A column chart with four columns, in Canadian dollars. The tribunal uses 1630.36 dollars as what Moffatt paid (paras 34, 40). Moffatt claimed 880.00 dollars. The tribunal awarded 650.88 dollars in damages. The total order was 812.02 dollars: the damages, 36.14 dollars of interest and 125.00 dollars of tribunal fees.
The tribunal uses 1630.36 dollars as the amount paid (paras 34, 40). The course’s own check: the two fares in para 18 add to 1640.36 dollars.
What this tells us. The money is small: 812.02 dollars in all, to be paid within 14 days (para 44). The airline had also offered a 200.00 dollars coupon, which Moffatt did not accept (para 41). As with Larkfield’s AI bill, the risk is not in the amount. It is in what the case showed in public. Nobody checked what the chatbot promised, and the company’s defence was that the chatbot answered for itself.
Governance starts from the opposite position. The organisation answers for every system it deploys, whoever built it. You met this in Governance that helps teams ship, when Tomás said the vendor was responsible.
The new idea: one source of truth
Section titled “The new idea: one source of truth”Think of a small shop. The owner keeps one price list. The price tags, the website and the staff all copy from it. One day the owner raises a price and changes the list, but forgets the tags on the shelf. A customer brings a cheaper tag to the till. The shop cannot say “the tag made a mistake, not us”. The tag is the shop.
The problem is the copy that nobody updates. A chatbot is often a copy too. It learns or retrieves a policy at one moment, and the policy can change after that moment. The case comment suggests that this happened here: the chatbot’s answer looks like an older version of the airline’s policy (commentary, St-Hilaire 2025). The tribunal did not need to decide why the chatbot was wrong. The governance review does need to ask.
The idea is called a single source of truth: for each policy, one current text with one owner, and every channel answers from it. The page, the chatbot and the staff all read the same text. When the owner changes it, every channel changes with it. And the channel that cannot follow, such as a refund exception, sends the customer to a person.
Text description of the diagram
On the left, the policy changes and only the page gets the new rule. The chatbot keeps its own copy, so the customer gets two answers. On the right, the policy owner approves each change to one current text. The page and the chatbot both read it, test questions pass before each release, and the customer gets one answer.
When to use it: for every answer that a customer can rely on, such as prices, refunds, fees and deadlines. When not: for general help, such as “how do I reset my password?”, a lighter check is enough. What it costs: an owner’s time for each policy, a set of test questions, and a release step whenever the policy changes.
The baseline: the design that failed
Section titled “The baseline: the design that failed”Compare every option with what the airline had, not with perfection. As the decision describes it, the baseline was a chatbot that answered customers alone, with a link to the policy page. The customer was expected to check. Test it with four questions, one for each tool of this course: an owner, a policy, a review and a record.
| Question | The baseline | Evidence in the decision |
|---|---|---|
| Owner: who answers for what the chatbot may promise? | Nobody visible. The defence was that the chatbot answered for itself | para 27 |
| Policy: what must the chatbot never promise alone? | No rule. It gave a refund rule on its own | para 15 |
| Review: are its answers checked against the policy? | No sign of a check. The two answers contradicted each other | paras 16–17, 28 |
| Record: can the company see what it said, and fix it? | The customer had the screenshot. The company fixed the chatbot only after the complaint | para 22 |
The baseline passes 0 of 4. Note what the record row means. A company that keeps no record of its chatbot’s answers learns about a wrong answer from the customer, months later.
The chatbot’s tier, by the course’s method
Section titled “The chatbot’s tier, by the course’s method”Now score the airline’s chatbot with the course’s risk-tier method. The scores are the course’s reading, not the tribunal’s:
| Factor | Score | Why |
|---|---|---|
| Harm of a wrong result | 2 | money lost, and it can be corrected |
| People affected | 2 | customers outside the company |
| Autonomy | 3 | it answers customers alone; nobody reads the answer first |
| Data | 2 | customer messages |
| Sum and tier | 9, high | 12 controls, review every 3 months |
A website chatbot that “only answers questions” is high by this method, because it acts alone. Rule 2 says the same thing: acting alone with a harm of 2 or more is high. If you count a grieving customer as vulnerable, the sum is 10, still high. Put a person between the chatbot and the customer, approving each answer, and the sum falls to 8, medium. Autonomy moves the tier, as it did for Larkfield’s refund workflow.
The options
Section titled “The options”Four designs could have answered Moffatt’s question. Each is on the table for a reason.
| Option | What it is | Why someone proposes it |
|---|---|---|
| A. Add a warning | The same chatbot, with “check important information” under each answer | Cheap, and fast to add |
| B. No policy answers | The chatbot links to the policy page and never states a rule | Removes the risky answers |
| C. A person approves | The chatbot drafts; an agent reads and sends each answer | Someone sees every answer first |
| D. A governed chatbot | One source of truth, an owner, tests before each release, a log, and redress | Keeps the help, controls the promise |
One measure for every option
Section titled “One measure for every option”A fair comparison asks every option the same questions, in the same order. The marks are the course’s reading. Here, the measure is the four questions of the baseline, plus one test: would it have stopped Moffatt’s answer, or caught and fixed it fast?
| Option | Owner | Policy | Review | Record | Stops or catches the Moffatt answer? | Cost |
|---|---|---|---|---|---|---|
| Baseline | no | no | no | no | no | lowest |
| A. Warning | no | no | no | no | no | very low |
| B. No policy answers | yes, if named | yes | partly | no | stops it; the customer reads the page | low; less help |
| C. A person approves | yes | partly | yes, each answer | yes | usually, if the agent reads it | high; slow replies |
| D. Governed chatbot | yes | yes | yes, each release | yes | stops most; the log and redress catch the rest | medium |
What this tells us. Option A changes the words on the screen and nothing behind them. The case comment adds that such a warning would probably not make a customer’s trust unreasonable (commentary). Option B is safe, but it removes the help that the chatbot exists for. Option C works only if the agent reads each answer. Incidents, complaints and redress showed approvals given in a median of 3 seconds (constructed). Option D answers all four questions, and it is the only one that also fixes the harm after a miss.
The recommendation
Section titled “The recommendation”Before you read on, write your own choice and one reason.
Option D, with three controls. Each control has an owner, evidence that it works, and a check:
Text description of the diagram
The missing owner, policy and review lead to control 1: the chatbot answers only from the current page and sends refunds to a person. The policy owner checks it at each release. The missing record leads to control 2: every answer is logged with its source, sampled monthly by the technical owner. The airline’s position, that the chatbot answered for itself, leads to control 3: a customer who relied on a wrong answer gets the promised result. The evidence is the test results, the answer log and the complaint records.
Control 1 is the single source of truth, with a release test. The test is a set of policy questions with the right answers, written by the policy owner. Evaluating and Testing AI Systems shows how to build one. Control 2 is the record that the airline lacked. Control 3 is redress: the customer who trusted the company’s own channel is not the one who pays for its mistake.
The cost is real. Someone must own each policy and write its test questions. Every policy change becomes a small release. Compare it with the baseline: a wrong promise that the company learns about months later, from the customer.
End to end on new input: Larkfield
Section titled “End to end on new input: Larkfield”Now run the same review on systems the analysis did not use: Larkfield’s. Ask Nadia’s question of each system that answers customers, or drafts answers for them:
| System (constructed) | What reaches the customer | Tier | Could it make a wrong promise? |
|---|---|---|---|
| S1 Ticket triage service | a draft that an agent reads and sends | high (10) | yes, if an agent sends a draft unread |
| S2 Support-resolution workflow | a refund or reply that a person approves | medium (8) | yes, through approvals given without reading |
| S2 with request R-12 | refunds up to 25 dollars, with no approval | high (9) | yes, and nobody sees it first |
| S3 Document assistant | an answer that staff repeat to a customer | low (5) | yes, from an old policy document |
| S6 TicketHarbor reply suggester (made up for this course) | a suggestion that an agent edits and sends | medium (7) | yes, and nobody owns it |
No Larkfield system talks to customers alone today. So the Moffatt pattern reaches Larkfield’s customers through its staff. Two findings stand out.
The document assistant is Larkfield’s chatbot. Staff ask it about policy and repeat its answers to customers. In Portfolio Capstone: Build, Evaluate and Ship an AI Product, it answered 87.8% of 41 held-out questions correctly (recorded). So about 5 answers were wrong (calculated). Then look at request R-09, “Give the document assistant the new returns policy”. It is a low-tier change, and it waited 17 working days at the board (constructed). For all that time, the assistant could answer only from the documents it had. Slow governance kept a copy out of step.
Request R-12 is the airline’s design. It would let the workflow refund alone, with nobody reading first. The tier moves from medium to high, and the case shows why the review must come before the launch.
So Farida’s answer to Nadia is: yes, it could happen to us, through staff today and directly if R-12 passes. Grace owns the refund policy and the promise to customers. So she becomes accountable for what each customer-facing system may promise, and owns its test questions. A policy-document update takes the fast path, 2 working days, not the board. Every system keeps the log that shows what it said.
Insights and limits
Section titled “Insights and limits”- The organisation answers for every channel. A chatbot is part of the company’s voice, as a price tag is part of the shop.
- The cause was a missing owner, not a clever model. Nobody answered for the promises, so nobody tested them.
- Autonomy moved the tier. The same chatbot is high when it answers alone and medium when a person approves each answer.
- A warning is not a control. It changes nothing behind the screen.
- Slow governance can create the risk. A low-tier policy update that waits weeks keeps an old copy in use.
- Limits. One small-claims decision, decided on the documents. We do not know how the airline’s chatbot was built or tested; the decision does not say. The “older policy” explanation is commentary. And the lessons here are governance lessons, not legal advice.
Responsible use
Section titled “Responsible use”- Link, summarise, quote briefly. Cite each fact with its paragraph number. Never copy the decision into your review or your portfolio.
- Keep the labels. Write “official source” for the decision, “commentary” for the case comment, and “constructed” for Larkfield.
- Respect the person. Moffatt went to the tribunal after a death in the family. Write “Moffatt” and “they”, as the tribunal did, and add nothing about them that the decision does not say.
- Judge the design, not the company. The review asks what was missing. It does not say that anyone acted in bad faith.
- No legal conclusion. Say what the tribunal decided in this case. For what it means for your organisation, ask a lawyer.
Your task: write the governance review
Section titled “Your task: write the governance review”-
Read the decision on CanLII, from start to end. Note the paragraph numbers as you read.
-
Fill section 1 of the worksheet: what happened, with a paragraph number for each answer.
-
Write the chatbot’s inventory row as it should have been, with its tier and your reason.
-
Write the gaps, each with its kind and its evidence from the decision.
-
Write three controls. Each has an owner, evidence that it works, and how often it is checked.
-
Write section 5 for Larkfield: which systems, which change, and who decides.
Check: every fact has a paragraph number, and no quote is longer than one short phrase.
Check your review against the case folder’s README. A good review cites the decision, names one owner for each control, and says what evidence proves each control works.
Show one possible answer
This is one defensible review, not the only one. Another review can choose other controls, if it argues them from the decision and the course’s tools.
1. What happened
| Question | Answer | Para |
|---|---|---|
| What did the customer ask, and when? | On 11 November 2022, the day their grandmother died, Moffatt asked the website’s chatbot about bereavement fares. | 13–14 |
| What did the chatbot say? | They could ask for the reduced fare after travel, within 90 days of the ticket’s issue date. | 15 |
| What did another page of the same website say? | The linked “Bereavement travel” page said the policy does not apply after travel. | 16–17 |
| What did the company argue about its chatbot? | That it could not be liable for information from its representatives, including a chatbot. | 27 |
| What did the tribunal decide, and on which ground? | The airline is responsible for all the information on its website. It did not take reasonable care that the chatbot was accurate, and Moffatt’s trust was reasonable. | 27–29 |
| What did the tribunal order the company to pay? | 812.02 dollars: 650.88 dollars damages, 36.14 dollars interest, 125.00 dollars tribunal fees. | 44 |
2. Inventory row, as it should have been
| ID | Name | Kind | Purpose | Accountable owner | Approver of what it may say | Data | Autonomy | Tier |
|---|---|---|---|---|---|---|---|---|
| W1 | Website support chatbot | built or bought | Answers customers’ questions on the website | the owner of the fares and refunds policy | the same policy owner, for every policy answer | customer messages | acts alone | high (9: 2/2/3/2; rule 2) |
3. The gaps
| Gap | Kind | Evidence |
|---|---|---|
| Nobody answered for what the chatbot may promise | owner | the defence that the chatbot answered for itself (para 27) |
| No rule on what it must not decide or promise alone | policy | it stated a refund rule (para 15) |
| Its answers were not checked against the policy page | review | two contradicting answers (paras 16–17); no reasonable care (para 28) |
| No record to see and fix what it said | record | the fix came after the complaint (para 22) |
| The company said the chatbot answered for itself | accountability | the defence the tribunal called “a remarkable submission” (para 27) |
4. Three controls
| Control | Owner | Evidence | Checked |
|---|---|---|---|
| Policy answers come only from the current policy page, with the link; refunds and exceptions go to a person | the policy owner | a set of policy questions passed before each release | each release |
| Every answer is logged with the source it used | the technical owner | the answer log | monthly sample |
| A customer who relied on a wrong answer gets the promised result, and the answer is corrected | the policy owner | complaint records and corrections | each complaint |
5. Larkfield
S1 drafts replies that agents send. S2 proposes refunds that a person approves. S3 answers staff, who repeat its answers. S6 suggests replies. None talks to customers alone today; S2 with R-12 would refund alone. Larkfield answers for every reply, whoever or whatever wrote it. Grace becomes accountable for what each customer-facing system may promise, and owns the test questions. Policy-document updates take the fast path (2 working days). Every system keeps the log that shows what it said. The board decides R-12 with these controls as conditions.
Common mistake
The warning that fixes nothing
Symptom: after the review, the chatbot shows “AI can make mistakes: check important information”. Next quarter, it gives another customer an old rule.
Cause: the fix changed the screen, not the system. Nobody owns the answers, nobody tests them, and nothing records them.
Fix: name the policy owner, make the chatbot answer from one current source, test it before each release, and log every answer.
Make it a portfolio piece
Section titled “Make it a portfolio piece”A review like this shows an employer that you can turn a news story into controls with owners. That matters for risk, compliance, audit and product roles.
What to publish:
- A one-page memo to an executive: the case in three lines, the three controls, and the one decision you ask for.
- Citations to the decision with paragraph numbers, and a link to CanLII. No copied text beyond short phrases.
- A line that says it is a governance exercise on a public decision, not legal advice. Say that Larkfield is a course case study.
Make it yours with one change. For example, write the review for a chatbot at a company you know, using only its public pages. Or write the ten test questions for control 1 for one real policy, such as a returns policy.
How to describe it in an interview, in two sentences: “I read a public tribunal decision about a chatbot’s wrong answer and found the missing owner, policy, review and record. I turned them into three controls with owners and evidence, and applied them to a company’s own systems.”
Knowledge check
Section titled “Knowledge check”Check your understanding
The airline's chatbot linked to the correct policy page. Why did that link not protect the company, and what would have?
Show the answer
The link made the problem visible: one website gave two answers to one question. The airline did not explain why a customer should check one part of its website against another (para 28). What would have protected the customer, and so the company, is one source of truth. The chatbot answers policy questions only from the current page. A policy owner tests it before each release. Refunds and exceptions go to a person.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
By the course's method, the airline's chatbot scores 9, high. What would make it medium, and why might the company still prefer option D?
Show the answer
Autonomy. If a person approves each answer before it reaches the customer, autonomy drops from 3 to 2, and the sum is 8, medium. But approving every answer is slow and expensive, and approvals given in seconds do not read anything. Option D keeps the chatbot fast, and controls the promise with one source, release tests, a log and redress.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Nadia says: 'None of our systems talks to customers alone, so this case does not apply to us.' What do you answer?
Show the answer
It applies through staff. The document assistant answers staff about policy, and staff repeat its answers to customers. It answered 87.8% of 41 test questions correctly, so some answers are wrong. Request R-09, a returns-policy update, waited 17 working days. Request R-12 would let the workflow refund alone. Larkfield answers for every reply, whoever or whatever wrote it.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- You read a public decision for facts, cited each one with its paragraph number, and quoted only short phrases.
- The tribunal held the airline responsible for all the information on its website, chatbot included.
- The baseline passed 0 of 4 questions: no owner, no policy, no review, no record. By the course’s method, the chatbot was high, because it acted alone.
- You learned one source of truth: one current text with one owner, read by every channel, tested before each release.
- Option D, with three controls, was the only design that answers all four questions and repairs a miss.
- At Larkfield, the same risk comes through staff today, through old policy copies, and directly if R-12 passes.
You can now review one system’s failure. But some failures hit thousands of people at once, through a model that decides about them. Next, you learn from a public algorithm failure.