Skip to content
Français

A tribunal case about a chatbot's answer

  • Case study 1
  • 120 min
  • Optional · 1 case study needed for the certificate

Where we are

In this course, we gave Larkfield's systems owners, policies, reviews and records. So far, every case was Larkfield's own, and made up. This case study tests those tools on a real public decision: a tribunal held an airline responsible for what its website chatbot told a customer. We find what was missing, and we write the governance review that would have prevented it.

What you will be able to do

Test the whole course on a real decision. You read a public tribunal decision about a chatbot’s wrong answer. You find the missing owner, policy, review and record, and write the review that would have prevented it. Then you apply it to Larkfield.

Case studyPortfolio piece

Time
About 2 hours
Runs in
A browser and a document tool
Skills used
  • Read a decision for facts
  • Paragraph-numbered citations
  • Inventory row and risk tier
  • Owner, policy, review, record
  • Controls with evidence
  • One source of truth
  • Governance memo

The case files are documents, not code: read them in your browser, or download them to work offline. Write your answer in your own document tool.

The case folder has a README, a source card and a worksheet. You read the decision itself on CanLII, not in the folder: the course links to it and does not copy it.

Before you startWhat to know and to have readyAbout 120 minutes

What you need to know

This case study uses the whole course. These lessons matter most:

The page teaches one new idea: one source of truth for every answer about a policy.

What you need

  • A browser and a document tool (a word processor, a text editor or a spreadsheet).
  • The decision on CanLII, and the README, source card and worksheet of the case folder.
  • No account, no installation and no money.

Nadia, Larkfield’s head of customer operations, read about the case in the news. She asks Farida, Larkfield’s risk and compliance lead: “Our support systems answer customers too. Could this happen to us? What would have prevented it?” Larkfield, Nadia and her question are constructed for this course. The case is real.

Your objective is a two-page governance review, in the case folder’s worksheet. It says what happened, what was missing, which three controls would have prevented it, and what Larkfield should change. This page walks through the analysis with you. You write the review at the end, in your own words.

The “data” of this case study is one document: Moffatt v. Air Canada, 2024 BCCRT 149, a decision of the Civil Resolution Tribunal of British Columbia, issued on 14 February 2024. Read it on CanLII (official source, checked 22 October 2026), or search for “2024 BCCRT 149” on the tribunal’s decisions site. The source card says what the decision is and how you may use it.

Before you analyse a source, check what kind of source it is. Three facts change how much weight the decision can carry:

  • It is a small claim. The tribunal decided it on the documents, with no oral hearing. Moffatt represented themself, and an employee represented the airline (paras 5 and 8).
  • It decides one dispute. What it means for other cases, or for your organisation, is a question for a lawyer. This page draws governance lessons, not legal ones.
  • The tribunal calls the applicant “Mr. Moffatt” and uses “they”. This page says “Moffatt” and “they”.

A published case comment adds context: Isabelle St-Hilaire, in the UBC Law Review (2025 CanLIIDocs 1963). Where this page uses it, it says commentary: one author’s view, not the tribunal’s finding.

Read the decision once, quickly, from start to end. Then read only the paragraphs below again. They hold every fact that the review needs.

The case in three groups. The chat, 11 November 2022: Moffatt's grandmother dies, and Moffatt asks the airline website's chatbot about bereavement fares (paras 13 to 14). The chatbot answers: ask for the reduced rate after travel, within 90 days (para 15). Its own link opens the page Bereavement travel, which says no requests after travel (paras 16 to 17); a dashed line between the two says one website, two answers. What Moffatt did: relies on the chatbot and books two one-way flights (para 18), applies on 17 November 2022, within the 90 days (para 20). The dispute: on 8 February 2023 the airline admits misleading words and says it will update the chatbot (para 22); the airline argues it cannot be liable for its chatbot, and the tribunal rejects it (paras 27 to 29); on 14 February 2024 the order is 812.02 dollars (para 44), drawn with a thick border.
The case in seven steps, with the decision's paragraph numbers. The chatbot's answer (red) and the page it linked to (dashed) said different things.
Text description of the diagram

On 11 November 2022, Moffatt’s grandmother died, and Moffatt asked the chatbot about bereavement fares (paras 13–14). The chatbot said to ask for the reduced rate after travel, within 90 days (para 15). Its own link went to a page that said the opposite (paras 16–17). Moffatt booked two flights and applied within the 90 days (paras 18, 20). The airline admitted “misleading words” (para 22). The tribunal rejected its defence (paras 27–29) and ordered it to pay 812.02 dollars (para 44).

In our words: Moffatt asked the airline’s website chatbot about fares for a family death. The chatbot said they could ask for the reduced fare after travel, “within 90 days of the date your ticket was issued” (para 15). The words “bereavement fares” in that answer linked to the airline’s own policy page. That page said the policy does not apply after travel (paras 16–17). Moffatt trusted the chatbot, flew, and applied in time. The airline did not give the reduced fare. In its answer, it admitted that the chatbot had used “misleading words” (para 22).

Look at the dashed line in the diagram. One website gave two answers to one question. That is the whole case in one picture.

The airline’s main argument was that it could not be held liable for information from its agents or representatives, including a chatbot. The tribunal called this “a remarkable submission” (para 27). In plain words, its reasons were these:

  1. The company answers for its whole website, chatbot included. “It makes no difference whether the information comes from a static page or a chatbot.” (para 27)
  2. The company did not take care. It “did not take reasonable care to ensure its chatbot was accurate” (para 28). It also did not explain why a customer should check one part of its website against another.
  3. Moffatt’s trust was reasonable (para 29).
  4. The contract terms did not help the airline. It relied on its tariff, but did not file it, so it did not prove that defence (para 31).

The tribunal then counted the money.

The money in the case (official source, paras 3, 34, 38–40 and 44). The order includes interest and tribunal fees.
Text description of the chart

A column chart with four columns, in Canadian dollars. The tribunal uses 1630.36 dollars as what Moffatt paid (paras 34, 40). Moffatt claimed 880.00 dollars. The tribunal awarded 650.88 dollars in damages. The total order was 812.02 dollars: the damages, 36.14 dollars of interest and 125.00 dollars of tribunal fees.

The tribunal uses 1630.36 dollars as the amount paid (paras 34, 40). The course’s own check: the two fares in para 18 add to 1640.36 dollars.

What this tells us. The money is small: 812.02 dollars in all, to be paid within 14 days (para 44). The airline had also offered a 200.00 dollars coupon, which Moffatt did not accept (para 41). As with Larkfield’s AI bill, the risk is not in the amount. It is in what the case showed in public. Nobody checked what the chatbot promised, and the company’s defence was that the chatbot answered for itself.

Governance starts from the opposite position. The organisation answers for every system it deploys, whoever built it. You met this in Governance that helps teams ship, when Tomás said the vendor was responsible.

Think of a small shop. The owner keeps one price list. The price tags, the website and the staff all copy from it. One day the owner raises a price and changes the list, but forgets the tags on the shelf. A customer brings a cheaper tag to the till. The shop cannot say “the tag made a mistake, not us”. The tag is the shop.

The problem is the copy that nobody updates. A chatbot is often a copy too. It learns or retrieves a policy at one moment, and the policy can change after that moment. The case comment suggests that this happened here: the chatbot’s answer looks like an older version of the airline’s policy (commentary, St-Hilaire 2025). The tribunal did not need to decide why the chatbot was wrong. The governance review does need to ask.

The idea is called a single source of truth: for each policy, one current text with one owner, and every channel answers from it. The page, the chatbot and the staff all read the same text. When the owner changes it, every channel changes with it. And the channel that cannot follow, such as a refund exception, sends the customer to a person.

Two panels. Left, two copies that nobody keeps in step: the policy changes; the policy page gets the current rule; a dashed arrow labelled nobody updates it goes to the chatbot, which has its own copy, maybe an older rule (red). Both lead to: the customer gets two answers (red). Right, one source of truth, one owner: the policy owner approves each change to the current policy, the one source (thick border). The policy page and the chatbot both read from it; the chatbot answers with the link, and refunds and exceptions go to a person. A dashed arrow: test questions pass before each release. Both lead to: the customer gets one answer (green).
Left: two copies drift apart, and the customer gets two answers. Right: one current text with one owner, read by every channel and tested before each release.
Text description of the diagram

On the left, the policy changes and only the page gets the new rule. The chatbot keeps its own copy, so the customer gets two answers. On the right, the policy owner approves each change to one current text. The page and the chatbot both read it, test questions pass before each release, and the customer gets one answer.

When to use it: for every answer that a customer can rely on, such as prices, refunds, fees and deadlines. When not: for general help, such as “how do I reset my password?”, a lighter check is enough. What it costs: an owner’s time for each policy, a set of test questions, and a release step whenever the policy changes.

Compare every option with what the airline had, not with perfection. As the decision describes it, the baseline was a chatbot that answered customers alone, with a link to the policy page. The customer was expected to check. Test it with four questions, one for each tool of this course: an owner, a policy, a review and a record.

Question The baseline Evidence in the decision
Owner: who answers for what the chatbot may promise? Nobody visible. The defence was that the chatbot answered for itself para 27
Policy: what must the chatbot never promise alone? No rule. It gave a refund rule on its own para 15
Review: are its answers checked against the policy? No sign of a check. The two answers contradicted each other paras 16–17, 28
Record: can the company see what it said, and fix it? The customer had the screenshot. The company fixed the chatbot only after the complaint para 22

The baseline passes 0 of 4. Note what the record row means. A company that keeps no record of its chatbot’s answers learns about a wrong answer from the customer, months later.

The chatbot’s tier, by the course’s method

Section titled “The chatbot’s tier, by the course’s method”

Now score the airline’s chatbot with the course’s risk-tier method. The scores are the course’s reading, not the tribunal’s:

Factor Score Why
Harm of a wrong result 2 money lost, and it can be corrected
People affected 2 customers outside the company
Autonomy 3 it answers customers alone; nobody reads the answer first
Data 2 customer messages
Sum and tier 9, high 12 controls, review every 3 months

A website chatbot that “only answers questions” is high by this method, because it acts alone. Rule 2 says the same thing: acting alone with a harm of 2 or more is high. If you count a grieving customer as vulnerable, the sum is 10, still high. Put a person between the chatbot and the customer, approving each answer, and the sum falls to 8, medium. Autonomy moves the tier, as it did for Larkfield’s refund workflow.

Four designs could have answered Moffatt’s question. Each is on the table for a reason.

Option What it is Why someone proposes it
A. Add a warning The same chatbot, with “check important information” under each answer Cheap, and fast to add
B. No policy answers The chatbot links to the policy page and never states a rule Removes the risky answers
C. A person approves The chatbot drafts; an agent reads and sends each answer Someone sees every answer first
D. A governed chatbot One source of truth, an owner, tests before each release, a log, and redress Keeps the help, controls the promise

A fair comparison asks every option the same questions, in the same order. The marks are the course’s reading. Here, the measure is the four questions of the baseline, plus one test: would it have stopped Moffatt’s answer, or caught and fixed it fast?

Option Owner Policy Review Record Stops or catches the Moffatt answer? Cost
Baseline no no no no no lowest
A. Warning no no no no no very low
B. No policy answers yes, if named yes partly no stops it; the customer reads the page low; less help
C. A person approves yes partly yes, each answer yes usually, if the agent reads it high; slow replies
D. Governed chatbot yes yes yes, each release yes stops most; the log and redress catch the rest medium

What this tells us. Option A changes the words on the screen and nothing behind them. The case comment adds that such a warning would probably not make a customer’s trust unreasonable (commentary). Option B is safe, but it removes the help that the chatbot exists for. Option C works only if the agent reads each answer. Incidents, complaints and redress showed approvals given in a median of 3 seconds (constructed). Option D answers all four questions, and it is the only one that also fixes the harm after a miss.

Before you read on, write your own choice and one reason.

Option D, with three controls. Each control has an owner, evidence that it works, and a check:

Red boxes for what was missing, each with an arrow to a green control. Owner, nobody answered for what the chatbot may promise; policy, no rule on what it must not promise alone; review, its answers were not checked against the policy page: all three lead to control 1, answers policy questions only from the current page, with the link, and refunds and exceptions go to a person; owner: the policy owner; checked each release. Record, no log of what it said, to whom, from which source, leads to control 2, every answer is logged with the source it used; owner: the technical owner; checked by a monthly sample. Accountability, the company said the chatbot answered for itself, leads to control 3, a customer who relied on a wrong answer gets the promised result, and the answer is corrected; owner: the policy owner; checked at each complaint. The three controls lead to the evidence, with a thick border: test results per release, the answer log, complaint records.
The governance review on one page: what was missing (red), three controls (green), and the evidence that each control works.
Text description of the diagram

The missing owner, policy and review lead to control 1: the chatbot answers only from the current page and sends refunds to a person. The policy owner checks it at each release. The missing record leads to control 2: every answer is logged with its source, sampled monthly by the technical owner. The airline’s position, that the chatbot answered for itself, leads to control 3: a customer who relied on a wrong answer gets the promised result. The evidence is the test results, the answer log and the complaint records.

Control 1 is the single source of truth, with a release test. The test is a set of policy questions with the right answers, written by the policy owner. Evaluating and Testing AI Systems shows how to build one. Control 2 is the record that the airline lacked. Control 3 is redress: the customer who trusted the company’s own channel is not the one who pays for its mistake.

The cost is real. Someone must own each policy and write its test questions. Every policy change becomes a small release. Compare it with the baseline: a wrong promise that the company learns about months later, from the customer.

Now run the same review on systems the analysis did not use: Larkfield’s. Ask Nadia’s question of each system that answers customers, or drafts answers for them:

System (constructed) What reaches the customer Tier Could it make a wrong promise?
S1 Ticket triage service a draft that an agent reads and sends high (10) yes, if an agent sends a draft unread
S2 Support-resolution workflow a refund or reply that a person approves medium (8) yes, through approvals given without reading
S2 with request R-12 refunds up to 25 dollars, with no approval high (9) yes, and nobody sees it first
S3 Document assistant an answer that staff repeat to a customer low (5) yes, from an old policy document
S6 TicketHarbor reply suggester (made up for this course) a suggestion that an agent edits and sends medium (7) yes, and nobody owns it

No Larkfield system talks to customers alone today. So the Moffatt pattern reaches Larkfield’s customers through its staff. Two findings stand out.

The document assistant is Larkfield’s chatbot. Staff ask it about policy and repeat its answers to customers. In Portfolio Capstone: Build, Evaluate and Ship an AI Product, it answered 87.8% of 41 held-out questions correctly (recorded). So about 5 answers were wrong (calculated). Then look at request R-09, “Give the document assistant the new returns policy”. It is a low-tier change, and it waited 17 working days at the board (constructed). For all that time, the assistant could answer only from the documents it had. Slow governance kept a copy out of step.

Request R-12 is the airline’s design. It would let the workflow refund alone, with nobody reading first. The tier moves from medium to high, and the case shows why the review must come before the launch.

So Farida’s answer to Nadia is: yes, it could happen to us, through staff today and directly if R-12 passes. Grace owns the refund policy and the promise to customers. So she becomes accountable for what each customer-facing system may promise, and owns its test questions. A policy-document update takes the fast path, 2 working days, not the board. Every system keeps the log that shows what it said.

  • The organisation answers for every channel. A chatbot is part of the company’s voice, as a price tag is part of the shop.
  • The cause was a missing owner, not a clever model. Nobody answered for the promises, so nobody tested them.
  • Autonomy moved the tier. The same chatbot is high when it answers alone and medium when a person approves each answer.
  • A warning is not a control. It changes nothing behind the screen.
  • Slow governance can create the risk. A low-tier policy update that waits weeks keeps an old copy in use.
  • Limits. One small-claims decision, decided on the documents. We do not know how the airline’s chatbot was built or tested; the decision does not say. The “older policy” explanation is commentary. And the lessons here are governance lessons, not legal advice.
  • Link, summarise, quote briefly. Cite each fact with its paragraph number. Never copy the decision into your review or your portfolio.
  • Keep the labels. Write “official source” for the decision, “commentary” for the case comment, and “constructed” for Larkfield.
  • Respect the person. Moffatt went to the tribunal after a death in the family. Write “Moffatt” and “they”, as the tribunal did, and add nothing about them that the decision does not say.
  • Judge the design, not the company. The review asks what was missing. It does not say that anyone acted in bad faith.
  • No legal conclusion. Say what the tribunal decided in this case. For what it means for your organisation, ask a lawyer.
  1. Read the decision on CanLII, from start to end. Note the paragraph numbers as you read.

  2. Fill section 1 of the worksheet: what happened, with a paragraph number for each answer.

  3. Write the chatbot’s inventory row as it should have been, with its tier and your reason.

  4. Write the gaps, each with its kind and its evidence from the decision.

  5. Write three controls. Each has an owner, evidence that it works, and how often it is checked.

  6. Write section 5 for Larkfield: which systems, which change, and who decides.

    Check: every fact has a paragraph number, and no quote is longer than one short phrase.

Check your review against the case folder’s README. A good review cites the decision, names one owner for each control, and says what evidence proves each control works.

Show one possible answer

This is one defensible review, not the only one. Another review can choose other controls, if it argues them from the decision and the course’s tools.

1. What happened

Question Answer Para
What did the customer ask, and when? On 11 November 2022, the day their grandmother died, Moffatt asked the website’s chatbot about bereavement fares. 13–14
What did the chatbot say? They could ask for the reduced fare after travel, within 90 days of the ticket’s issue date. 15
What did another page of the same website say? The linked “Bereavement travel” page said the policy does not apply after travel. 16–17
What did the company argue about its chatbot? That it could not be liable for information from its representatives, including a chatbot. 27
What did the tribunal decide, and on which ground? The airline is responsible for all the information on its website. It did not take reasonable care that the chatbot was accurate, and Moffatt’s trust was reasonable. 27–29
What did the tribunal order the company to pay? 812.02 dollars: 650.88 dollars damages, 36.14 dollars interest, 125.00 dollars tribunal fees. 44

2. Inventory row, as it should have been

ID Name Kind Purpose Accountable owner Approver of what it may say Data Autonomy Tier
W1 Website support chatbot built or bought Answers customers’ questions on the website the owner of the fares and refunds policy the same policy owner, for every policy answer customer messages acts alone high (9: 2/2/3/2; rule 2)

3. The gaps

Gap Kind Evidence
Nobody answered for what the chatbot may promise owner the defence that the chatbot answered for itself (para 27)
No rule on what it must not decide or promise alone policy it stated a refund rule (para 15)
Its answers were not checked against the policy page review two contradicting answers (paras 16–17); no reasonable care (para 28)
No record to see and fix what it said record the fix came after the complaint (para 22)
The company said the chatbot answered for itself accountability the defence the tribunal called “a remarkable submission” (para 27)

4. Three controls

Control Owner Evidence Checked
Policy answers come only from the current policy page, with the link; refunds and exceptions go to a person the policy owner a set of policy questions passed before each release each release
Every answer is logged with the source it used the technical owner the answer log monthly sample
A customer who relied on a wrong answer gets the promised result, and the answer is corrected the policy owner complaint records and corrections each complaint

5. Larkfield

S1 drafts replies that agents send. S2 proposes refunds that a person approves. S3 answers staff, who repeat its answers. S6 suggests replies. None talks to customers alone today; S2 with R-12 would refund alone. Larkfield answers for every reply, whoever or whatever wrote it. Grace becomes accountable for what each customer-facing system may promise, and owns the test questions. Policy-document updates take the fast path (2 working days). Every system keeps the log that shows what it said. The board decides R-12 with these controls as conditions.

Common mistake

The warning that fixes nothing

Symptom: after the review, the chatbot shows “AI can make mistakes: check important information”. Next quarter, it gives another customer an old rule.

Cause: the fix changed the screen, not the system. Nobody owns the answers, nobody tests them, and nothing records them.

Fix: name the policy owner, make the chatbot answer from one current source, test it before each release, and log every answer.

A review like this shows an employer that you can turn a news story into controls with owners. That matters for risk, compliance, audit and product roles.

What to publish:

  • A one-page memo to an executive: the case in three lines, the three controls, and the one decision you ask for.
  • Citations to the decision with paragraph numbers, and a link to CanLII. No copied text beyond short phrases.
  • A line that says it is a governance exercise on a public decision, not legal advice. Say that Larkfield is a course case study.

Make it yours with one change. For example, write the review for a chatbot at a company you know, using only its public pages. Or write the ten test questions for control 1 for one real policy, such as a returns policy.

How to describe it in an interview, in two sentences: “I read a public tribunal decision about a chatbot’s wrong answer and found the missing owner, policy, review and record. I turned them into three controls with owners and evidence, and applied them to a company’s own systems.”

Check your understanding

The airline's chatbot linked to the correct policy page. Why did that link not protect the company, and what would have?

Show the answer

The link made the problem visible: one website gave two answers to one question. The airline did not explain why a customer should check one part of its website against another (para 28). What would have protected the customer, and so the company, is one source of truth. The chatbot answers policy questions only from the current page. A policy owner tests it before each release. Refunds and exceptions go to a person.

Check your understanding

By the course's method, the airline's chatbot scores 9, high. What would make it medium, and why might the company still prefer option D?

Show the answer

Autonomy. If a person approves each answer before it reaches the customer, autonomy drops from 3 to 2, and the sum is 8, medium. But approving every answer is slow and expensive, and approvals given in seconds do not read anything. Option D keeps the chatbot fast, and controls the promise with one source, release tests, a log and redress.

Check your understanding

Nadia says: 'None of our systems talks to customers alone, so this case does not apply to us.' What do you answer?

Show the answer

It applies through staff. The document assistant answers staff about policy, and staff repeat its answers to customers. It answered 87.8% of 41 test questions correctly, so some answers are wrong. Request R-09, a returns-policy update, waited 17 working days. Request R-12 would let the workflow refund alone. Larkfield answers for every reply, whoever or whatever wrote it.

Recap

  • You read a public decision for facts, cited each one with its paragraph number, and quoted only short phrases.
  • The tribunal held the airline responsible for all the information on its website, chatbot included.
  • The baseline passed 0 of 4 questions: no owner, no policy, no review, no record. By the course’s method, the chatbot was high, because it acted alone.
  • You learned one source of truth: one current text with one owner, read by every channel, tested before each release.
  • Option D, with three controls, was the only design that answers all four questions and repairs a miss.
  • At Larkfield, the same risk comes through staff today, through old policy copies, and directly if R-12 passes.

You can now review one system’s failure. But some failures hit thousands of people at once, through a model that decides about them. Next, you learn from a public algorithm failure.

Next: Learn from a public algorithm failure