Air Canada held liable after its website chatbot invented a bereavement-fare policy
2024-air-canada-chatbot-liability · 2024-02-14
Air Canada's website chatbot told a customer they could claim a bereavement fare retroactively — a policy that did not exist. Canada's Civil Resolution Tribunal (Moffatt v. Air Canada) held the airline liable for the chatbot's negligent misrepresentation and awarded CAD 812.02.
System
- Framework
- Air Canada website support chatbot
- Tools
- website-chat
- Vendor
- Air Canada
- Autonomy
- human-in-the-loop
Classification
- Primary class
- hallucination
- Chain
- hallucination/fabricated-policy
- Attack vector
- n-a
- Causation
- entity: ai · intentionality: unintentional · timing: post-deployment
Trigger
A customer asked the website chatbot about bereavement fares. The chatbot stated they could apply for a reduced bereavement rate retroactively within 90 days by submitting a form, which contradicted Air Canada's actual policy shown elsewhere on its site.
Root cause
The chatbot produced confidently incorrect policy information, and Air Canada did not ensure the chatbot's output was accurate. The tribunal rejected the argument that the chatbot was a separate legal entity.
Contributing factors
- The bot's answers were not grounded in the authoritative policy pages.
- No accountability treating bot output as an official company representation.
Detection
Surfaced through the customer's complaint and the subsequent tribunal case, Moffatt v. Air Canada.
Recovery
The tribunal ordered Air Canada to pay damages; the ruling established that a company is liable for negligent misrepresentations made by its chatbot.
Prevention
Ground customer-facing bots in authoritative, current policy sources; constrain answers to verified content; test for policy-contradiction; treat bot output as company representation with clear accountability.
Blast radius
- Cost
- $812
- User harm
- The customer relied on the incorrect advice and did not receive the expected fare treatment; the tribunal awarded CAD 812.02 in damages and fees. economic legal-regulatory
- Scope
- single customer; precedent for chatbot liability
- Reversibility
- reversible
References
- OWASP LLM
- LLM09
- Tags
- chatbot liability customer-facing
Sources
Cite this incident
Permalink: https://swarmproof.github.io/agent-postmortems/2024-air-canada-chatbot-liability/
@misc{2024-air-canada-chatbot-liability,
title = {Air Canada held liable after its website chatbot invented a bereavement-fare policy},
year = {2024},
howpublished = {agent-postmortems},
url = {https://swarmproof.github.io/agent-postmortems/2024-air-canada-chatbot-liability/}
}