Learn from a public algorithm failure
- Case study 2
- 120 min
- Optional · 1 case study needed for the certificate
Where we are
This case study takes the impact assessment and the risk tiers from Module 3 to a real public system. In 2020, England’s exam regulator adjusted A level grades with a statistical model, and four days after results day the government went back to the teachers’ grades. We read the official reports and write the impact assessment that should have come before launch.
Case study 2 of 3 · AI Governance: Policies, Risk and AccountabilityLearn from a public algorithm failure
1. What governance is for
2. Laws and standards in plain words
3. Inventory, risk tiers and impact assessment
4. Policies people can follow
5. Controls, documentation and evidence
What you will be able to do
Write the impact assessment that a real public system needed before it decided about people. You read the official reports on England’s 2020 grading model and find what the evidence showed before results day. Then you decide what should have stopped the launch.
The case folder has a README and a source card, which gives every figure on this page with its table and page. You write your answer in the course’s impact-assessment template.
Before you startWhat to know and to have ready
What you need to know
These lessons matter most:
- The seven sections of an impact assessment, and who signs it, from Impact assessments.
- The four factors, the two rules and the two questions before scoring, from Risk tiers and proportionate controls.
- Redress: how a person gets a wrong result corrected, from Incidents, complaints and redress.
The page teaches one new idea: a system can be right on average and wrong for a person.
What you need
- A browser and a document tool (a word processor, a text editor or a spreadsheet).
- The case folder and the impact-assessment template, both in labs
C33. - No account, no installation and no money.
The situation
Section titled “The situation”On 18 March 2020, because of the pandemic, the government cancelled the summer exams in England. Students still needed grades for university, college or work.
So teachers gave each student a centre assessment grade (CAG): the grade that the student would most likely have achieved in the exam. Teachers also ranked their students in each subject. Then Ofqual, the exams regulator for England, adjusted the teachers’ grades with a statistical model, to keep results in line with earlier years.
A level results came out on 13 August 2020, and many students got a lower grade than their teacher had proposed. On 17 August, the regulator and the Department for Education switched to the teachers’ grades, or the model’s grade where it was higher.
At Larkfield, the governance board asks Farida for training material. The request is constructed for the course: “Write the impact assessment that should have been done before these grades were issued. What would it have found?”
The model was a statistical method, not a large language model. That does not matter here: an impact assessment asks the same questions of any system that decides about people.
What you will produce
Section titled “What you will produce”You write the one-page impact assessment for the grading model, as if before results day, with the course’s template. On the way, this page walks through the evidence, the tier, the options, one fair comparison and a recommendation. Then it tests two stop lines on the real data.
The evidence
Section titled “The evidence”The “data” of this case study is three official documents on GOV.UK (official source, checked 22 October 2026):
| Document | What it holds |
|---|---|
| Interim report, Ofqual, 13 August 2020, 319 pages, with a 5-page executive summary and Annex Q | How the model works, how it was tested on past years, and the A level results by grade, region, type of school and group of students |
| Statement from the Chair of Ofqual, 17 August 2020 | The switch to the teachers’ grades, and why |
| Department for Education news story, 17 August 2020 | The same decision, from the government |
First checks
Section titled “First checks”Before any analysis, check what the numbers count:
- They count grades, not students. A student with three A levels has three grades. The tables give shares of 718,276 A level grades.
- England only, A level and AS. The GCSE analysis came later.
- The report is the regulator’s own. Read its tables as evidence, and its conclusions as one reading of them.
- The report came out on results day. But its results tables use data from “around a week before results were issued” (p. 136). So the evidence existed before launch.
- Shares are rounded to one decimal. So “about 39%” is our sum of three rounded shares, not an official figure.
Frame the decision
Section titled “Frame the decision”An impact assessment starts with the decision, not the model. Here, the decision was: should the model’s grade replace the teacher’s grade, for every student, on results day?
The regulator wrote five aims for the process (interim report, p. 60). One of them was “to protect, so far as is possible, all students from being systematically advantaged or disadvantaged”. The aims cover accuracy, consistency, delivery and explanation. But none of them says how a student gets a wrong grade corrected, or how fast. Section 5 of the template asks exactly that.
Now score the system with the course’s method. First, the two questions before scoring. Education is an example in the first one. It is also one of the areas that the EU AI Act calls high-risk today. That law did not exist in 2020: we use the method here only to practise it. The second question also gets a yes: the calculated grade was the grade issued, and a student could only challenge it afterwards, through an appeal. So the legal specialist reads the assessment first.
Then the four factors (the scores are this case study’s reading):
| Factor | Score | Why |
|---|---|---|
| Harm of a wrong result | 3 | Access to university, college or a job; hard to undo after offers close |
| People affected | 3 | A decision about each person’s access to education |
| Autonomy | 3 | It acts alone: the calculated grade was the grade issued |
| Data | 2 | Personal data: grades, ranks and past results |
The sum is 11: high. The second rule gives the same answer, because the system acts alone with a harm of 2 or more. If you score data 3, because many students are under 18, the sum is 12 and the tier does not change. A high tier needs 12 controls, among them a full impact assessment that the board reviews, a release gate with written thresholds, and pause conditions.
How the model set a grade
Section titled “How the model set a grade”The regulator chose the Direct Centre Performance model (DCP). “Centre” means a school or college.
Text description of the diagram
The school’s past results in the subject give predicted grade shares for the school. The teachers’ rank order fills those shares from the top down, and that gives each student’s calculated grade. With fewer than 15 students, the model puts more weight on the teacher’s grade. So for most students, the grade comes from the school’s history and the rank.
In the report’s words, the prediction came from “the historical performance of the school or college in that subject” (executive summary, p. 4). Small groups got more weight on the teachers’ grades, because no model predicts well for very small groups.
Explore what the decision needs
Section titled “Explore what the decision needs”Read three things only: how many grades changed, how accurate the model was for one student, and whether some schools gained more.
How many grades changed
Section titled “How many grades changed”Text description of the chart
A column chart of 718,276 A level grades. Lowered by three grades or more: 0.2%. Lowered by two grades: 3.3%. Lowered by one grade: 35.6%. The same as the teacher’s grade: 58.7%. One grade higher: 2.2%. The three “lower” columns are in a different colour.
What this tells us. About 39% of the teachers’ grades were lowered: about 281,000 A level grades (our calculation from Table 9.8). Most of them went down by one grade. But 3.5% went down by two grades or more: about 25,000 grades. A two-grade drop can turn a university offer into a refusal.
The new idea: right on average, wrong for a person
Section titled “The new idea: right on average, wrong for a person”Think of a shoe shop that orders shoes for a whole town. It knows from past years how many of each size the town buys, so the order is right. But then it hands out the shoes by street, not by foot: every street gets the sizes it had last year. The total is right, and many people get the wrong size.
That is the problem here: the national results looked right. The teachers proposed A* for 13.9% of A level grades, against 7.7% in 2019 (executive summary, p. 2). The grades issued gave 8.9% (Table 9.6, p. 134), close to an ordinary year. But each student’s grade came from the school’s past, not from the student.
The idea: a system can be right on average and wrong for a person. So an impact assessment asks two questions. Is the result fair for each group? And how big can one person’s error be, and how does that person get it corrected? The second one belongs in sections 4 and 5 of the template.
Text description of the diagram
On the left, the country: the A* share was 7.7% in 2019, teachers proposed 13.9%, and the grades issued gave 8.9%, so the average looks right. On the right, one constructed student: her teacher proposed A*, and she is ranked 1 of 20. Her school had no A* in the subject in past years, so the model predicts 0 A* and 4 A. She gets an A, whatever she did.
The student is constructed, but the rule is the model’s own. With 20 students, the school’s history decides, and the ranking only chooses who gets the grades that the history allows.
How accurate the model was for one student
Section titled “How accurate the model was for one student”The regulator tested its candidate models on 2019 students, whose real exam grades were known. Its key measure was “the probability of the predicted grade matching the grade actually achieved by the student” (interim report, p. 49). Table E.8 gives that measure for the chosen model, in every A level subject.
Text description of the chart
Two rows of 55 dots, one dot per A level subject. Exact grade: from 26.7% (Italian) to 67.9% (History), median about 54%. No subject reached 70%. Within one grade: from 64.0% to 98.7%; 41 of 55 subjects at 90% or more.
What this tells us. Within one grade, the model looked good: 41 of 55 subjects at 90% or more. But for the exact grade, it was right for 26.7% to 67.9% of students, depending on the subject. In Mathematics, the largest subject, it missed the real grade for about 4 students in 10. These tests were run before any grade was issued.
Read it fairly, too. Ordinary exams are not perfect either: the report compares its accuracy with how consistent marking is (executive summary, p. 5). And the test ranked students by their real 2019 marks (p. 49), which teachers’ ranks may not match. But the point stands: the system’s own test said that many individual grades would be wrong, and it did not say which.
Whether some schools gained more
Section titled “Whether some schools gained more”Section 5 of the template asks whether the result differs between groups. The report checks two kinds.
For groups of students, it found “no evidence that this year’s process of awarding grades has introduced bias” (executive summary, p. 5). Annex Q compared students of lower and higher socio-economic status: “The levels of adjustment are similar across groups” (Annex Q, p. 1).
For types of school, Table 9.10 (p. 136) gives the change in A level grades at A and above, in percentage points:
| Type of school | 2019 to 2020 | 2018 to 2019 |
|---|---|---|
| Secondary comprehensive | +2.0 | −1.5 |
| Secondary selective | +1.2 | −1.5 |
| Independent | +4.7 | −1.9 |
| Sixth form, further education, tertiary | +0.3 | −1.1 |
| Academy | +1.7 | −1.3 |
| Other | +3.6 | +0.8 |
What this tells us. Every type of school went up, by 2.4 points nationally (executive summary, p. 4). But independent schools went up by 4.7 points, and sixth form and further education colleges by 0.3. The gap between the highest and the lowest type was 4.4 points; from 2018 to 2019 it was 2.7 (our calculation). The report’s reading: “This reflects differences in a typical year.” Another reader can disagree. So an impact assessment writes down in advance how big a difference is too big.
The baseline: the model decides, as launched
Section titled “The baseline: the model decides, as launched”Compare every option with what was launched, not with perfection:
| The model decides (as launched) | Value | Source |
|---|---|---|
| Teachers’ grades lowered | about 39 in 100 | Table 9.8 (our sum) |
| Lowered by two grades or more | 3.5 in 100, about 25,000 grades | Table 9.8 (our calculation) |
| A* share | 8.9%, against 7.7% in 2019 | Table 9.6; executive summary, p. 2 |
| Exact grade in the test, by subject | 26.7% to 67.9% | Table E.8 |
| What a student receives | the calculated grade | DfE news story |
| How a student gets it corrected | an appeal, sent by the school | Chair’s statement |
After the switch, the Chair of Ofqual wrote: “Expecting schools to submit appeals where grades were incorrect placed a burden on teachers”. The Department for Education quoted the Education Secretary: the process had “more inconsistency and unfairness than can be reasonably resolved through an appeals process”.
The options
Section titled “The options”A fair assessment compares real options, each on the table for a reason. Options C and D are this case study’s way to frame the choice before launch. C is what happened after the switch.
| Option | What it means | Why it is on the table |
|---|---|---|
| A. Teachers’ grades as sent | Issue every CAG | Teachers know their students; no school history decides |
| B. The model decides | The design as launched | Results in line with earlier years, one method for every school |
| C. The higher of the two | The model’s grade or the teacher’s, “whichever is higher” | Nobody is lowered by a model; the model can still raise a grade |
| D. The model flags, a person decides | The model marks schools whose grades are far above their history; the exam board reviews them with the school before results day | Keeps a check on optimistic grades, with a person and the school’s evidence |
Option D changes the autonomy score from 3 to 2. The sum becomes 10, and the tier stays high. Here the harm and the people keep it high. What changes is who decides: a person can stop a wrong grade before it reaches a student.
One measure for every option
Section titled “One measure for every option”A fair comparison judges every option by the same rule. The measure is the harm that the tier names first. How many teachers’ grades, per 100, does a model lower with no person’s review before results day? The last column shows what each option costs on the other aim: results in line with earlier years.
| Option | Lowered with no person’s review, per 100 grades | A* share (7.7% in 2019) |
|---|---|---|
| A. Teachers’ grades | 0 | 13.9% (as sent) |
| B. The model decides | about 39 | 8.9% |
| C. The higher of the two | 0 | at least 13.9% |
| D. The model flags, a person decides | 0 (every lowered grade had a review) | between 8.9% and 13.9%: not known |
What this tells us. Only option B lowers grades with nobody looking. The others cost something else. A and C accept a rise in results bigger than “we have ever seen in a single year” (executive summary, p. 2). Somebody must accept it by name. D costs people’s time before results day, and nobody measured whether there was time. The assessment states this trade-off instead of hiding it.
The recommendation
Section titled “The recommendation”Before you read on, write your own decision in one line, with its reason.
Option B, the model decides: no go as designed. Its harm is high and hard to undo. Its own test missed the exact grade for many students, and it did not say which ones. Its only redress was an appeal that schools had to send.
Option D: go with conditions, if the reviews can end before results day. The conditions: publish the method and the group results before results day. Give each student a direct, fast appeal route. Set the two stop lines below. If the reviews cannot end in time, choose C. Then the people who asked for results in line with earlier years accept the higher grades by name.
This is one defensible decision from the evidence of the time. Another reader can choose A, if they argue it from the same evidence.
End to end on new input: test the stop lines
Section titled “End to end on new input: test the stop lines”A stop line is useful only if it is written before the results and checked on them. The timeline shows when each step was possible.
Text description of the diagram
Before results day: exams cancelled on 18 March 2020; models tested and one chosen in April and May; over 5 million teacher grades and ranks in early June; the grade data reached the regulator about a week before results. Then A level results and the interim report on 13 August, the switch on 17 August, and GCSE results on 20 August. The assessment and its stop lines belong in April and May. The check on the real data belongs in the week before results.
Write two lines in April and May, from the design and from past years only. Both are this case study’s choices:
- Line 1, people: if the model lowers more than 10 in 100 teachers’ grades, and students cannot appeal directly before university places are confirmed, stop.
- Line 2, groups: if the types of school differ more than in the last normal year, stop and explain before results day. The measure is the change at grade A and above. The last normal year is in Table 9.10’s 2018-to-2019 column: a gap of 2.7 points.
Now check them on the new input: the 2020 data, a week before results day.
| Line | Limit, written before | The 2020 data | Stop? |
|---|---|---|---|
| 1. Lowered grades, no direct appeal | more than 10 in 100 | about 39 in 100; appeals through schools only | yes |
| 2. Gap between types of school | more than 2.7 points | 4.4 points | yes |
What this tells us. Both lines fire about a week before results day, on data that the regulator already had. With the options written down in advance, a stop leads to a calm choice, not to a switch four days after results. Two limits remain. The lines are our choices: with a limit of 5 points, line 2 would stay quiet. And we know the end of the story, so any line looks wise now. That is why lines are written before the data, and kept.
Insights and limits
Section titled “Insights and limits”- The evidence of harm was in the system’s own report. The test results, the grades lowered and the gap between types of school all existed before results day.
- Right on average, wrong for a person. The national shares and the group checks looked fair. The harm was in single students’ grades. Larkfield saw the same pattern. In the MLOps course, the dashboard showed 100% of tickets accepted, while 10 tickets that needed a person reached nobody (recorded, MLOps and LLMOps: Operating AI in Production).
- Redress is part of the design, not the last step. No aim of the process said how a student gets a wrong grade corrected. When the appeals could not cope, the whole system stopped.
- A high tier does not mean “no”. It means the strongest controls, and a person who decides where a wrong result is hard to undo.
- Limits. We read one interim report and two statements, not the later reviews. Hindsight makes every decision look easy. The model’s builders worked fast, in a pandemic, under a direction to keep results in line with earlier years.
Responsible use
Section titled “Responsible use”- Keep the labels. Every figure names its document, table and page. Write “our calculation” for a sum, and “constructed” for an example student.
- Acknowledge the licence with the statement from the source card, and use no logos.
- Judge the system, not the people. Name decisions and evidence, not blame.
- No legal conclusions. This page does not say whether any law was broken. At work, ask your lawyer which rules apply to your own systems.
- Never use real students. Do not look for, or write about, real students or schools.
Your task: write the impact assessment
Section titled “Your task: write the impact assessment”- Copy the impact-assessment template into your document tool. Write the date as “before results day, summer 2020”.
- In section 1, name the system, the decision being assessed and the accountable owner who signs.
- In section 2, write what the model does alone, and what it must never be used for.
- In section 3, list the groups affected. Mark who is more exposed, and why.
- In section 4, write the benefit and the harms. Give each harm its figure, table and page.
- In section 5, write the group checks, the error for one person, and the redress route with its speed.
- In section 6, give each harm a control, an owner and the evidence that it works.
- In section 7, write the decision, its conditions and your two stop lines. Say what would change the decision.
Check your page against the case-study README. A good assessment uses only evidence from before results day. It names the error for one student, not only the averages. It sets stop lines in advance, and it names who accepts each trade-off.
Show one possible answer
This is one defensible assessment, not the only one. All figures come from the official documents on GOV.UK (Open Government Licence v3.0).
| Section | One possible answer |
|---|---|
| 1. Scope | The Direct Centre Performance model (DCP), used to adjust A level and AS centre assessment grades for summer 2020. Accountable owner: the regulator’s board, which signs; the legal specialist reads it first (both questions before scoring are yes). Next review: a week before results, on the real data, and again after the first appeals. |
| 2. The system | It predicts a grade distribution for each school or college from “the historical performance of the school or college in that subject” and fills it with the teachers’ rank order. Groups under 15 rely more on the teachers’ grades. It acts alone: the calculated grade is the grade issued. Not for: judging an individual student’s work. Foreseeable misuse: treating a school’s past results as a student’s ability. |
| 3. People affected | Every student with a grade (over 5 million CAGs and rank order positions), their schools, and the universities and employers who use the grades. More exposed: high-achieving students in schools with weaker past results, and students in groups of 15 or more, where history weighs most. |
| 4. Benefits and harms | Benefit: grades in line with earlier years (teachers’ grades were “in general, optimistic”: A* 7.7% in 2019 against 13.9% proposed). Harm: a student gets a lower grade than the teacher proposed because of the school’s history, with university places at stake. Evidence: in the test on 2019 students, the exact grade was right for 26.7% to 67.9% of students, by subject (Table E.8). In 2020, about 39% of A level teachers’ grades were lowered, 3.5% by two grades or more (Table 9.8, our sums). Hard to undo: university places are confirmed by a deadline. |
| 5. Fairness, rights, redress | Groups of students: no evidence of bias (executive summary, p. 5; Annex Q). Types of school: the gap between them was 4.4 points against 2.7 from 2018 to 2019 (Table 9.10, our calculation): explain it before results day. One student: the error can be one grade or more, and nobody knows for whom. Redress: each student must see why the grade changed and appeal directly, with a decision before university places are confirmed. |
| 6. Controls | Publish the method and the group results before results day. A direct, fast appeal route that can restore the teacher’s grade. A person at the exam board reviews schools flagged by the model, with the school’s evidence. Two stop lines, written in April and May. A fallback chosen in advance (the higher of the two grades). |
| 7. Decision | No go as designed. Go with conditions as option D, if the reviews end before results day; otherwise option C, with the higher grades accepted by name by those who set the direction. Stop lines: more than 10 in 100 grades lowered with no direct appeal; a gap between types of school larger than from 2018 to 2019. Both fired on the data a week before results. What would change it: evidence that the adjustment did not depend on the school, and an appeal route tested at the scale of the lowered grades. |
What happened (for the discussion): on 17 August 2020, the regulator said “The switch to centre assessment grades will apply to both AS and A levels and to the GCSE results which students will receive later this week.” The department said “the algorithm has revealed a number of anomalies that had not been anticipated by Ofqual and which severely undermined confidence in the system”. The lesson for governance: the evidence of harm was in the system’s own report before launch. An impact assessment asks the question that turns that evidence into a decision.
Common mistake
Only the averages were checked
Symptom: the assessment says “no evidence of bias” and “96.4% within one grade”, and it is signed. On results day, many students are lowered by a grade that their own work did not decide.
Cause: every check measured a group or a national share. No check asked how big one person’s error could be, or how that person would get it corrected.
Fix: in section 4, give the error for one person with its evidence: the exact-grade test, not only “within one grade”. In section 5, write the redress route and its speed. In section 7, add a stop line on people, not only on groups.
Make it a portfolio piece
Section titled “Make it a portfolio piece”This piece shows an employer that you can turn an official report into a decision: useful for risk, compliance, audit, policy and product roles.
What to publish:
- A one-page memo, “Before we issue grades by model”, with your impact assessment as an appendix.
- The official documents as links, with their tables and pages, and the licence statement from the source card.
- A line that says the Larkfield request and the example student are constructed for a course.
Make it yours with one change. Set your own stop lines, and show what moves when you change the limit of line 2. Or apply the seven sections to a system at your work, without its confidential details.
How to describe it in an interview, in two sentences: “I wrote the impact assessment that England’s 2020 grading model needed, from the evidence that existed before results day. It showed a model right on average but often wrong for one student, and two stop lines that fired a week before launch.”
Knowledge check
Section titled “Knowledge check”Check your understanding
The report says the model put 96.4% of A level grades within one grade of the teacher's grade, and found no evidence of bias between groups of students. Why is that not enough for section 4 of the impact assessment?
Show the answer
Both are checks of averages and groups. They do not say how big one student’s error can be. The model’s own test on 2019 students found the exact grade for only 26.7% to 67.9% of students, by subject (Table E.8). And about 39 in 100 teachers’ grades were lowered (Table 9.8, our sum). Section 4 must give the harm for one person, with its evidence. Section 5 must say how that person gets a wrong grade corrected, and how fast.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Score the grading model with the course's method. What is the tier, and does it change if the model only flags schools and a person at the exam board decides?
Show the answer
Harm 3: access to education, hard to undo. People 3: a decision about each person’s access. Autonomy 3: the calculated grade is the grade issued. Data 2: personal data. The sum is 11: high, and the second rule agrees. Both questions before scoring are yes, so the legal specialist reads it first. If a person decides, autonomy becomes 2 and the sum is 10: still high. The tier stays, but fewer wrong grades reach students, because a person checks each change with the school.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Check your understanding
Your stop line 2 says: stop if the gap between types of school is larger than in the last normal year. Why must you write it, and its limit, before you see the 2020 data?
Show the answer
Because after the data arrives, any gap can be explained away. The report read a gap of 4.4 points as “differences in a typical year”, while the gap from 2018 to 2019 was 2.7. A limit written in advance turns that reading into a test that anyone can check. With a limit of 5, the line would stay quiet. So the choice of limit is part of the decision, made before the result can influence it.
Does your answer have the main points of ours?
Sign in and enroll in the course to save your answers. Every check counts toward the certificate.
Recap
- You read three official documents as evidence, and checked what their numbers count: grades, not students, from data a week before results day.
- The course’s method puts the grading model at high (11), and both questions before scoring send it to the legal specialist.
- You learned that a system can be right on average and wrong for a person. In its own test, the model got the exact grade for 26.7% to 67.9% of students.
- One measure, grades lowered with no person’s review, showed that only the launched design lowered grades with nobody looking.
- Two stop lines, written in advance, fired on the 2020 data about a week before results day.
You can now write the impact assessment that a public system needed. But one system at a time is slow, and governments publish whole registers of their assessments. Next, you build an inventory from a published register and check its tiers.