Skip to content
Français

Measure governance

  • Lesson 19
  • 30 min
  • Module 6: Run governance

Where we are

In the last two lessons, we set up a board and handled an incident and a complaint. But nobody can yet say whether this work reduces risk, or only adds forms and waiting. In this lesson, we build a governance scorecard and learn the signs of governance theatre, the way a doctor reads several results at a check-up, not just one.

Plays from YouTube. Nothing loads from YouTube until you press play. Privacy

What you will be able to do

Show with a few numbers whether governance helps: are systems known and owned, are low-risk decisions fast, and is risk going down? Then find the work that only looks like governance.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

What you need

At a yearly check-up, a doctor does not read only your weight. Weight can fall because you are fit or because you are ill. So the doctor reads several results together: weight, blood pressure, how you sleep. Each result has a range that is fine and a range that needs a question.

Now think of a teacher who gives every student an A. The marks look perfect. But they tell nobody who has learned the subject, because the test never says no.

After its first quarter, Larkfield’s board (constructed) has decided 11 of 12 requests. The minutes are on time and every decision has a record. Everybody feels the board works hard.

Then the arguments start. Omar says that the board is slow. Farida says that it keeps Larkfield safe. Both are opinions, and neither can win, because nobody measures anything. And the last lesson showed a worse case: 212 approvals in a week, 0 of them opened. Every record looked correct. The records proved clicks, not reading.

The idea: a scorecard, and the signs of theatre

Section titled “The idea: a scorecard, and the signs of theatre”

A governance scorecard is a short set of measures, read together, that shows whether governance helps. Each measure has a good sign and a warning sign, as each result at the check-up has a normal range. The course’s template has eight measures: inventory coverage, systems with a named owner, time to decision by tier and path, decisions that are not a plain yes, overdue reviews, open risks, exceptions, and staff trained.

The opposite of a scorecard that helps is governance theatre: work that looks like governance, but changes no decision and no risk. The name compares it to a stage play. Everything looks real, but it is a performance for an audience. Two signs are common: forms that nobody reads, and approvals that never say no.

Read the measures together, because one measure alone can be gamed. If the board must say no more often, it can return requests for small reasons. If decisions must be fast, it can approve without reading.

Farida starts with the measure that she and Omar argue about. She takes the “Days” column of the log and groups it by path:

  • Fast path: R-07, 2 days. Median: 2.
  • Owner: R-08 (1) and R-10 (3). Median: 2.
  • Board, all 8 requests: 6, 7, 9, 15, 16, 17, 18, 21. The two middle values are 15 and 16, so the median is 15.5.

Omar is right: 15.5 working days is slow, and the terms of reference allow 10. But a median can hide two groups. So Farida splits the board’s row by tier:

Every decided request of the board's first quarter, by path and tier (constructed situation, calculated). The board was fast for high tier and slowest for low tier.
Text description of the chart

One row of dots per group, each dot one request. Fast path, low tier: 2 days. Owner, medium tier: 1 and 3 days, median 2. Board, high tier: 6 and 7 days, median 6.5. Board, medium tier: 9 days. Board, not an AI use: 15 days. Board, low tier: 18, 21, 17, 16 days, median 17.5.

What this tells us. The board decided its two high-tier requests in a median of 6.5 days. It decided its four low-tier requests in a median of 17.5 days, and approved all 4 with no change. Five requests went over the 10-day limit: R-01, R-03, R-05, R-09, R-11. Every one of them is low tier or not an AI use at all. So the slowest decisions were the ones with the least risk, and the wait changed none of them.

The second measure answers Farida. Of 11 decisions, 3 were not a plain yes: R-02 got conditions, R-04 went back, and R-06 was rejected. So the board does say no, and for reasons that the records show.

Farida fills in the other measures from the registry extract, the system cards and the board’s own lists.

Larkfield's scorecard after the board's first quarter, in four groups. Known and owned: inventory coverage 4 of 8 systems in the registry; 5 systems without a named accountable owner. Decisions: median days to decide, fast path 2, owner 2, board 15.5, low tier at the board 17.5; not a plain yes, 3 of 11 decisions, marked as a good sign. Risk under control: 1 overdue review; 9 open risks, 3 without an owner; 4 exceptions open, 2 past their end. People: 42% of staff trained on acceptable use. Every measure except 'not a plain yes' is marked as a warning sign.
Larkfield's governance scorecard after one quarter (constructed situation, calculated). One good sign, seven warnings.
Text description of the diagram

The scorecard has four groups. Known and owned: 4 of 8 systems are in the registry, and 5 systems have no named accountable owner. Decisions: the median days to decide are 2 on the fast path, 2 for the owner, 15.5 at the board and 17.5 for low tier at the board. Decisions that are not a plain yes: 3 of 11, a good sign. Risk under control: 1 overdue review, 9 open risks with 3 without an owner, and 4 exceptions open with 2 past their end. People: 42% of staff trained on acceptable use. Every measure except “not a plain yes” is a warning sign.

One value needs a word. In Module 1, you named Grace as the accountable owner of the refund workflow. But its system card still says “Support team”, so the count is still 5. A scorecard counts records, not intentions. The fix is to change the card.

Each sign of theatre has a measure that makes it visible. Larkfield has evidence for three of them.

Four signs of governance theatre, each with what makes it visible. Forms nobody reads: measure approvals opened before approving; pilot week, 0 of 212. Approvals that never say no: measure decisions that are not a plain yes; first quarter, 3 of 11. A wait that buys nothing: measure days for low tier at the board and the changes made; 17.5 days, 4 of 4 unchanged. A scorecard nobody acts on: rule, each warning gets an owner and an action at the next meeting.
Four signs of governance theatre and what makes each one visible. The approvals in the pilot were theatre; the board's decisions were not.
Text description of the diagram

Forms nobody reads: the measure is approvals opened before approving, and the pilot week had 0 of 212. Approvals that never say no: the measure is decisions that are not a plain yes, and the first quarter had 3 of 11, which is healthy. A wait that buys nothing: the measure is days for low tier at the board, with the changes made. It was 17.5 days, and 4 of 4 were unchanged. A scorecard nobody acts on: the rule is that each warning gets an owner and an action at the next meeting.

Farida brings the scorecard to item 4 of the next meeting.

  • Omar: “The median is 15.5 days. Let’s scrap the board and let teams decide.”
  • Tomás: “A slow board is a careful board. Let’s add a form for every request.”
  • Kwame: “High tier waited 6.5 days. The board is fast where it matters.”
  • Sam: “Which number should we change first?”

Before you read on, answer these:

  1. Which measure shows that governance slows teams without reducing risk?
  2. What should change, and how will the next quarter show that it worked?
  3. What is wrong with Omar’s and Tomás’s proposals?

The course’s answer. Kwame found the key. The measure is the time to decision for low-tier requests sent to the board. It was a median of 17.5 working days for 4 requests, against 2 on the fast path. All 4 were approved with no change, so the wait bought no risk reduction. It did not even buy a careful reading. R-01, the meeting-notes tool for customer calls, waited 18 days and was approved with no conditions, before anyone read the answers of MinuteMoss (made up for this course). Vendor and model checks sends it back with conditions.

The change: low tier goes to the fast path by rule, and the board sees it only as a list. A request that is not about AI goes back within a day. Next quarter, the low-tier median should be about 2 days, and the share of decisions that are not a plain yes should stay above zero.

Omar would remove the part that works: the board rejected version 3 of the triage prompt and sent back the reply suggester. Tomás would add a form, which slows every request and reduces no risk. More forms that nobody reads is the theatre that INC-07 showed.

What would change the answer: if the board had changed or refused some low-tier requests, the wait would have bought something. Then the problem would be the tier method: those requests were not really low.

Fill in the scorecard once a quarter, from records that already exist: the board’s log, the registry and the system cards. It takes Farida about an hour. Do not add a measure that needs a new form. And do not turn one measure into a target alone; read the eight together, and bring each warning to the board with an owner.

Common mistake

A scorecard that is always green

Symptom: every measure is in its good range, quarter after quarter, while incidents still happen.

Cause: the measures count what is easy to count: forms filed, meetings held, approvals given. None of them can show a problem.

Fix: keep only measures with a warning sign that you could actually see. Two examples are approvals opened before approving and decisions that are not a plain yes. Check each measure against the last incident: would it have shown it?

  1. Open the first-quarter log CSV in a spreadsheet.

  2. Filter the rows where “path” is “board”.

  3. Compute the median of the “days” column.

    Check: 15.5.

  4. Filter again: “path” is “board” and “tier” is “low”.

  5. Compute the median of “days”, and count the rows where “decision” is “approved”.

    Check: 17.5 days, and 4 of 4 approved.

  6. Remove the filters. Count the decided rows whose decision is not “approved”.

    Check: 3 of 11. R-12 is open, so it is not counted.

For each warning sign on Larkfield’s scorecard, write one action, its owner and the value you expect next quarter.

Show one possible answer
  • Inventory coverage, 4 of 8: register the digest script (or stop it) and the three bought features (Kwame, Farida). Expect 8 of 8.
  • 5 systems without a named owner: change the five cards to a named person; the board confirms (Nadia). Expect 0.
  • Low tier at the board: the fast path by rule (Farida). Expect about 2 days.
  • 1 overdue review: the document assistant’s review (Mei, its owner). Expect 0.
  • 3 risks without an owner, 2 exceptions past their end: assign or close each one at the next meeting (Nadia). Expect 0 and 0.
  • 42% of staff trained: short training on the acceptable-use policy for every team (Farida). Expect most staff trained.
  • Keep: decisions that are not a plain yes above zero, and a new measure from INC-07, approvals opened before approving.

Check your understanding

Why is 'decisions that are not a plain yes: 0' a warning sign? And why is a high share not good by itself?

Show the answer

A board that never says no is probably not reading: its approvals are theatre, like the teacher who gives every student an A. But a high share is not good by itself either. A board can return requests for small reasons to look strict, and teams then wait for nothing. Read it with the time to decision and the reasons in the records.

Check your understanding

Predict: next quarter, low-tier requests take the fast path by rule. What happens to the board's median time to decision, and which measure do you watch to be sure that risk does not go up?

Show the answer

The board’s median falls, because only high-tier, vendor and exception requests remain. In the first quarter, those took 6, 7, 9 days. The low-tier median falls to about 2 days. To check that risk does not go up, watch the incidents and samples for low-tier systems. Also watch that the board’s share of decisions that are not a plain yes stays above zero.

Check your understanding

Module completion check: name one measure that shows governance is slowing teams without reducing risk, and what to change.

Show the answer

The time to decision for low-tier requests sent to the board. It was a median of 17.5 working days for 4 requests, against 2 on the fast path. All 4 were approved with no change, so the wait bought no risk reduction. One more request at the board was not about AI. Change: low-tier requests go to the fast path by rule, and the board sees them only as a list. A request that is not about AI goes back within a day. Next quarter, expect about 2 days for low tier. Also check that the board still says something other than a plain yes.

Recap

  • A governance scorecard is a short set of measures read together: inventory coverage, named owners, time to decision by tier and path, decisions that are not a plain yes, overdue reviews, open risks, exceptions and staff trained.
  • Split a median before you trust it: Larkfield’s board was fast for high tier and slowest for low tier.
  • Governance theatre looks like governance but changes no decision and no risk: forms nobody reads, approvals that never say no, waits that buy nothing.
  • Each warning on the scorecard gets an owner and an action at the next meeting.

In this course, you built Larkfield’s governance: owners, rules, an inventory with tiers, policies, controls, evidence, a board, an incident process and a scorecard. But Larkfield is constructed. Next, you test these ideas on a real case: a tribunal held a company responsible for what its chatbot told a customer.

Next: A tribunal case about a chatbot's answer