Evidence over self-ratings

What you will be able to do

Turn your skills inventory from “what I think” into “what I can show”. Employers trust evidence, and evidence also shows you exactly what to learn first.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• The five areas and the three levels, from Skills inventory: Five areas, not one and Skills inventory: Three levels.

What you need

• Your inventory, under ## Skills inventory in notes.md, from Skills inventory: Guided practice: your first inventory. You copy it in this lesson.
• Your domain card, under ## Domain card, from Domain knowledge in practice: Guided practice: write your domain card.
• For task 1, a spreadsheet program, such as Google Sheets, Microsoft Excel or LibreOffice Calc. If you can use Python or SQL, you can use them instead.
• Do each task without help from a person or an AI assistant. The aim is to measure, not to get the correct answer.

Two kinds of evidence

A self-rating is a guess. Evidence is something that another person can check. There are two kinds:

1. A small diagnostic task: a short task that you do now, with a known correct answer. It shows what you can do today.
2. Existing work: things that you made before, such as a report, a spreadsheet, a project plan, code or a presentation. It shows what you have done in real conditions.

For each area of your inventory, you want at least one of the two. Diagnostic tasks are quick and fair. Existing work is stronger, because other people used it.

Five diagnostic tasks

Do each task in 10 to 15 minutes. Write your answers in notes.md under ## Diagnostics. Check your answers only after you finish all five.

Task 1: programming and data tools

Copy this data into a spreadsheet. It is ten support tickets from Larkfield. Each row has the ticket’s ID, its category, its priority and how many minutes the customer waited for the first reply.

id,category,priority,first_reply_minutes
T-2001,returns,2,35
T-2002,delivery,1,12
T-2003,returns,3,140
T-2004,payment,2,48
T-2005,delivery,2,20
T-2006,returns,1,15
T-2007,warranty,3,95
T-2008,delivery,2,30
T-2004,payment,2,48
T-2010,returns,2,60

If you paste it into a spreadsheet, all the text may go into column A. Then use the spreadsheet’s “split text to columns” command, with a comma as the separator.

1. One ticket is in the data twice. Which one? Remove the second copy.
2. After you remove it, how many tickets are there in each category?
3. What is the average (mean) first reply time? What is the median, the middle value when you sort them?

Task 2: mathematics and statistics

1. Use your answers to task 1. Why are the average and the median so different?
2. In task 1, “warranty” has the slowest first reply. Can you tell the support lead: “Warranty tickets are our slowest”? Why or why not?
3. Version A of the assistant answered 45 of 50 test questions correctly. Version B answered 88 of 100 correctly. Is version A better?

Task 3: domain knowledge

Explain one rule from your field to a person who has never worked in it, in five sentences or fewer. Include two exceptions to the rule, and why each exception exists.

Task 4: communication

Ruth, the director at Larkfield, has two minutes. Write her a message of five sentences or fewer about the evaluation in week 5 of One project, many roles. The results: 106 of 120 answers were fully correct, and 5 answers wrongly promised a refund. Tell her what the team recommends.

Task 5: project delivery

Describe one project that you finished with other people, in five lines: the goal, your part, a problem on the way, what you did about it, and the result. The project can be from work, university or a community group.

Check your answers

Task 1 answers

1. T-2004 is in the data twice. After you remove one copy, there are 9 tickets.
2. returns: 4, delivery: 3, payment: 1, warranty: 1.
3. The average is 455 ÷ 9 = 50.6 minutes. The median is 35 minutes. The sorted values are 12, 15, 20, 30, 35, 48, 60, 95, 140.

Your level: if all three are correct and you needed no help, your level for this area is at least developing. If you found the duplicate only by reading every row, you are at the start of developing. Spreadsheet tools such as “remove duplicates” and pivot tables do it faster and safely on 10,000 rows.

Task 2 answers

1. Two slow tickets, 95 and 140 minutes, pull the average up. The median does not change much when a few values are very large. For waiting times, the median is often closer to what a typical customer experiences.
2. No. There is only one warranty ticket. One slow ticket can be bad luck. You need many more warranty tickets before you say anything about warranty tickets in general.
3. Not clearly. A is 90% and B is 88%, a small difference. With only 50 and 100 questions, a difference this small can come from chance: one or two different questions could change it. Also, were the questions the same? If not, the results cannot be compared at all.

Your level: if you got all three, including the reason for questions 2 and 3, your level is at least developing. Questions 2 and 3 are about the most common statistical mistake in AI work: trusting differences that come from small samples.

Task 3: how to check

A good answer:

• explains the rule in plain words, without the jargon of the field, or with each term explained;
• gives two real exceptions;
• says why each exception exists, for example a law, a risk or a customer need.

If you did all three easily, and could give five more exceptions, your domain knowledge is probably demonstrated. That is the knowledge that AI teams need from you.

Task 4: an example answer and how to check

In our test of 120 real questions, the assistant gave a correct answer with the correct source for 106. Five answers wrongly promised a refund that the policy does not allow. We recommend no pilot until those errors are fixed. We expect the fix in about one week, and we will test all 120 questions again. Then we will ask for your decision on a pilot with 10 agents.

A good answer: starts with the result, says the serious problem clearly, gives one recommendation, and says what happens next. It uses no technical words. It does not hide the bad news, and it does not make it sound worse than it is.

Task 5: how to check

A good answer has all five parts. Your part is clearly yours: “I planned the training”, not “we did everything”. The problem is real, and the result has a fact or a number. If your project had all of this, and you led it or co-ordinated it, your level is demonstrated.

Add your existing work

Now look for existing work for each area. Here are examples of things that count:

Area	Examples of existing work
Programming and data tools	A spreadsheet that your team uses, SQL queries, a script, a project on GitHub
Mathematics and statistics	An analysis that changed a decision, a report with statistics that you calculated
Domain knowledge	Years in the role, training that you gave, procedures that you wrote, a professional qualification
Communication	Reports, presentations, documentation, articles, training materials
Project delivery	A project that you led or co-ordinated, with its result

Some work is confidential, such as client documents or company data. You do not need to show it. You need to describe it accurately, and be able to explain it in an interview.

Worked example: Daniela updates her inventory

Daniela, the accountant, did the five tasks. Her results:

• Task 1: correct in 6 minutes, with a pivot table. Developing, as she guessed.
• Task 2: questions 1 and 3 correct. For question 2, she said “yes, warranty is slowest”. She missed the sample size. Developing. Her gap is now specific: statistics for comparing groups and samples.
• Tasks 3, 4 and 5: all strong.

Then she added her existing work. Her inventory now has an evidence column:

Area	Level	Evidence
Programming and data tools	Developing	Diagnostic task 1. Monthly close spreadsheet used by 5 people.
Mathematics and statistics	Developing	Diagnostic task 2. Gap: samples and comparisons.
Domain knowledge	Demonstrated	7 years of accounts. Trains new staff. Wrote the invoice checking procedure.
Communication	Demonstrated	Monthly results report for the finance director. Diagnostic task 4.
Project delivery	Demonstrated	Led the move to a new accounting system: 3 months, 5 people, finished on time.

Before, Daniela had a vague feeling: “I am weak at the technical parts.” Now she has two specific gaps: programming beyond spreadsheets, and statistics for comparisons. A specific gap can go into a study plan. A vague feeling cannot.

Guided practice: update your inventory

1. Copy your inventory from ## Skills inventory to a new heading: ## Inventory with evidence.
2. Add a column with the name Evidence.
3. For each area, write the diagnostic task result and at least one piece of existing work.
4. If the evidence does not support your level, change the level.
5. Under the table, write your two most important gaps as specific sentences, like Daniela’s.

Check: every level has evidence next to it. Every gap names a specific skill, such as “comparing groups with small samples”, not “maths”.

Common mistake

A certificate as the only evidence

Symptom: your evidence column says “Completed Python course”, “Completed machine learning course” and “Certificate in data science”. In an interview, the question “Show me something that you built” gets no good answer.

Cause: a certificate shows that you finished a course. It does not show what you can do. Employers know this. A certificate never guarantees a job, including this platform’s certificates.

Fix: after each course, keep the thing that you made in it: the final project, with your own changes. Make that the evidence, and list the certificate next to it. In Module 5, you learn how to present this work.

Check your understanding

Why is a diagnostic task done without help, even from an AI assistant?

Show the answer

Because its purpose is to measure what you can do today, so you can plan what to learn. If an assistant does the task, you measure the assistant. You would then skip learning that you actually need, and the gap appears later, in an interview or on the job, when it costs more.

Check your understanding

A learner rates their statistics as 'developing', and gets all of task 2 correct, with reasons. Should they raise the level to 'demonstrated'?

Show the answer

Not yet. Task 2 supports “developing”: they understand the ideas. “Demonstrated” needs existing work: statistics that they used to make or check a real decision, which other people relied on. They should look for such work. If they have none, they can create it in a portfolio project.

Recap

• A self-rating is a guess. Evidence is a diagnostic task or existing work that another person can check.
• Small diagnostic tasks show what you can do today. Existing work shows what you did in real conditions.
• A specific gap, such as “comparing small samples”, can go into a study plan. A vague feeling cannot.
• A certificate shows that you finished a course. Keep what you made as the evidence.

Next, you use your evidence to choose a target role and one adjacent option.

Next: Choose a realistic route