Problem framing

What you will be able to do

Describe an AI project in a way that a whole team can agree on, before anybody chooses a tool. A good frame prevents the most expensive mistake in AI work: building a good solution to the wrong problem.

Before you startWhat to know and to have readyAbout 30 minutes

What you need to know

• You completed Module 2. You know the five parts of an AI product from AI in everyday products: Five parts of an AI product.
• Why AI needs domain knowledge, from Your experience is a head start: Why AI needs it: the data does not explain itself.

What you need

• Your notes.md file in the ai-career folder. You made it in AI in everyday products: Set up your course notes. If it is missing, make it again as in that section. This lesson adds the heading ## Problem frame.

The new problem at Larkfield

The document assistant works. Now Ruth, the director of customer service, has a new request: “Can AI sort our support tickets?”

Today, every customer message becomes a ticket in one shared queue. One co-ordinator reads each ticket and sends it to a team: returns, delivery, payment, warranty or account. The co-ordinator also sets a priority from 1 (urgent) to 3 (normal). About 1,200 tickets arrive each week. Sorting takes the co-ordinator about two hours every day.

“Sort our tickets with AI” is a wish, not a frame. It does not say who is helped, what decision changes, or what happens when the AI is wrong.

Four questions that frame a problem

1. Who is the user? The person who uses the result, and the people affected by it.
2. What decision does it improve? AI almost always helps somebody decide or act. Name the decision exactly.
3. What data exists? What is recorded today, how much, and how good is it?
4. What does each kind of error cost? An AI system will be wrong sometimes. Different mistakes cost different amounts.

Then add a fifth line: what does success look like, as a number?

Worked example: framing the ticket project

Omar, the product manager, asks the four questions with Grace, the support lead, and the co-ordinator.

1. Who is the user? The co-ordinator, who sorts the tickets. The teams, who receive them. The customers, who wait.

2. What decision does it improve? Two decisions for each ticket: which team gets it, and which priority it has.

3. What data exists? 18 months of tickets, about 90,000. Each one has the customer’s text, the team that finally handled it, and the priority. Grace adds what the numbers do not say. About 1 ticket in 10 was moved to a different team later, so the first team is not always correct. The final team is the better answer.

4. What does each kind of error cost? Grace and the co-ordinator describe the errors:

Error	What happens	Cost
Wrong team	The team sends the ticket back. The customer waits about 4 hours more.	Medium
Urgent ticket marked as normal	For example “charged twice” waits two days. The customer may complain or ask the bank to reverse the payment.	High
Normal ticket marked as urgent	An agent answers it earlier than needed.	Low

The table shows something important: the errors are not equal. Missing an urgent ticket is much worse than the opposite mistake. A system that is “90% correct” can still be bad if most of its 10% are missed urgent tickets.

5. Success as a number. Omar writes:

For the co-ordinator, when a new ticket arrives, suggest the team and the priority, using the text of the ticket and 18 months of past tickets with their final team. A wrong team costs a customer about 4 hours. A missed urgent ticket costs much more. Success: at least 90 of 100 tickets go to the correct team, at least 95 of 100 urgent tickets are marked urgent, and sorting time falls from 2 hours a day to under 30 minutes.

That one paragraph is the frame. Each part of it is something that the team can check.

Is AI needed at all?

Before the team builds anything, Grace asks a question that every frame needs: could we solve this without AI?

• A form: the contact form could ask the customer to choose a topic. Then many tickets arrive already sorted.
• A rule: every message with “charged twice” or “fraud” could be marked urgent by a simple rule.
• A process change: two people could share the sorting at the busiest hours.

These options are cheap, and they may be enough. In this case, the team decides to try the topic menu first, and to measure how often customers choose the wrong topic. AI is still possible later, for the tickets that the menu does not sort well. You see the next steps in the next lesson.

A reason not to use AI is a good result, not a failure. It saves money, and it is easier to explain and to fix.

Guided practice: frame a second problem

Here is a new request at Larkfield: “Use AI to find customers who are about to stop buying from us.”

1. In notes.md, add a heading: ## Problem frame.
2. Answer the four questions for this request. Where you do not know, write the question that you would ask, and whom you would ask.
3. Write an error table with at least two kinds of error and their cost.
4. Write one success measure as a number.
5. Write one way to solve the problem, or part of it, without AI.

Show an example answer

1. User: the customer team that sends offers. Also the customers who get them.
2. Decision: which customers get a personal offer this month.
3. Data: order history, returns, support tickets, email opens. Question for the data engineer: how far back do the orders go? Question for Grace: do complaints in tickets predict leaving?
4. Errors: a customer who would leave gets no offer → we lose them (high cost). A loyal customer gets an offer → we give a discount that was not needed (low to medium cost, but it adds up).
5. Success: of the customers who get an offer, more stay than in a similar group without the offer, measured after 3 months.
6. Without AI: a simple rule, such as “no order in 6 months and one complaint”, may find most of them.

Check: your decision is one specific action, such as “who gets an offer”, not a general aim such as “keep customers”.

Common mistake

Starting with a tool

Symptom: the first sentence of the project is “We will use a large language model for tickets” or “We need a neural network”. After two months, there is a demo, but nobody can say if it is better than before.

Cause: the team chose the solution before it framed the problem. Without a frame, there is no success measure, so no result can be judged.

Fix: stop, and answer the four questions. Write the success measure. Then ask whether a form, a rule or a process change can do it. Choose the tool last.

Check your understanding

Why does the frame use the final team of each past ticket, and not the first team?

Show the answer

About 1 ticket in 10 was moved later, so the first team was sometimes wrong. An AI that learns from the first team learns those mistakes. The final team is closer to the correct answer. Grace knew this from her work. It is not visible in the data unless somebody asks.

Check your understanding

A system sorts 92 of 100 tickets to the correct team. Is it good enough? What else must you know?

Show the answer

You cannot tell yet. You must know which 8 are wrong and what those errors cost. In particular: how many urgent tickets did it mark as normal? The frame says at least 95 of 100 urgent tickets must be marked urgent. You also need to know if sorting time fell, because that was the reason for the project.

Recap

• A frame answers four questions: the user, the decision, the data, and the cost of each kind of error. Then it adds a success measure as a number.
• Errors are not equal. The frame says which ones matter most.
• Always ask whether a form, a rule or a process change solves the problem without AI.
• Choose the tool last.

Next, you follow the ticket project from a simple baseline to a system that runs every day.

Next: From baseline to deployment