How the interviews run

Think about taking your car to a good mechanic. You say, “It makes a strange noise.” A bad mechanic changes three parts and sends you a bill. A good mechanic asks questions first: “When does it happen? Only when the engine is cold? Only when you turn left? Since when?” Each question comes from your last answer, and after a few minutes the mechanic knows where to look.

A data science interview at a serious company works in a similar way, in two directions. The interviewer asks you questions like a good mechanic: they listen, and every next question comes from your answer. And in the product case, you must be the mechanic: the company gives you a strange number, and the interviewer watches which questions you ask before you touch anything.

This page explains both, and the five interviews in Parlo’s process.

The five interviews at Parlo

Parlo’s posting does not describe its process. That is normal, so ask the recruiter at the start. Here is what a candidate hears in that first call. Each interview checks something different, and each one has its own rule about AI tools.

1. The recruiter call (30 minutes, Mira Okafor, Talent Partner). Mira checks the basics: does your experience match the level, what pay do you expect, when can you start, and can you explain your work simply. Prepare a one-minute introduction, your pay range, and your questions. This is the right moment to ask which tools the team really uses, and what the AI rule is in each round.

2. The SQL and analysis screen (45 minutes, one of the team’s data scientists). You share a code editor with the interviewer and work on a small set of tables: users, lessons, subscriptions. You write queries and talk while you write. An AI assistant is allowed in this round, and the rule is simple: you must explain everything it writes. The interviewer wants to see how you use it. Do you ask it a clear question? Do you read its answer line by line? Do you check the result, for example by counting rows before and after a join, or by testing the query on one user whose numbers you know? Slow and checked usually beats fast and unexplained.

3. The product case (60 minutes, video, Thandiwe Mokoena, Senior Data Scientist). This is the interview in Part 3 of this guide. Thandiwe has nine years of experience. She is warm and quick, she asks “why?” a lot, and she likes to draw small tables on the shared whiteboard. No AI tools in this round: it is just two people talking. The hour has a fixed shape:

Minutes	What happens
0–5	Introductions, the format, and the AI rule for the round
5–20	An analysis or experiment that you did that changed a decision, and then deeper questions about it
20–50	The Parlo case: a real situation from the team today
50–57	Your questions
57–60	Close and next steps

4. The take-home and the presentation (Simone Achterberg, Head of Data Science, and a product manager). You get a small dataset and a question, and you have three hours at most. AI tools are allowed, and you must declare them: say which tool you used, for which parts, and what you checked yourself. Then you present your results for 30 minutes. Simone checks your judgement: did you answer the real question, are your numbers right, and do you say clearly what the data cannot tell you? The product manager checks something else: could they act on your recommendation tomorrow? A time limit shows the team what you choose to do first.

5. Values and collaboration (45 minutes, Kenji Ito, Product Manager). Kenji is the product manager for the feature in the case. He asks how you work with people who are not data scientists: when a result is not what someone wanted, when a deadline is close and the data is not ready, when you made a mistake in a number that someone already used.

After the last interview, the interviewers meet in a debrief: each one shares their written notes, and together they decide. A strong result in one interview does not cancel a serious concern from another one.

Other kinds of interview that you may meet

Other companies use other rounds for Data Scientist roles. Which ones you meet depends a lot on the kind of job (see Read it honestly). You may meet:

• A statistics and experiments round: questions about sample size, confidence intervals, what to do when a test has two metrics that disagree, or why stopping a test early is a problem.
• SQL with no AI help: the same kind of queries as Parlo’s screen, but from your own head. Joins, dates, window functions, and retention or funnel queries are the most common.
• Python or ML coding: for ML-type roles. You clean a small dataset, build a simple model, and explain how you would know that it works.
• A metric-drop case: “Sign-ups fell 10% yesterday. What do you do?”
• A longer take-home, over a weekend or 48 hours. Ask what they expect, so that you do not spend 20 hours on it.

The AI rules are different in every company and often in every round. Some companies also use AI to record or summarise interviews, and say so in advance. If nobody tells you the rule, ask. Then follow it exactly (our advice).

How a serious interviewer goes deeper

The second part of the product case starts with an open question: “Tell me about an analysis or experiment that you did that changed a decision.” Then the interviewer follows your answer. Here is how one sentence can grow into fifteen minutes of conversation.

Suppose that a candidate says: “I ran an A/B test on our checkout page, and the new design increased conversion by 4%.” The interviewer goes down level by level:

From one sentence to the edge

1. 1

   “What decision did the test help to make, and who made it?”

   Is the work tied to a real decision? Many tests are run, and few change anything.

2. 2

   “How were users split between the two designs, and what was the main metric?”

   Did the candidate design the test, or only read the results page? Only the designer knows these details.

3. 3

   “How many users did you need, and how did you decide how long to run it?”

   Do they understand power? This is where many candidates start to give general answers.

4. 4

   “Is that 4% relative or in percentage points? What was the confidence interval?”

   Can they read a result honestly? A 4% increase of a 2% conversion rate is a very small change in absolute terms.

5. 5

   “Did anyone look at the results before the end? Did anything else change in those weeks?”

   Do they know the common traps: looking early, a sale or a season, a bug in one group?

6. 6

   “Six months later, is the 4% still there? How would you know?”

   The edge: long-term effects, and how a team checks them. Few candidates have done this.

The interviewer plans the areas to probe, not the exact questions. The questions come from the answers. A candidate who says “we ran it for two full weeks, because shopping is different at weekends” will get a question about weeks. A candidate who says “the platform told us it was significant” will get “What did the platform calculate, and what did it assume?”

So lists of interview questions help little. The questions come from your own work, and what you can do is know it very well: the decision, the numbers, the method, and what went wrong.

The edge of what you know

Every candidate reaches an edge. A good interviewer goes there on purpose, because the most useful information is there. They want to see what you do when you do not know:

• Do you say clearly that you do not know, or do you talk around it?
• Do you reason from what you do know, step by step, out loud?
• Do you ask a question that makes the problem clearer?
• Do you stay calm, and stay honest?

Here is a short example. The candidate mentioned that their team used a method called CUPED to make tests shorter.

An example: reaching the edge

Thandiwe(interviewer):

You said CUPED made your tests shorter. How does it do that?

Candidate(candidate):

Hmm. Okay, honestly, our platform did it. There was a checkbox. I know the idea more than the maths.

Thandiwe(interviewer):

That’s fine. Tell me the idea.

Candidate(candidate):

So, users are very different from each other. Some buy a lot, some almost nothing. This difference is noise in the test. CUPED uses data from before the test, like how much each user bought last month, to remove part of that noise. Less noise, so you need fewer users, or fewer days. I think that’s right. What I can’t tell you is the exact formula, or when it doesn’t help.

Thandiwe(interviewer):

Okay. When do you think it wouldn’t help?

Candidate(candidate):

Let me think… If there’s no data from before. Like new users. They have no history. So… for new users, it probably helps very little?

Why this works

The candidate does not pretend. They say where their knowledge stops, explain the idea that they do know in plain words, and then reason to a new point out loud. The last answer is a good guess, and they say it as a guess. It also matters at Parlo, where the case is about new users.

The Parlo case

In the main part of the product case, Thandiwe gives every candidate the same situation. It is real for the team today, and it is a big part of the reason they are hiring:

“Okay, now our real situation. Five weeks ago we launched ‘Talk with Pip’, AI conversation practice, to 20% of new users. Weekly active users in that group went up about 6%. But day-30 retention for new users went down from 31 to 29%. Our product manager wants to give it to everyone before January. If you were our data scientist, what would you do?”

Terms in the case:

• Talk with Pip: the learner speaks out loud with an AI tutor about everyday topics, such as ordering food or talking about the weekend.
• Weekly active users: how many people used the app in a week. The exact meaning of “used” is a choice that someone made.
• Day-30 retention: of the people who joined, the share who still use the app 30 days later.
• January: many people decide to learn a language as a New Year resolution, so January is the biggest month of the year for new users.

There is no single right answer. Thandiwe wants to see how you start. Which questions do you ask before you say anything about the result? Do you check whether the comparison is fair? What does “good” mean for Parlo? What would you recommend to the product manager this week, when the evidence is not complete? Thandiwe has more facts about the launch. She gives most of them only when a candidate asks the right question. Sometimes, late in the case, she gives one fact herself, to see what the candidate does with it.

Try it before you read the candidates

Take 25 minutes, and write or say out loud how you would approach the Parlo case. Answer these for yourself:

1. Which three questions would you ask Thandiwe before you say anything about the result?
2. “Down from 31 to 29%”: compared with which group of users, and over which dates? Why does it matter?
3. The feature launched five weeks ago. Which new users can already have a day-30 retention number? What does that tell you about the size and timing of the group?
4. Weekly active users went up, and retention went down. Can both be true? Think of a reason for each.
5. What does “good” mean for a language app: more activity, more learning, or more paying users? Which would you choose as the main metric, and which would you watch so that nothing important gets worse?
6. Each AI conversation costs Parlo money. What else do you want to know before you say “give it to everyone”?
7. Write what you would tell the product manager on Friday, in three or four sentences, with how sure you are.

Keep your notes, and compare your questions with the five candidates’ questions.

Check two numbers

Question 3. Day-30 retention needs 30 days after a user joins. After five weeks (35 days), only users who joined in roughly the first five days of the launch can have a day-30 number. So the 29% comes from a small group, and all of them joined in the same few days at the start of the launch. Both facts should make you careful.

Question 6. Parlo pays about CAD 0.04 per conversation. A heavy user has 20 or more conversations a week. That is 20 × 0.04 = CAD 0.80 a week, or about CAD 3.44 a month (0.80 × 4.3 weeks). A Parlo Plus subscription is CAD 14.99 a month, so for a paying learner the cost is clear but not huge. For a free learner, Parlo pays it and gets nothing back. So you want to know: who uses Pip the most, free or paying learners?

How to read the candidate pages

Each candidate page shows the same 60-minute product case with Thandiwe, as a conversation. In Artem’s interview, Kenji Ito, the product manager, joins for the last 15 minutes. Blue turns are the interviewer; green turns are the candidate. Between the turns, notes explain what is happening:

• What the interviewer is checking: the real question behind the question.
• How the interviewer goes deeper: why the next question follows from the answer.
• Why this works: what made an answer strong.
• Where this goes wrong: a weak moment, and why it is weak.
• A stronger answer: the same moment, answered better, in the candidate’s own words.
• Disagreeing well: how a candidate disagrees with an interviewer, with reasons.

Each page ends with Thandiwe’s written notes, what happened next, and advice for readers with a similar profile. Every candidate has strong moments and weak ones. Some candidates speak English as their second or third language, like many people in real interviews. Thandiwe listens for their thinking, not for perfect grammar.

The five candidates are:

1. The analyst: Chloe has strong SQL and about 30 A/B tests behind her, and is less sure about the statistics under them.
2. The economist: Rahul has a PhD and deep training in causal inference, and must learn to give a simple answer by Friday.
3. The career changer: Inés taught languages for eight years, then trained in data, and knows how people really learn.
4. The modeller: Artem builds excellent models, and reaches for one before he asks what the decision is.
5. The senior: Gabriel has nine years of experience, lost his job in a restructuring, and applies for a lower level.

Read them in any order. Start with the one that is most like you, and then read the one that is least like you.