The senior

Think of an experienced cook whose restaurant closes. For months, no head-cook job opens in the city. A smaller restaurant needs a line cook. The cook can do that job easily. The owner of the small restaurant is happy, and a little worried: “Will this person leave the first time a head-cook job opens?”

Many data scientists are in this position in 2026. Companies have cut teams, and senior people look for work for months. When the savings get smaller and a family depends on you, you start to apply for roles one level below your last job. That is a sensible decision. Our advice on not cheating says the same: apply widely, and include roles one step below your target.

Gabriel is this person. He is very good, and he needs a job. The technical part is not hard for him. The hard part is a question that every interviewer thinks about and few ask: “Will you stay?” This page shows how he answers it, and why the honest answer is also the best one.

GS

The candidate

Gabriel Santos

Senior data scientist, 9 years, applying for an intermediate role

• 9 years in data science
• Laid off in March 2026
• Experiments, pricing, marketplaces
• Toronto

• Senior Data Scientist, Forkline (food delivery, Canada), 2022 to March 2026 (role eliminated in a restructuring)
• Data Scientist, Forkline, 2019 to 2022
• Led the move from user-level A/B tests to switchback tests for pricing and fees; about 40 pricing tests a year
• Small-order fee: designed and analysed the test that changed the launch plan; about CAD 3 million a year in profit
• Mentored 4 data scientists; wrote the team’s guide to reading experiment results
• Data Analyst, Northvale Insurance (Toronto), 2018 to 2019; analyst at a bank in São Paulo, 2017
• Python, SQL, dbt, Airflow, Looker; BSc Statistics (Brazil)

What Thandiwe sees before the call

Thandiwe also has nine years of experience. She reads the CV and thinks: “He could do my job.” That is not a threat to her. It is a question for the team.

The notes from the SQL screen are very good. Gabriel used the AI assistant, which was allowed in that round, and said each time what he asked it to do. Once he rejected its query and explained why: it counted test accounts as learners. The colleague who ran the screen wrote: “Uses the tool like a senior. Checks everything.”

The gap since March is not a problem for her. Many good people were laid off this year. But she sees a senior title, and this role is intermediate.

Her plan for the call:

• Is he as strong as the CV says, or did a large team do the work around him?
• Can he work alone with a product manager? At Parlo, each data scientist owns one product area.
• Where is his edge? His experience is in delivery and pricing. Parlo is about learning.
• Will he stay? She decides to ask this kindly, near the end, and only once.

Terms on this page

• Day-30 retention: of the people who joined on a day, the share who still use the app 30 days later.
• Percentage rollout: a feature is switched on for a share of users, for example 20%. It uses the same tool as an experiment, but nobody decided the metric and the length before the start.
• Switchback test: instead of splitting users, you switch the whole system between A and B, for example by city and by hour. It is useful when users share something, such as couriers.
• Standard error and confidence interval: the standard error says how much a number would move if you repeated the study with other people. A 95% confidence interval goes about two standard errors to each side. See Estimate uncertainty.
• Peeking: looking at results every day and stopping when they look good or bad. Each look gives chance one more opportunity to fool you.
• Holdout: a small group that does not get the feature for a long time, so that you can measure the long-run effect.

The interview

Minutes 0 to 5: the opening

Gabriel(candidate):

Hi Thandiwe. Sorry, one thing first. My son is home sick today, so if you hear a door, it’s him. My wife is with him.

Thandiwe(interviewer):

No problem at all. I have two at home, I know. So, I’m a senior data scientist on Growth and Learning. First you tell me about an analysis that changed a decision, and I ask a lot of “why”. Then our real problem. Then your questions. This round is without AI tools, only conversation. I know you used one in the SQL round. That was allowed there.

Gabriel(candidate):

Okay. Just us. Paper is okay?

Thandiwe(interviewer):

Paper is fine. So, an analysis that changed a decision.

What the interviewer is checking

The AI rule changes from round to round. If you are not sure what is allowed, ask, as Gabriel does. See Do not use AI or other help to cheat.

Minutes 5 to 20: an analysis that changed a decision

Gabriel(candidate):

Okay. At Forkline, the small-order fee. Small orders, under 15 dollars, lose money. The courier costs the same for a coffee as for a family dinner. Finance wanted a fee of about 2 dollars on those orders, everywhere, in one month.

So we ran a normal A/B test, by user. And it looked very good. Revenue per user up, I think about 4%. Everybody was happy. But I was not comfortable, because the users share the couriers. If the fee group orders less, there are more free couriers, and the other group gets faster delivery. So the control group is not really a control.

Thandiwe(interviewer):

Mm-hm. So what did you do?

Gabriel(candidate):

A switchback. In six cities, we switched the fee on and off for the whole city, every two hours, random. Then the couriers are the same for everybody in those two hours. And the result was much smaller. Around 1%, and the interval included zero.

Thandiwe(interviewer):

Why every two hours? Why not every day?

Gabriel(candidate):

Hmm. With days, you have too few periods. We only had a few weeks. Shorter periods, you get more of them, so more data. But too short is also bad. An order from 7:55 is still on the road at 8:10, in the next period. So it leaks. We tried one hour first, and the leak was too big. Two hours was the compromise.

Why this works

Thandiwe’s “why” goes one level below the method. Gabriel’s answer has a real trade-off: shorter periods give more data, but orders cross into the next period. He also says that they tried one hour first. Only the person who did the work knows details like this.

Thandiwe(interviewer):

Your CV says you led the move to switchback tests. What was your part exactly?

Gabriel(candidate):

I proposed it, I wrote the design document, and I did the analysis. The assignment system, two engineers built that, maybe six weeks. “Led” is… okay. I led the idea and the analysis. Not the code.

And the decision: not everywhere. Only in busy hours, in the big cities, when couriers are really short. There the fee helped. In quiet hours it only made people angry. And the 3 million on my CV, that’s an estimate for that smaller launch. Finance made it, I checked it. It’s not measured.

What the interviewer is checking

Thandiwe tests the word “led”, and Gabriel makes it exact. Without being asked, he also says that the 3 million is an estimate. Senior people often stretch their numbers a little, because they are used to selling results inside a company. Interviewers notice when a candidate does the opposite.

Thandiwe(interviewer):

Clear, thanks. What would you do differently?

Gabriel(candidate):

I would keep a holdout. We measured three weeks. But a person who pays the fee three times, maybe two months later he goes to another app. We never measured that. We

The video freezes for a few seconds.

Thandiwe(interviewer):

Gabriel? You froze. I heard “we never measured that”.

Gabriel(candidate):

Sorry, my internet. Yes. I asked for a holdout, 5% of users without the fee for six months. Product said no, because of the money. I accepted too fast, I think. I should have argued more.

Why this works

After the freeze, Gabriel goes back to his point calmly. And his answer is a real weakness, not a polished one like “I work too hard”.

Minutes 20 to 50: the Parlo case

Thandiwe(interviewer):

Okay, I’m moving on now. Our real situation. Five weeks ago we launched “Talk with Pip”, AI conversation practice, to 20% of new users. Weekly active users in that group went up about 6%. But day-30 retention for new users went down from 31 to 29%. Our product manager wants to give it to everyone before January. If you were our data scientist, what would you do?

Gabriel(candidate):

Okay. Let me think a moment.

About ten seconds of silence. He writes on paper.

First thing. Five weeks, and day-30 retention. So only people who joined in the first five days or so have a day-30 number? How many is that?

Thandiwe(interviewer):

Yes, good. Around 2,400 in the Pip group. The comparison group is about 25,000.

Gabriel(candidate):

Okay. So at 30%, the standard error for 2,400 people is… square root of 0.3 times 0.7, around 0.46, divided by the square root of 2,400, about 49. So a bit less than one point. With the other group, maybe one point in total. So two points is about two standard errors. The interval is roughly zero to four points. Borderline.

But this number is not the biggest problem. The comparison is. How did you choose the 20%?

Why this works

Gabriel rounds out loud and ends with a plain word: “borderline”. He is right: the 95% interval for the drop is about 0.1 to 3.9 points. You do not need to be this fast. But you should know that 2,400 people give a much wider range than 25,000.

Thandiwe(interviewer):

Percentage rollout in our experiment tool. 20% of new users on iOS. Android was not ready.

Gabriel(candidate):

And the 31%, that’s the other new users on iOS?

Thandiwe(interviewer):

Yes, the other 80% of iOS. No, wait. Sorry, let me check.

She looks at her notes.

No, I was wrong. The dashboard compares with all other new users. Both platforms.

What the interviewer is checking

Thandiwe makes a mistake, notices it and corrects it. Good interviewers do this. Gabriel does not comment on it. He just uses the new fact.

Gabriel(candidate):

Okay, then it’s strange. In most apps I know, iOS users retain better than Android. Is the same here?

Thandiwe(interviewer):

Yes. A few points better.

Gabriel(candidate):

Then the comparison helps Pip, and Pip still looks worse. The comparison group has Android inside, and that pulls it down. If I take out Android, the comparison goes up, a bit above 31. So against iOS only, the gap is maybe bigger, not smaller. But I want to see it, not guess.

He opens the shared whiteboard and draws a small table. The rows are iOS with Pip, iOS without Pip, and Android. The columns are new users, day-30 retention, lessons per week, and conversations per week.

This first. iOS against iOS, same days. Do you have that number?

Thandiwe(interviewer):

I do. iOS without Pip, same days: about 33%. Around 9,600 people.

Gabriel(candidate):

Thirty-three. So it’s four points, not two. Hmm. The standard error is similar, about one point. So four points is almost four standard errors. That’s not noise. Was the 20% random, by user ID?

Thandiwe(interviewer):

Yes. The tool assigns by user ID.

Why this works

This is the key moment of the case. The dashboard compared iOS users with a mix that includes Android, which retains worse. The fair comparison makes the gap bigger. Gabriel’s numbers are right: the standard error is about 1.04 points, so the 95% interval for the drop is about 2 to 6 points.

Gabriel(candidate):

Then inside iOS this is almost a real experiment. Nobody chose the metric before, and it’s only five days of users, so I’m careful. But I take it seriously now.

Thandiwe(interviewer):

Mm-hm. Go on.

Gabriel(candidate):

The date. Five weeks ago is end of August, so these first users joined just before school started. People who join in August, maybe on holiday, behave differently in September. But both groups joined on the same days, so the season hits both. It’s a big problem for “before and after”. Less for “same days, two groups”. But maybe August learners react to Pip differently from January learners. That I can’t know from this data.

Thandiwe(interviewer):

Okay. What else?

Gabriel(candidate):

Lessons. Weekly active is up 6%. But what is “active”? If one conversation with Pip counts, then Pip makes the number go up just by existing. Maybe people talk two minutes and stop the lessons. And maybe the lessons are what keeps them for 30 days. Do Pip users do fewer lessons?

Thandiwe(interviewer):

A conversation does count as active. And yes, lessons completed went down in the Pip group.

Gabriel(candidate):

Okay. So one story is: Pip is fun, it replaces lessons, and the lessons were the habit. But it’s a hypothesis. I’d look at users who talked with Pip in week one and stopped lessons. Do they leave more?

What the interviewer is checking

Part of the 6% may come from how “active” is counted. Gabriel calls his story a hypothesis and names the analysis that would test it.

The next few minutes, in short

Gabriel asks what one Pip conversation costs Parlo. About 4 cents, Thandiwe says, and a heavy user has 20 a week. He works it out on the whiteboard: so about 80 cents a week, or about 3.50 dollars a month. “A free user who talks a lot costs us three and a half dollars and pays nothing,” he says. “In delivery we had the same problem with free delivery. So I want to know: do Pip users buy Plus more often?” Nobody has looked at that yet. Thandiwe writes it down.

Thandiwe(interviewer):

Okay, a harder one. Pip is about learning to speak. Say retention is a bit lower, but people really learn more. How would you know?

Gabriel(candidate):

Hmm.

A long pause.

Honestly, this part I don’t know. In delivery the outcome is easy. The food arrived, or it didn’t. Here… how do you measure that someone speaks better? Do you have a test?

Thandiwe(interviewer):

There’s a short placement test. But few people take it, and even fewer take it twice.

Gabriel(candidate):

Then the people who take it twice are special. Very motivated. So if Pip users who retake it do better, I don’t trust it. Maybe only the good ones retake it. That I can say from statistics. But what is a good measure of speaking, from the teaching side, I don’t know. What has Parlo tried?

Thandiwe(interviewer):

We tried three short questions inside a lesson, once a month. Completion dropped, people hated it, we stopped after two weeks. Honestly, we haven’t solved it.

Gabriel(candidate):

Okay. Then I would not promise Kenji a learning number this quarter. I’d say it openly: now we measure engagement and retention, and learning is a separate project. Maybe with someone who knows language teaching.

The edge

This is Gabriel’s edge. He knows experiments very well, but not how to measure learning, and he says so. Then he uses what he does know: people who retake the test are not a random group. And he asks what Parlo has already tried. Nine years of experience made “I don’t know” easier for him, not harder, because he trusts the rest of his answers.

Thandiwe(interviewer):

Good. Last part. Kenji wants an answer this week. What do you tell him?

Gabriel(candidate):

That the dashboard is misleading. The fair comparison is worse: about four points down, not two. But it’s only five days of users, from one unusual week, and “active” includes Pip itself. So I’d run a proper test. New iOS users, 50/50, and we decide before the start: day-30 retention is the main metric, lessons and cost are guardrails. And no looking every day.

Thandiwe(interviewer):

Mm-hm. And is that a yes or a no for January?

Gabriel(candidate):

Ah. Sorry, I didn’t say it. It’s “not yet”. To see one point at 30%, I need, I think, around 30,000 people per group. Maybe a bit more, I’d calculate it properly. With half of new iOS users in each group, about four weeks to get them. Then 30 days more for day 30. If we start in two weeks, we know by the middle of December. Not much time, but it’s before January.

Where this goes wrong

Gabriel’s first answer is a list of problems and a test design. All of it is correct, but he forgot what Kenji asked for: a decision. Experienced people often start with the method. A product manager hears “the dashboard is misleading” and still does not know what to do on Monday.

A stronger answer

Start with the decision, then the reason, then the plan:

“My recommendation: not to everyone yet. Keep Pip at 20% on iOS. The dashboard compares iOS with all users. iOS against iOS, Pip users are about four points worse at day 30. That’s only five days of sign-ups, from one unusual week, so I’m not sure it will hold. But it’s a real warning. So let’s run a clean 50/50 test on new iOS users, starting in two weeks. We’ll have a day-30 answer by mid-December. If it’s neutral or better, we launch in January. If it’s worse, we look at Pip and the lessons first.”

Same content, different order. Kenji can repeat the first sentence to his VP. Gabriel’s numbers hold up, too: to see a one-point change at 30% with the usual 80% power, you need about 33,000 people per group.

Minutes 50 to 54: the question about the level

Thandiwe(interviewer):

Before your questions, can I ask something more personal? You can say no. You were a senior for four years. This role is intermediate. And today you did the case better than some seniors I know. So I wonder: what happens in six months, if a senior job comes?

Gabriel(candidate):

For me the title is not important. I just like the work, and…

He stops.

No. Sorry. Let me answer you properly. You asked honestly.

Thandiwe(interviewer):

Take your time.

Gabriel(candidate):

I was laid off in March. I have two kids, and I’m the main salary at home. Seven months is long. So yes, I apply to intermediate roles, because I need a stable job, and I know I can do this one well. I also apply to senior roles. I have one other process now.

But I don’t apply to jobs I plan to leave. If I come here, I want to stay. I can’t promise you three years, nobody can. What I can say is: if the work and the people are good, I don’t move for a title. I was seven years at Forkline.

Thandiwe(interviewer):

Thank you. That’s a real answer.

Gabriel(candidate):

Can I ask one thing back? Is there a path to senior here? Not tomorrow. But is it possible?

Thandiwe(interviewer):

People have moved up here, yes. I can’t promise a timeline. Simone can tell you more.

Where this goes wrong

“The title is not important for me” is what many people say here. Thandiwe hears it often, and it rarely convinces her. Everybody cares about pay and level a little. The sentence sounds prepared, to calm the interviewer. Gabriel hears it too, and he stops.

Why the honest answer works

• It explains the choice. A senior who applies one level down for no reason makes people nervous. “I need stability, and I can do this well” is a reason people understand and respect.
• It does not promise too much. “I can’t promise three years” sounds true, so the rest sounds true too.
• It gives evidence. Seven years at one company says more than any promise.

He also does not hide the other process. If Parlo learned about it later, it would hurt the trust the interview built.

Look at what Gabriel did not do. He did not give simpler answers in the case to look like an intermediate. And he did not tell a sad story to get sympathy. He said the facts once, briefly, and went back to the work. The pressure at home is real. You are allowed to say it, and you do not have to.

Minutes 54 to 60: Gabriel's questions and the close

Gabriel(candidate):

My questions. You said each data scientist has one product area. Who checks my analysis before it goes to the product manager?

Thandiwe(interviewer):

Another data scientist reads it. For big decisions, Simone too. It’s not perfect. In busy weeks it’s quick.

Gabriel(candidate):

And the Plus conversion and the cost of Pip. Who looks at that? Today it sounds like nobody.

Thandiwe(interviewer):

Laughs. You noticed. There’s a Monetisation team, but they have no data scientist yet. They borrow time from us.

Okay, we’re at time. Thanks, Gabriel. I hope your son feels better. Mira will contact you about the next step. It’s a take-home with a short presentation.

Why this works

Both questions are about the real job: how mistakes are caught, and where the open problems are. The second one also shows Thandiwe what Gabriel cares about. See An interview is a conversation in both directions.

After the call: Thandiwe’s notes

Candidate: Gabriel Santos. Product case, 60 minutes.

Strong evidence

• Experiment design and statistics. Switchback story is real and deep. On our case: found the day-30 window problem, asked for iOS against iOS (33%, so four points down, not two), and the “active” definition. Correct rough interval in his head.
• Product sense and cost. Raised cost per heavy free user and conversion to Plus without a prompt. We have not looked at conversion. We should.
• Honesty. Made “led” and the 3 million exact. Said “I don’t know” at the edge.

Some evidence

• Working with a PM. Accepts that the PM decides. First answer for Kenji started with the method, not the decision. Fixed it at once when asked.

No evidence

• Learning measurement. Does not know it, said so, asked what we tried. Same as all of us.

Concern

• Level. He is senior. Answered “will you stay” honestly: needs stability, has another process, would not move for a title. I believe him. But I think we are hiring him at the wrong level.

Recommendation: strong yes. Simone, please read the last note.

What happened next

Gabriel did the take-home: a small dataset of learners, three hours at most, AI tools allowed and declared. He used an AI assistant, and at the end he wrote a short section called “How I used AI”:

• He asked the assistant for first drafts of the plotting code and of the SQL for the retention table.
• He checked the SQL by hand on three learners. The assistant’s join dropped learners who had no sessions after day 1. That made retention look higher than it was. He fixed it, and the section shows the numbers before and after.
• He wrote the conclusions and the recommendation himself.

In the presentation, Simone asked about the dropped learners, and he showed the three he had checked. This is what “use AI tools to increase speed and quality” looks like when it is done well. The tool did the typing. Gabriel did the checking, and he said so.

Then came the hard part. The role had one vacancy, and it went to Chloe, another finalist, who fit the intermediate level better. Gabriel’s other process was moving fast. He wrote to Mira, told her why the timeline mattered, and asked where things stood. Mira was honest too. The intermediate role was going to someone else, but Simone was asking the director for something, and Mira would know more in about ten days.

Simone had read Thandiwe’s last note. The Monetisation team had a Senior Data Scientist role planned for the first quarter of next year, and Gabriel’s questions about cost and conversion to Plus were exactly that team’s problem. Simone asked to open the role now, and the director agreed. After one more interview, with the Monetisation product lead, Gabriel had an offer for the senior role, two weeks after the product case.

There was luck in this. A senior role was planned on another team, and it could be moved earlier. That does not happen often. See Luck is part of it. But the luck needed honesty to work. If Gabriel had made himself look smaller, nobody would have thought of him for a senior role. If he had promised to stay forever, or hidden the other process, Mira and Simone would have trusted him less when it mattered.

What to take from this

• Applying one level down is a reasonable choice, not a failure. Be ready to explain it.
• Answer “will you stay?” with your real reasons. Say why you applied, what you can and cannot promise, and give evidence from your past.
• Do not make yourself smaller. Show your real level. The team may find a better place for you, as Parlo did.
• With a product manager, start with the decision. The method comes second.
• Senior people have an edge too. “I don’t know. What have you tried?” made the rest of Gabriel’s answers more believable.
• Use AI where it is allowed, and say what you checked. One wrong join, found by hand, did more for Gabriel than a perfect-looking report.

If your profile is like Gabriel’s

1. Prepare the “level” answer before anyone asks. Write it down, then say it out loud: why you are applying, what you can promise, and one piece of evidence, such as how long you stayed in past roles. Practise until it sounds calm, not defensive.
2. Use AI to prepare, as much as you like. Ask an assistant to play an interviewer who worries that you are overqualified, and to push back on your answer. That is preparation, not cheating. See Do not use AI or other help to cheat.
3. Check every number and verb on your CV. For each “led” and each money figure, know exactly what you did and where the number came from. See Explain decisions.
4. Practise starting with the decision. Take three past analyses and write each one as: the recommendation in one sentence, the main reason, then the plan. Compare alternatives helps you say what a result does and does not show.
5. Be honest with recruiters about your timeline, and stay in touch with former colleagues while you search. Ask them for advice, not for a job. See Prepare for applications.