How the interviews run

Think about a friend who comes back from a long trip. You ask, “How was it?” They say, “Great, we took the train across the country.” Then you ask small questions, one after the other: “Where did you sleep?”, “What happened when the train was late?”, “How much did it cost?” A friend who really took the trip answers easily, with small details that nobody would invent. A friend who only read about the trip runs out of details after two or three questions.

A serious technical interview works the same way. The interviewer listens first, and then asks about what you said, one step deeper each time. This page explains that pattern and the five interviews in Tidewell’s process. Then you are ready for Part 3, where five candidates go through the same pipeline design interview.

The five interviews at Tidewell

Tidewell’s posting does not describe its process. Most real postings do not. So the first thing to do in the recruiter call is to ask. Here is what the recruiter told the candidates in this guide. Each interview checks something different, and each one is run by a different person.

1. The recruiter call (30 minutes, Aisha Rahimi, Recruiter). Aisha checks the basics:

• Does your experience match the level?
• What pay do you expect, and when can you start?
• Can you explain your work to someone who is not an engineer?

She also explains the process and answers questions. In this call, candidates learned three useful facts that are not in the posting. The team uses Kafka, Databricks (Spark and Delta tables), Airflow and dbt every day. There is an on-call rotation, one week in seven, because billing runs at 5 a.m. And the level is intermediate, with some flexibility for a strong senior candidate. Remember that in British Columbia, Aisha may ask what pay you expect, but she must not ask what you were paid before.

2. SQL and Python (60 minutes, a data engineer from the team). You write code in a shared editor while you talk. AI tools are not allowed in this round. The tasks are close to the team’s daily work. For example: find the meters that sent fewer readings than expected yesterday, or keep only one copy of each reading when the same reading arrived twice. Then a small Python task, such as reading a messy file from the meter vendor and finding the broken lines. The interviewer checks that your code is correct, that you test it on strange cases (an empty day, a duplicate, a missing value), and that you explain what you are doing while you type.

3. The pipeline design deep dive (60 minutes, video, Esther Adeyemi, Senior Data Engineer). This is the interview in Part 3 of this guide. Esther has ten years in data engineering. She grew up in Lagos, studied in the UK, and has lived in Canada for six years. She was on call the night of the double-billing incident that you will read about below, so she cares about it personally. She is direct and patient. She likes to draw boxes on a shared whiteboard while you talk, and she often asks, “And what happens if it runs twice?” She asks about a pipeline that you built, goes deep into it, and then gives you Tidewell’s real problem.

4. Data modelling (45 minutes, Pierre Gagnon, Staff Data Engineer). Pierre asks you to design tables for a real need, for example “billing needs the usage of each account, per day, for the last two years”. He checks how you choose the grain, how you separate facts from descriptions, and how you keep history when things change. A meter swap is his favourite example: the account stays the same, but the meter changes. Many candidates find this round the hardest, because there is no code; you must explain every choice in words.

5. Values and collaboration (45 minutes, Rosa Delgado, Director of Billing Operations). Rosa is not an engineer. Her team answers the phone when a bill is wrong. She asks how you behave in difficult situations: when a user says your numbers are wrong, when you find a mistake that already reached customers, when a deadline and correctness conflict at 5 a.m. She also checks that you can explain a data problem without jargon, and that you respect the people who use your data.

After the last interview, the interviewers meet in a debrief with Graham Whitfield, the Data Engineering Manager, who is hiring for this role. Each interviewer shares written notes, and together they decide. A strong result in one interview does not cancel a serious concern from another one.

Other kinds of interview that you may meet

You may meet other rounds at other companies:

• A take-home task: you build a small dbt project or a notebook with a pipeline, usually in a few hours. Many reviewers read your README as carefully as your code. Write down what you assumed, what you tested, and what you would do with more time.
• A Spark task: the same kind of task as the SQL round, but in PySpark, sometimes with a question about why a job is slow.
• Coding with an AI assistant allowed: some companies now allow it and watch how you check what it writes. Others do not allow any AI help. Follow the rules of each round, and ask if you are not sure.
• A platform design round: “Design the data platform for a food delivery company.” It is wider than Tidewell’s deep dive, with less depth.
• A behavioural round: questions about past situations. Expect at least one about a production incident, and keep your own part separate from the team’s.

Some companies have also brought back at least one interview in person, partly because some candidates used hidden AI tools during video interviews. We say more about that in our advice.

Your English does not need to be perfect

Many people in data engineering interviews speak English as their second or third language. That includes many interviewers. In this guide, several of the five candidates grew up speaking another language first, and you will hear it in how they talk. A good interviewer listens for your thinking, not for perfect grammar. If you do not understand a question, ask for it again. If you need a moment to find a word, take it. See our advice.

How a serious interviewer goes deeper

The pipeline deep dive starts with an open question: “Tell me about a pipeline that you built, or that you own.” Then the interviewer follows your answer. Here is how one sentence can grow into twenty minutes of conversation.

Suppose that a candidate says: “I built a nightly job that loads our online orders into the warehouse.” The interviewer goes down level by level:

From one sentence to the edge

1. 1

   “Where does the data come from, how much is there, and who uses the result?”

   Is the pipeline real, and how big is it? A real pipeline has real sources, a real size and real users.

2. 2

   “When does it run, and what does it read? All the data, or only the new data?”

   Did the candidate build it, or only use it? The builder knows how the job decides what is new.

3. 3

   “It fails halfway. You run it again. What happens to the rows that were already written?”

   Is the job safe to run twice? This is the question that separates many candidates.

4. 4

   “An order from Monday arrives on Wednesday. Where does it go?”

   Does the candidate know the difference between when something happened and when the data arrived?

5. 5

   “How do you know that the numbers are right? Tell me about a time they were wrong.”

   Can they check their own data, and do they learn from incidents?

6. 6

   “What would change with fifty times more data, or if finance wanted it every hour?”

   The edge: how the candidate thinks beyond their own experience, including cost.

The interviewer plans the areas to probe, not the exact questions. The questions come from the answers. A candidate who says “we delete yesterday’s partition and write it again” will get a question about partitions. A candidate who says “we just append the new rows” will hear Esther’s favourite question very soon.

This is why lists of interview questions help so little. The questions come from your own work, so you cannot learn the answers by heart. What you can do is know your own pipeline very well: every decision, every number, every failure.

The edge of what you know

Every candidate reaches an edge. A good interviewer goes there on purpose, because the most useful information is there. They want to see what you do when you do not know:

• Do you say clearly that you do not know, or do you talk around it?
• Do you reason from what you do know, step by step, out loud?
• Do you ask a question that makes the problem clearer?
• Do you stay calm, and stay honest?

Here is a short example. The candidate has just said that their nightly job also “compacts small files” in their Delta tables.

An example: reaching the edge

Esther(interviewer):

You said you compact the small files every night. Why are small files a problem?

Candidate(candidate):

Sorry, do you mean why we run it, or how the compaction works?

Esther(interviewer):

Why it is needed. What goes wrong if you don’t do it?

Candidate(candidate):

Okay. Hmm. Honestly, the job was there when I joined, and I did not look inside it much. What I saw is that queries got slower when we skipped it for some days.

Esther(interviewer):

Mm-hm. Any idea why slower?

Candidate(candidate):

Let me think. I think each file has a cost to open, even if it is small. So ten thousand small files is ten thousand opens, and the table has to keep track of all of them. A few big files, fewer opens. That is my guess. I would check the documentation before I say it in a design review.

Why this works

The candidate first checks what the question means, which is normal and useful. Then they say exactly where their knowledge stops, and reason from what they saw. The guess is mostly right. Esther learns more from this than from a definition learned by heart.

The Tidewell problem

In the second half of the deep dive, Esther gives every candidate the same problem. It is the real reason the team is hiring. First, a few words that you will need.

Think about a letter. The date on the stamp says Monday, but the letter arrives on Thursday. If you sort your letters by the day they arrived, the Monday letter is in Thursday’s pile. Meter readings work the same way:

• Event time: when the reading was taken at the meter. This is the date on the stamp.
• Arrival time (also called processing time): when the reading reached Tidewell’s systems. This is the day the letter came.
• Partition: a part of a big table, usually one day of data, that a job can read or replace on its own.
• Idempotent: a job that gives the same result whether it runs once or twice.
• Backfill: running a pipeline again for past days, for example after you fix a bug.
• Watermark (in streaming): how long a streaming job waits for late events before it closes a time window. In batch, teams usually call this a cut-off or a lookback.

Here is how Esther gives the problem:

“Okay, now our real problem. Our meters send a reading every 15 minutes. Sometimes a meter loses network and the readings come late, up to three days late. Our nightly job reads by arrival date. Last month, after a failed run, we re-ran the job and one customer was billed twice. And the forecast team says their history has gaps. How would you design the meter-readings pipeline so this does not happen again?”

There is no single right answer. Esther does not want a finished design in 25 minutes. She wants to see where you start. Which questions do you ask first? What would you change first? How would you know that the data is correct? What does billing do while it waits for late readings? Then she follows your answer, in the same way as with your own pipeline. If you go deep on one part, she will ask about another: the meters that get replaced, the days when the clocks change, the AI assistant, or the cost.

Try it before you read the candidates

Take 25 minutes, and write or say out loud how you would approach the Tidewell problem. Use what you learned from Read it honestly. Answer these for yourself:

1. Which three questions would you ask Esther before you propose anything?
2. Why did the re-run bill a customer twice? What would you change so that a re-run is safe?
3. A reading arrives three days late. Which day does it belong to, and which job picks it up?
4. How long would you wait for late readings before billing uses the data? What does billing do with a missing reading?
5. How would you know, every morning, that yesterday’s data is complete? How many readings do you expect from one meter in one day? Is that number always the same?
6. Does any part of this need streaming?

Keep your notes, and compare them with the five candidates. Each candidate starts from their own experience. You should do the same.

How to read the candidate pages

Each candidate page shows the same 60-minute interview with Esther, as a conversation. Blue turns are the interviewer; green turns are the candidate. Notes between the turns explain what is happening:

• What the interviewer is checking: the real question behind the question.
• How the interviewer goes deeper: why the next question follows from the answer.
• Why this works: what made an answer strong.
• Where this goes wrong: a weak moment, and why it is weak.
• A stronger answer: the same moment, answered better, in the candidate’s own words.
• Disagreeing well: how a candidate disagrees with an interviewer, with reasons.

Each page ends with Esther’s written notes, what happened next, and advice for readers with a similar profile. Every candidate has strong moments and weak ones, because every real candidate does.

The five candidates are:

1. The BI developer: Fatima Zahra has nine years of SQL, reports and careful data models. Streaming and Spark at this size are new to her.
2. The software engineer: Lukas builds reliable payment services in Java. Tables for analysts are new to him.
3. The modern stack: Jordan is fast and careful with managed tools. They have rarely needed to look under them.
4. The veteran: Vikram has 24 years in data and many incidents behind him. Some of the newest words are not his words yet.
5. The domain expert: Nicole worked in meter operations for seven years and learned to build in the evenings.

Read them in any order. Start with the one that is most like you, and then read the one that is least like you.