The returner

Think of someone who drove a car every day for years, then lived without one for three years. The first week back, the dashboard looks different. But they still know how to drive. They still see the child near the road before anyone else does.

Many engineers come back to work after a break: for children, a sick parent, their own health, a move or study. They often apply with a fear: “The tools changed. Everyone will see the gap on my CV.” The tools did change, a little. The judgment that took years to build did not go away.

Follow Hana through the ML deep dive at Fennick. Watch how quickly the career break comes and goes, where her experience makes her the strongest candidate of the week, and where she reaches her edge.

HK

The candidate

Hana Kowalczyk

ML engineer, 7 years on pricing and ETA models, back after a career break

• 7 years as an ML engineer
• Ride-hailing pricing and ETA
• Career break 2022 to 2025
• Toronto, Ontario

• Machine Learning Engineer, Tandem Rides (2015 to 2022). Pricing and ETA models for a ride-hailing company in 9 Canadian cities.
• Rebuilt the pickup ETA model (gradient boosting, real-time driver features). Cut late pickups from about 14% to about 8% of rides.
• Built the surge pricing model’s training pipeline and its monitoring. On-call for pricing and ETA services.
• Career break (2022 to 2025). Full-time care for my two young children.
• Return to ML (2026 to now). Courses on current ML tooling. Public project: hourly demand forecast for Toronto bike share, with LightGBM, quantile predictions, a FastAPI service and weekly monitoring.
• Python, SQL, LightGBM, XGBoost, scikit-learn, Kafka, Redis, Airflow, Docker. Some Kubernetes.

What Ioana sees before the call

Ioana reads the CV twice. Pickup ETA, real-time features, late pickups as the metric: that is very close to Fennick’s problem. The bike-share project is recent and uses quantile predictions.

She sees the career break. The CV already explains it, and it is not part of the job. She wants to find out other things:

• How much of the ETA model did Hana build herself, and does she remember why she made each decision?
• Is the “14% to 8%” number real? How was it measured, and on which rides?
• On the Fennick problem, does her ETA experience turn into good decisions here, where couriers carry groceries, not passengers?
• Where does her knowledge stop? The tooling of 2026 is not the tooling of 2022. What does Hana do when she meets something new?

Terms on this page

• ETA (estimated time of arrival): here, the time from the order to the door that the app shows before the customer pays.
• Quantile (or percentile): the 90th percentile of delivery time is the time that 90% of deliveries beat. A quantile prediction predicts that time for one order, instead of the average.
• Pinball loss (or quantile loss): the training loss for a quantile prediction. For the 90th percentile, a prediction that is too low (the order arrives later) costs 9 times more than one that is too high by the same number of minutes.
• Coverage and calibration: if you predict the 90th percentile, about 90% of real deliveries should arrive before it. If that is true, the prediction is calibrated. Calibration by condition means you check this separately at dinner rush, in rain, in each city, and not only on average.
• Time-based split: train on older data, test on newer data.
• Training/serving skew: a feature is computed one way for training and another way in production, so the model sees different numbers live than it learned from.
• Feature store: a system that computes features once and gives the same values to training (the offline store) and to the live service (the online store).
• Feature freshness: how old a real-time feature value is at the moment the model uses it.
• Switchback test: an online test where a whole zone uses one version of the model for a time window, then switches. Useful when customers share couriers, so a normal A/B test by customer is not fair.

The interview

Minutes 0 to 5: the opening

Ioana(interviewer):

Hi Hana. So, I’m Ioana, staff ML engineer in Delivery Intelligence. I built the ETA service, so I will probably ask too many questions about ETA. First, a model you put in production. Then a real problem from our team. Then your questions. No AI assistants today, please. Paper and pen is fine.

Hana(candidate):

Okay. I have paper. Good.

Minutes 5 to 25: a model in production, then deeper

Ioana(interviewer):

So. Tell me about a model you put in production. Any one you like.

Hana(candidate):

I will take the pickup ETA at Tandem Rides. It’s from 2019, so it’s not new, but it’s the one I know best.

So, when you ask for a ride, the app shows “driver arrives in 6 minutes”. The old model was quite simple, a linear model on distance and the time of day. It was too optimistic, especially downtown. About 14% of pickups were more than three minutes later than what we showed. Riders cancelled, and drivers lost money.

What I did is, I built a new model with gradient boosting, XGBoost at that time. The big change was the features: the driver’s real position and speed from the last minute, and traffic on the route. After launch, late pickups went to about 8%.

Ioana(interviewer):

Mm-hm. And the 14 and the 8, they are measured how? Same kind of days?

Hana(candidate):

Good question. The 14 was the four weeks before the launch, the 8 was four weeks after. So not the same weeks, and it was spring. We also did a test with half the cities on the new model for two weeks, and the difference there was smaller. I think 13 to 9, something like that. So the real effect is about four or five points, not six.

Why this works

A before-and-after number mixes the model with everything else that changed, like the season. Hana knows this and corrects her own number with the better evidence. That makes her other numbers more believable.

Ioana(interviewer):

How did you split the data? And why?

Hana(candidate):

By time. Train on maybe ten months, validation on the next month, test on the month after. Never random, because rides on the same evening are very similar. Same rain, same concert, same traffic jam. With a random split, the model sees Tuesday evening in training and in test. It looks very good, but it’s only remembering Tuesday. In real life, it always predicts a day it never saw.

Ioana(interviewer):

Okay. And the driver features. Where did they come from, in training and live? Were they the same?

Hana(candidate):

Ah, yes. In the first version, they were not the same. For training, we computed speed from the GPS history table, which was cleaned later. Live, we had the raw GPS, with jumps and gaps. So the model was worse live than offline. We saw it in the first week.

After that, the service logged the exact feature values it used for each prediction, and we trained the next versions on these logs. It takes some weeks to collect, but then training and live are the same numbers.

Ioana(interviewer):

Smiles. We learned the same thing, but it took us six weeks to notice. How did you know the model still worked, three months later?

Hana(candidate):

A dashboard with the late rate by city and by hour, every day. The ride ends, you know the real pickup time, so the label comes quickly. And an alert if a city was over 12% two days in a row. On the features we had only simple checks, like the share of missing GPS. I think today people do more on drift.

What the interviewer is checking

Ioana is checking training/serving skew, the bug that hurt Fennick last year. Hana describes a real case, how she found it, and the fix: log the features at prediction time and train on the logs. That is still a standard answer in 2026.

Ioana(interviewer):

Okay. And after 2022?

Hana(candidate):

I took a career break for three years, to take care of my two children. They’re both in school now, so this year I came back. I did some courses, and the bike-share project.

Ioana(interviewer):

Okay, thank you. In the project I see quantiles. How do you check that your 90th percentile is really the 90th?

A career break is not something to apologise for

Hana says it in one sentence, with no apology, and then points forward to what she did to come back. Ioana moves on, because there is nothing to judge. You do not owe anyone details about your family or your health. Prepare one calm sentence about the break and one about how you returned. Then go back to your work.

Hana(candidate):

I count. On the test weeks, how often the real number of trips was below my 90th prediction. Overall it was about 88%, so okay. But on rainy days it was much worse, maybe 75%.

And, um, I found one problem myself. For training I used the real weather of that hour. But when you predict the day before, you only have the forecast. So my results are too good. I wrote it in the limits. I did not fix it yet.

Ioana(interviewer):

That’s fine. At least you know it. Let’s go to our problem.

Minutes 25 to 48: the Fennick problem

Ioana(interviewer):

Okay, now a real problem from our side. Customers see a delivery time in the app before they pay. At dinner time and when the weather is bad, our estimate is too optimistic. On normal days about 9% of orders arrive more than ten minutes late compared to the estimate; on those days it’s about 18%. People are angry, support tickets go up. How would you approach this?

Hana(candidate):

Can I take one minute to write?

Ioana(interviewer):

Of course.

About forty seconds of silence. Hana writes.

Hana(candidate):

Okay. First, what does the customer see? One number, or a range? And the model predicts what, the average time?

Ioana(interviewer):

One number, rounded to five minutes. The model is trained with the normal squared error. So yes, more or less the average.

Hana(candidate):

Then I think this is the main reason. Delivery times have a long tail on the late side. It’s hard to be twenty minutes early, but easy to be twenty minutes late. At dinner and in rain, the tail gets much longer. The average moves a little, but the late orders move a lot. So the same model, with the same small buffer, is late much more often on those days.

So I would predict a high quantile instead. Maybe the 80th or the 85th percentile. LightGBM does it directly with the quantile objective. Not the 95th, because then the time is long and people don’t order. Late is worse than early, but too pessimistic also costs. Do you know how much a longer time changes the orders?

Ioana(interviewer):

A little. In March, operations added five minutes when it rained, in two cities. Orders went down about one or two percent, I think. I don’t remember exactly.

Hana(candidate):

Okay, so it’s real but not huge. Then I would not choose the quantile alone. I would show operations and product a small table: for the 75th, 80th, 85th, 90th, what is the late rate, and how many minutes longer the shown time is on average. They choose. It’s a business decision, not a model decision.

Why this works

Two questions about today’s system lead Hana straight to the cause: the model predicts an average, and the late tail grows at dinner and in rain. Then she treats the quantile as a business choice, and gives others a table to decide with. That is what owning a model looks like.

Ioana(interviewer):

Good. Say we choose the 85th. How do you evaluate it offline?

Hana(candidate):

Pinball loss to train and compare models, on a split by time, of course. But the real check is coverage. If I predict the 85th, about 85% of orders should arrive before it. And not only overall. By condition: dinner rush, rain, snow, each city, big stores and small stores.

Ioana(interviewer):

Why by condition? If it’s 85% overall, it’s correct, no?

Hana(candidate):

No, because that’s exactly your problem today. Fine on average, bad at dinner. 85% overall can be 92% on a quiet Tuesday afternoon and 70% at dinner in rain. The average hides it. So for each condition I want the coverage and the number of orders, because a small slice jumps around.

Ioana(interviewer):

And if the coverage is wrong at dinner in rain?

Hana(candidate):

First, features. The model must know it’s raining and it’s busy: weather, open orders at the store, free couriers in the zone. For weather, careful, use what the service knew at the time of the order, not the clean historical data. My bike mistake.

If it still misses, a correction after the model. On recent weeks, for each condition, you find how many minutes to add so the coverage is right. There is a method for this, conformal something. I read about it, but I never used it in production. In 2020 we did it by hand, with a table per city and hour.

A stronger answer: one risk of your own fix

Hana describes the method before the name, which is the right order. (“Conformalized quantile regression” is the name she was looking for.) A slightly stronger answer adds where the fix can fail: “The correction must be fit on recent data that the model did not train on, or it will be too small.” Correcting per condition is sometimes called group-conditional (Mondrian) conformal. It assumes the coming weeks look like the weeks you calibrated on, so recalibrate often and watch coverage. Interviewers like to hear that you know the limits of your own idea.

Ioana(interviewer):

Now you have a new model. How do you test it online?

Hana(candidate):

Before that, one question. The couriers, are they shared? One courier, orders from different customers?

Ioana(interviewer):

Yes. A courier can carry two or three orders, and dispatch picks from all couriers in the zone.

Hana(candidate):

Then a normal A/B test by customer is a problem. If group B sees longer times, some of them don’t order. Then there are more free couriers, and group A gets faster deliveries. A looks better than it really is.

At Tandem we had this with drivers. We used switchback tests. The whole zone uses model A for some time, then model B, and you switch many times, in random order.

Ioana(interviewer):

How long is one window?

Hana(candidate):

Let me think. Thirty minutes? No, wait, a delivery is longer than that. An order from the end of one window is still on the road. So, one hour, maybe. And I would not count the orders from the first part of each window. Also, the unit is the window, not the order, so you have fewer units than you think. The test must be some weeks, with dinners and bad weather in it.

The question before the answer

Hana asks how couriers are shared before she designs the test. That question is the whole point: when customers share couriers, one group changes the other group’s deliveries. Her first guess for the window was too short, and she corrected it herself: a window must be longer than one delivery. That is reasoning, not memory.

Ioana(interviewer):

Last part. Our features come from an in-house feature store. Some are batch, like the store’s average prep time. Some are real-time, like courier locations and open orders, from a stream. At dinner rush, the stream sometimes lags, two, three minutes. What happens to your model?

Hana(candidate):

Sorry, the stream lags, you mean the values arrive late? Or they are missing?

Ioana(interviewer):

They arrive late. So the online store has old values, and the service reads… how do you say, it reads whatever is there.

Hana(candidate):

Then the model sees “five open orders” when it’s already twelve. Exactly at dinner, when it matters most. So it’s optimistic again.

At Tandem, each real-time value had a timestamp, and the service logged its age. If the age was over a limit, we used a fallback, a simpler value for that hour. And because we trained on the logs, the model had seen old values too.

Ioana(interviewer):

Okay. Here, the store builds the training tables. For an order at 18:42, it takes the last value before 18:42. But before 18:42 by which time? When the event happened, or when the value arrived in the store?

Long pause.

Hana(candidate):

Hmm. I have not used this kind of feature store. At my old company we did not have this question, because we trained on the logs. If your store uses the event time, then training sees fresh values and live sees the late ones. So training looks better than live. Like your skew bug, but smaller. I think you need the arrival time. But how your store builds the table, I don’t know. I would need to read the code, or ask who built it.

Ioana(interviewer):

That’s a good answer. Honestly, the store supports both, and different teams use different settings. Some of us only found out last year.

The edge

This is the edge of what Hana knows. A feature store builds a training table by joining each order to the feature values “as of” that moment, and the timestamp it uses decides whether training sees the same delays as live. Hana does not pretend. She reasons from what she did before to the right risk, and says what she would check.

A stronger answer at the edge

Hana could go one step further with what she already knows: “Whatever the store does, I would log the age of each real-time feature at prediction time, and give the age to the model as a feature. Then, when the age goes up at dinner, the model can learn to add time. And I would put an alert on the age, not only on the error after delivery.”

Ioana then asked briefly about the 50 ms budget. Hana said a LightGBM model like this usually scores one order in about a millisecond or less, so reading the features was the real risk. Ioana said there is a timeout, with a fallback to batch values.

Minutes 48 to 57: Hana's questions

Hana(candidate):

First question: monitoring. Do you monitor the features, or only the error?

Ioana(interviewer):

Only the error after delivery, nothing on features. That’s why the skew took six weeks. We know it’s a risk.

Hana(candidate):

Then the feature monitoring would be the first thing I want to work on. And, sorry, last one. I know my tools are a bit old. Kubernetes, MLflow, I used them only in courses. Is that a problem?

Ioana(interviewer):

I can’t tell you the decision. I can say that our tools are learned in a few weeks here. Everybody learns our feature store from zero, because nobody outside has it.

Hana also asked how the weekly retraining runs, and about on-call. Ioana told her honestly: a cron job that one engineer understands well, and on-call one week in five for the model service, which the posting does not say. Most weeks are quiet.

The good questions

Hana asks about the real risks of the job, and each answer tells her something true about the work. See An interview is a conversation in both directions.

The apology

The last question starts with “sorry” and asks the interviewer to remove a doubt. Hana did not need it. Her answers already showed that her tools gap is small and her judgment is strong. If you worry about a gap, say your plan, not your worry: “I used MLflow in my project. I have not used it in a team yet, and it is the first thing I would learn.” Or ask for information: “What would you want me to own after three months?”

Minutes 57 to 60: the close

Ioana(interviewer):

Thank you, Hana. Next is system design with Ben, and a conversation with Laura from operations. Chantal will write you this week.

Hana(candidate):

Thank you, this was interesting. I liked the feature store question.

After the call: Ioana’s notes

Ioana writes her notes right after the call, against the job description.

Hana Kowalczyk, ML deep dive

• Owning a model in production: strong evidence. Rebuilt a pickup ETA model, measured it with a test by city, corrected her own before-and-after number. Monitoring by city and hour, with an alert.
• Evaluation: strong evidence. Time-based split, with a clear reason. Coverage by condition, without being asked. Found a leak in her own recent project.
• Problem framing for the ETA: strong evidence. Strongest of the week. Found the cause (average vs late tail), proposed a quantile with a trade-off table for product and operations.
• Online experiments: strong evidence. Asked about shared couriers first, then switchbacks, with a reason for the window length.
• Real-time features and our tooling: some evidence. Strong instinct (log features, fallback on old values). Has not used a modern feature store. Clear about where she stopped.
• Current tools (Kubernetes, MLflow, LLM topics): little evidence. Courses and one project. Not a concern: we teach our platform to everyone.

Recommendation: strong yes. She thinks like an owner of the ETA model. Her gaps are the tools, and those are the easiest thing for us to teach.

What happened next

Ben’s system design interview was a yes. Hana was slower on Kubernetes and said so. Laura’s values conversation was a strong yes. Laura wrote: “She asked how couriers feel when the app promises a time they can’t make. Nobody else asked about couriers.”

In the debrief, one person mentioned “the three years away”. Ravi stopped the discussion there. He asked the panel to judge only the evidence from the interviews. Ioana added that Hana’s ETA judgment was the best they had seen, and that it does not expire. There was one vacancy, and Mateo Rojas was a strong second choice. The team chose Hana.

Ravi made an offer at the upper-middle of the band: CAD 158,000 plus bonus and equity. She did not ask for more salary. She asked for one thing: a four-day week for the first three months, at 80% of the salary, while her family set up a new routine. She also asked to start on-call after those three months. Ravi agreed to both and put them in the offer letter. She accepted.

This is one realistic ending, not a promise. Another company could have been less fair about the break, or could have hired someone with more current tools. Luck is part of it.

What to take from this

• A career break is a fact, not a confession. Say it in one sentence, say how you came back, and move on. Hana spent ten seconds on it, and the interview spent ten seconds on it.
• Judgment does not expire. Hana learned about time-based splits, skew and shared couriers from real failures years ago. In 2026 these are still the questions that matter most.
• At the edge, use what you know. Hana had never used a feature store like Fennick’s. She explained how her old team solved the same problem, reasoned to the right risk, and said what she would check.
• Do not apologise for your tools. Name the gap and your plan. Interviewers for this kind of role care more about how you think than about which tools you used last.

If your profile is like Hana’s

1. Prepare your break sentence, and say it out loud. One sentence about the break, one about how you returned. For example: “I took three years off to care for my children. This year I came back, refreshed with courses, and built a forecasting project.” You do not need to give more detail. In Ontario, family status is protected under the Human Rights Code, so an interviewer should not ask about your plans for children.
2. Put the break on your CV as a normal line. “Career break, full-time caregiver, 2022 to 2025.” A clear line stops people from guessing. See Portfolio anatomy and Prepare for applications.
3. Build one recent project that shows your old strengths with current tools. Hana chose a forecast with quantiles and a time-based split, close to the job she wanted. Our case study Forecast bike-share demand is a good start. Then add what a production team would add: a small service, and monitoring of coverage by condition.
4. Refresh the fundamentals that interviews check, not every new tool. Start with Train, validation and test sets, Slice analysis and Plan production checks. Then learn one current tool for each job you did before, for example MLflow for experiment tracking.
5. Write down your old projects before you forget the details. For each one, note the target, the split, the metric, what broke and how you found it. Explain decisions shows how to turn them into answers you can say aloud.
6. Learn how a feature store builds training data. Read about “point-in-time” joins and the difference between event time and arrival time. This is where Hana reached her edge, and it often comes up when you talk about real-time features.