Skip to content
Français

Bike-share demand and fleet planning

  • Case study 1
  • 150 min
  • Optional · 1 case study needed for the certificate

Where we are

In eight modules, we forecast Larkfield's orders and tickets and turned them into decisions. In this case study, we use real open data: bike rentals with the weather. We compare four method families in one backtest and decide how many bikes and crews to plan.

What you will be able to do

Take two years of a real bike-share system’s rentals to two plans with their cost: the bikes in service for the next 14 days, and the rebalancing crews for the next week. You compare four method families in the course’s backtest and measure what the weather would be worth. Then you plan at a quantile from labelled costs, test the choice once, and say where it failed.

Case studyPortfolio piece

Licence
CC BY 4.0
Time
About 2.5 hours
Runs in
Colab, Kaggle or VS Code
Skills used
  • Daily and hourly series
  • Known and unknown inputs
  • Seasonal naive, ETS and MSTL
  • LightGBM with mlforecast
  • Rolling-origin backtest
  • Coverage of a range
  • Plans at a cost quantile
  • Final test once

Colab and Kaggle open your own copy. To use VS Code, download the notebook from GitHub. Stuck? See the solution notebook.

The notebook runs every step of this page, with the same code and the same output, plus three tasks with checks. Its setup cells install the packages and download the data (280 KB) and the course’s backtest.py. It runs in about one minute after the install.

Before you startWhat to know and to have readyAbout 150 minutes

What you need to know

This case study uses the whole course. These lessons matter most:

The machine learning course uses the same data in Forecast bike-share demand, for the next day as a regression. Here we plan two weeks ahead, where no good weather forecast exists.

What you need on your computer

Nothing: the notebook runs in Colab or Kaggle. To run it locally, use a virtual environment with Python 3.12, pandas 2.3.3, statsforecast 2.1.1, mlforecast 1.1.0, LightGBM 4.7.0 and scikit-learn 1.9.1. The setup cell makes a new folder bike-share-forecasting. To start again, delete that folder.

Tested on macOS with Python 3.12, in Jupyter from a fresh start. Colab, Kaggle, Windows and Linux are not tested.

A bike-share system rents bikes from stations across a city. We use the real system of Washington, D.C., in 2011 and 2012.

Amara runs operations, and Daniel analyses the data, as in the machine learning course’s case study. We invented both people; the rentals are real. Every Sunday evening, Amara makes two plans:

  • The fleet, for the next 14 days. How many bikes are in service each day? The workshop plans its repairs two weeks ahead.
  • The rebalancing crews, for the next week. A crew is a van and a driver. It moves bikes from full stations to empty ones. Crews get their shifts one week ahead.

Each plan can go wrong in two ways. The costs are this case study’s assumptions, not the operator’s numbers:

  • A bike short for a day: a bike in service makes at most 5 rentals a day. A missing bike loses 5 rentals at about 8 dollars each: 40 dollars.
  • An extra bike in service for a day: checks, cleaning and a place at a station: 10 dollars.
  • An hour one crew short: a crew keeps up with 100 rentals an hour. Without it, stations run empty or full, and riders give up: 135 dollars.
  • An extra crew hour: the driver and the van: 45 dollars. Crews work from 06:00 to 22:59.

Amara asks Daniel: “How many bikes and crews should I plan? And should I buy a weather forecast service?”

You will build two forecasts: rentals per day for 14 days, and rentals per hour for one week. They end in a fleet plan and a crew roster with their cost. You test the choice once on two unseen weeks, and run it on a new week.

The data is Bike Sharing from the UCI Machine Learning Repository. day.csv has one row per day, from 1 January 2011 to 31 December 2012, and hour.csv one row per hour. Both count the bikes rented in the whole system (cnt), with the calendar and the weather of the day or hour.

These are the columns that this case study uses:

Column What it means
dteday The date.
hr The hour, 0 to 23 (only in hour.csv).
cnt The number of bikes rented: our target.
workingday, holiday 1 on a working day; 1 on a public holiday.
weathersit The weather type: 1 clear, 2 mist or cloud, 3 light rain or snow, 4 heavy rain or snow.
temp, hum, windspeed Temperature, humidity and wind, observed, and scaled to between 0 and 1.

We also looked at two other datasets. Seoul’s bike-share data has only one year, too little for a backtest over a year. Toronto’s data has no weather, and its licence is not on our list.

day = pd.read_csv("day.csv", parse_dates=["dteday"])
hour = pd.read_csv("hour.csv", parse_dates=["dteday"])
hour["ds"] = hour["dteday"] + pd.to_timedelta(hour["hr"], unit="h")
every_hour = pd.date_range(hour["ds"].min(), hour["ds"].max(), freq="h")
missing = every_hour.difference(hour["ds"])
same = bool((hour.groupby("dteday")["cnt"].sum().to_numpy() == day["cnt"].to_numpy()).all())
print(day.shape, hour.shape, "| days:", day["dteday"].min().date(), "to", day["dteday"].max().date())
print(f"missing hours: {len(missing)} | the days add up the hours: {same}")
print("days with the most missing hours:", {str(k): v for k, v in pd.Series(missing.date).value_counts().head(3).items()})
print("lowest days:", day.nsmallest(3, "cnt")[["dteday", "cnt"]].to_string(index=False))
(731, 16) (17379, 18) | days: 2011-01-01 to 2012-12-31
missing hours: 165 | the days add up the hours: True
days with the most missing hours: {'2012-10-29': 23, '2011-01-27': 16, '2012-10-30': 13}
lowest days: dteday cnt
2012-10-29 22
2011-01-27 431
2012-12-26 441

What this tells us. Two years have 17,544 hours, but only 17,379 have a row. 110 of the 165 missing hours are between midnight and 05:59, when almost nobody rents a bike. On 29 and 30 October 2012, Hurricane Sandy closed the system: 29 October had only 22 rentals. The two files agree.

An hour with no row gets 0 rentals and the weather of the hour before. Two more columns, casual and registered, add up to cnt. They are a leak, and we do not use them.

Frame the forecast asks a few questions before any model. Daniel’s answers:

  • What: bikes rented, for the whole system. It is close to demand, but an empty station loses rentals that nobody records.
  • Unit of time and horizon: two forecasts. Per day, 14 days ahead, for the fleet. Per hour, 168 hours (one week) ahead, for the crews.
  • When: Sunday at 23:00. Amara plans with the data up to that hour.
  • Level: the whole system. The data has no stations.
Rentals per day. Summer is high and winter is low, and 2012 is much higher than 2011. The single low point in late October 2012 is Hurricane Sandy.
Text description of the chart

The line shows the rentals of each day from January 2011 to December 2012: between 431 and 1,985 a day in January 2011, about 4,600 in July 2011, and between 4,100 and 8,200 in summer 2012. The busiest day is 15 September 2012 (8,714). On 29 October 2012, it drops to 22. In late December 2012, it is between 400 and 3,100.

What this tells us. The mean day had 3,406 rentals in 2011 and 5,600 in 2012: +64.4% in one year. So “the same day last year” is far too low, and the yearly season is hard to separate from the growth. The course’s weekly baseline, “the same weekday last week”, follows the level better.

  • Working days
  • Weekends and holidays
Working days have two commuter peaks, at 08:00 and 17:00. Weekends and holidays have one wide peak around midday.
Text description of the chart

Two lines over the 24 hours of a day. Working days: about 100 rentals at 06:00, a peak of 473 at 08:00, about 200 at midday, a higher peak of 524 at 17:00, and 138 at 22:00. Weekends and holidays: 19 at 06:00, a slow rise to a wide peak of 373 at 13:00, then 324 at 17:00 and 116 at 22:00.

What this tells us. The hourly series has a day season (24 hours) and a week season (168 hours). A Saturday has a different shape than a Tuesday. So the hourly forecast needs both seasons, as in Several seasons and long series.

Days with rain or snow have far fewer rentals. Part of the gap is the season, because rain and snow are more common in winter. Against its own month's mean, a day with rain or snow has 52.5% fewer rentals.
Text description of the chart

A column chart with three columns. Clear days (463 days): 4,877 rentals on average. Mist or cloud (247 days): 4,036. Light rain or snow (21 days): 1,803, highlighted.

What this tells us. The weather moves a day by half. But does Amara know the weather of a day in two weeks? No.

Think of a picnic planned for a Saturday in two weeks. You know the date and the weekday. You do not know if it will rain.

Calendars, holidays and events called an input a known future input when the business really has it on the forecast day. The weekday, the month and the public holidays are known. The weather of a future day is not: a weather service forecasts a few days ahead well, but not two weeks.

The data has the observed weather of every day. A model can take it as an input for future days, and the code runs without an error. Its backtest then looks better than any real plan can be: a leak from the future, as in Gradient boosting with LightGBM.

On the left, the past: rentals per day and per hour, the weather so far and the calendar so far. A dashed line marks Sunday 23:00, when Amara plans. On the right, the next 14 days: the weekday, month, holidays and working days are known now and may be used; the rentals and the weather are not known and must be forecast. A note says that a model with the real weather of future days is only a test of what a perfect weather forecast would be worth.
What Amara knows on Sunday at 23:00. The calendar of the next 14 days is known; their weather is not.
Text description of the diagram

A dashed line at Sunday 23:00 splits the known past (rentals, weather and calendar so far) from the next 14 days. A green arrow leads to what is known now: the weekday, the month, the holidays and the working days. An orange arrow leads to what must be forecast: the rentals and the weather. A note says that a model with the real future weather only tests what a perfect weather forecast would be worth.

So Daniel trains LightGBM twice. The honest version has past rentals and the calendar only. The second version gets the real weather of each future day. Daniel labels it impossible and never plans with it. It shows what a perfect weather forecast would be worth.

A plan must also choose how much risk to take. From forecast to order and roster gave the rule. Plan to the quantile at the critical ratio: the cost of a unit short, divided by both costs added up.

  • Fleet: 40 ÷ (40 + 10) = 0.8. A bike short costs four times an extra bike, so Amara plans high: the 80% quantile.
  • Crews: 135 ÷ (135 + 45) = 0.75.
Two rows. The fleet: rentals per day for the next 14 days, then the 0.8 quantile (40 divided by 40 plus 10), then bikes in service per day equal to the plan divided by 5. The crews: rentals per hour for the next 7 days, then the 0.75 quantile (135 divided by 135 plus 45), then crews per hour equal to the plan divided by 100, from 06:00 to 22:59. The costs of each mistake are next to each row.
From the two forecasts to the two plans. The quantile comes from the costs; the plan comes from the quantile.
Text description of the diagram

Two rows of three boxes. The fleet: rentals per day for 14 days, the 0.8 quantile, then bikes per day, the plan divided by 5, rounded up. The crews: rentals per hour for 7 days, the 0.75 quantile, then crews per hour, the plan divided by 100, from 06:00 to 22:59. Each row shows its two costs. A note says that the costs are assumptions.

A tiny example. Daniel forecasts 5,000 rentals for a Tuesday, with an 80% range of 4,000 to 6,000. The spread is about 2,000 ÷ 2.56 = 780 rentals. The 0.8 quantile is about 5,000 + 0.84 × 780 = 5,657 rentals. That needs 5,657 ÷ 5, rounded up: 1,132 bikes, against 1,000 bikes at the forecast itself. (The 80% range runs from the 10% to the 90% quantile. In a normal range, they are 1.28 spreads from the middle; 0.84 spreads above it is the 80% quantile.)

So each method needs an honest 80% range.

The crew cost is the course’s bt.roster_cost() with the crew numbers. The fleet cost has the same shape, per day:

def fleet_cost(fc, column="yhat"):
"""The cost of a 14-day fleet plan, per origin: bikes in service each day = plan / 5, rounded up."""
bikes = np.ceil(fc[column].clip(lower=0) / RENTALS_PER_BIKE)
need = np.ceil(fc["y"] / RENTALS_PER_BIKE)
cost = np.maximum(need - bikes, 0) * BIKE_SHORT + np.maximum(bikes - need, 0) * BIKE_EXTRA
return float(cost.sum() / fc["origin"].nunique())
crew_cost = bt.roster_cost(per_agent=RENTALS_PER_CREW, agent_hour=CREW_EXTRA, short_hour=CREW_SHORT,
open_hours=CREW_HOURS)
def at_quantile(fc, q):
"""The plan at the q quantile: the forecast plus z(q) spreads, the spread from the 80% range (normal)."""
sd = (fc["q90"] - fc["q10"]) / (2 * norm.ppf(0.9))
return (fc["yhat"] + norm.ppf(q) * sd).clip(lower=0)

Daniel uses the course’s backtest.py. There are 13 cut-offs (the code calls them origins), every fourth Sunday from 18 December 2011 to 18 November 2012. So every season of 2012 is covered, with a year of history behind each cut-off. Each method is refit at every cut-off. The 14 days after 2 December 2012 are kept aside for the final test.

The methods, each with its reason:

  • Seasonal naive: the same weekday last week (per day), the same hour last week (per hour). The course’s baseline.
  • ETS with a weekly season, per day.
  • MSTL with a day season and a week season, per hour, fitted on the last 12 weeks, as in Several seasons and long series.
  • LightGBM, no weather: one model with past rentals, the weekday, the month and the holiday and working-day flags. Per day, every lag is at least 14 days old, so it is known for all 14 days. Its range comes from 4 earlier windows: a conformal range, as in Score the range.
  • LightGBM, real weather (impossible): the same, plus the observed weather of every future day or hour.
DAILY = {
"seasonal naive (7 days)": stats(lambda: SeasonalNaive(season_length=7), "D"),
"ETS (weekly season)": stats(lambda: AutoETS(season_length=7), "D"),
"LightGBM, no weather": lightgbm("D", CALENDAR, [14, 21, 28], {14: [RollingMean(7), RollingMean(28)]},
["dayofweek", "month", "dayofyear"], LGB_DAY),
"LightGBM, real weather (impossible)": lightgbm("D", CALENDAR + WEATHER, [14, 21, 28],
{14: [RollingMean(7), RollingMean(28)]},
["dayofweek", "month", "dayofyear"], LGB_DAY),
}
daily_table, daily_fc = run(DAILY, daily, 14, ORIGINS, 7, fleet_cost, FLEET_RATIO)

The notebook defines the hourly methods the same way. run() scores each method with bt.score() and bt.coverage(), and computes the plan’s cost at the forecast and at the cost quantile.

method MASE WAPE % bias % pinball 80% holds cost at forecast cost at quantile
seasonal naive (7 days) 1.216 18.1 1.4 0.050 0.808 66812 56404
ETS (weekly season) 1.198 17.8 3.6 0.051 0.813 60582 52815
LightGBM, no weather 1.205 18.0 -1.4 0.049 0.747 72882 48474
LightGBM, real weather (impossible) 1.025 15.2 -4.1 0.040 0.742 68329 37821

The costs are in dollars per 14-day fleet plan.

What this tells us. The three honest methods have almost the same error: WAPE 17.8% to 18.1%. MASE is above 1 for every method: 14 days ahead is harder than the one week ahead that sets the scale.

But the costs differ. At the cost quantile, LightGBM’s plan is the cheapest: 48,474 dollars, against 56,404 for the seasonal naive plan. Every plan costs much less at the quantile than at the forecast: planning high pays.

With the real weather, WAPE falls to 15.2%, and the plan to 37,821 dollars. That gap is the value of a perfect weather forecast.

method MASE WAPE % bias % pinball 80% holds cost at forecast cost at quantile
seasonal naive (168 hours) 1.184 23.8 0.1 0.072 0.839 7705 6781
MSTL (day and week) 1.134 22.7 0.2 0.125 0.992 7287 9768
LightGBM, no weather 1.116 22.3 0.2 0.059 0.700 7321 5853
LightGBM, real weather (impossible) 0.916 18.3 -1.0 0.051 0.684 6068 4915

The costs are in dollars per week of crews.

What this tells us. LightGBM without weather has the best error (22.3%) and the cheapest roster at the quantile (5,853 dollars a week). MSTL’s plan at the quantile is the most expensive: 9,768 dollars. Its 80% range holds 0.992 of the real hours; the failure case below shows why.

LightGBM’s range is too narrow: it holds 0.700, not 0.80. Its plan is still the cheapest, because the cost quantile pushes it up anyway.

Amara pays for both plans every week. So the measure is the cost of one week: half a 14-day fleet plan plus one week of crews, both at the cost quantile. The notebook ranks every pair of honest methods:

The cost of one week: half a fleet plan plus one week of crews, at the cost quantiles, over 13 cut-offs. LightGBM for both plans is the cheapest honest choice. *The real weather is impossible: it shows what a perfect weather forecast would be worth.
Text description of the chart

A column chart with four columns. The same weekday or hour last week: 34,983 dollars a week. ETS per day and MSTL per hour: 36,176. LightGBM for both: 30,090, the lowest honest one, highlighted. LightGBM with the real weather, impossible in practice: 23,826.

per day per hour fleet crews per week
LightGBM, no weather LightGBM, no weather 24237 5853 30090
LightGBM, no weather seasonal naive (168 hours) 24237 6781 31018
ETS (weekly season) LightGBM, no weather 26408 5853 32260
...
seasonal naive (7 days) seasonal naive (168 hours) 28202 6781 34983
with the real weather (impossible): 23826 dollars a week

What this tells us. Daniel chooses LightGBM without weather for both plans: 30,090 dollars a week, 4,893 dollars less than the baseline. Small differences in error hid large differences in cost.

Amara’s question about a weather service now has a number. A perfect weather forecast would save 30,090 − 23,826 = about 6,300 dollars a week. A real 14-day weather forecast would save less.

The notebook moves the quantile of the two LightGBM plans from 0.5 to 0.9:

The cost of the LightGBM fleet plan is lowest at the critical ratio, 0.8. The plan at the forecast itself (0.5) costs half as much again.
Text description of the chart

The line shows the cost of the fleet plan for seven quantiles: 0.5 72,882 dollars; 0.6 60,495; 0.7 51,881; 0.75 49,165; 0.8 48,474, the lowest point, marked as the critical ratio; 0.85 49,547; 0.9 52,755.

quantile fleet per 14 days crews per week
0.50 72882 7321
0.60 60495 6148
0.70 51881 5819
0.75 49165 5853
0.80 48474 5909
0.85 49547 6207
0.90 52755 6767

What this tells us. For the fleet, the cheapest quantile is the critical ratio, 0.8. For the crews, 0.7 is cheapest (5,819), next to the critical ratio 0.75 (5,853): the curve is flat there. Far from the bottom, the cost rises fast on both sides.

Common mistake

A range that is too wide

Symptom: per hour, MSTL has a better WAPE than the seasonal naive forecast (22.7% against 23.8%). But its roster at the 0.75 quantile costs 9,768 dollars a week, the most of all.

Cause: its 80% range is far too wide: 574.4 rentals an hour on average, against 151.9 for LightGBM. It holds 0.992 of the real hours instead of 0.80. The plan adds part of that width to every hour, so it books crews that nobody needs. At the forecast itself, MSTL’s roster costs 7,287 dollars.

Fix: score the range before you plan from it, as in Score the range. Choose by the cost of the plan, not by the error alone. Larkfield’s ticket roster showed the same with Theta, in From forecast to order and roster.

Consequence in AI work: a quantile plan is only as good as the range under it. A range that is too wide looks safe and costs money every hour.

  • Real rentals per hour
  • MSTL: top of its 80% range
  • LightGBM: top of its 80% range
One week of the backtest, from the cut-off of 3 June 2012. The top of MSTL's 80% range is far above the real rentals, even at night, and it widens every day. LightGBM's follows the daily shape.
Text description of the chart

The chart shows 168 hours, 4 to 10 June 2012. The real rentals have two peaks on each working day, up to 869 in an hour, and one wide peak at the weekend. They fall close to zero every night. The top of LightGBM’s 80% range follows the same shape, a little above: at 03:00, it is between 6 and 83. The top of MSTL’s range reaches 1,236 at the peaks. At 03:00, when about 5 bikes are rented, it rises from 132 on Monday to 652 on Sunday.

Daniel has chosen LightGBM without weather for both plans. Now he runs it once on the 14 days after 2 December 2012, which no step has seen. He runs the other methods too, and changes nothing after.

plan method WAPE % bias % cost at quantile
fleet, 14 days seasonal naive (7 days) 13.2 -4.7 46290
fleet, 14 days ETS (weekly season) 14.1 -6.0 32460
fleet, 14 days LightGBM, no weather 22.8 -21.5 55250
fleet, 14 days LightGBM, real weather (impossible) 20.7 -20.0 48270
crews, 1 week seasonal naive (168 hours) 21.0 -7.0 5715
crews, 1 week MSTL (day and week) 20.2 -7.4 10035
crews, 1 week LightGBM, no weather 23.2 -13.0 5490
crews, 1 week LightGBM, real weather (impossible) 20.8 -18.6 5130

What this tells us. The result is mixed. For the crews, the choice held: LightGBM’s roster was the cheapest honest one (5,490 dollars). For the fleet, the choice lost: LightGBM was 21.5% too low, and its plan cost 55,250 dollars against 32,460 for ETS.

Why? LightGBM learned December from December 2011. In the first half of December, the system had 3,210 rentals a day in 2011 and 5,266 in 2012. A tree cannot forecast above the values that it has seen, as Gradient boosting with LightGBM warned; ETS can follow a level that grows. No cut-off in the backtest had a December with a year of growth behind it.

Daniel reports this as it is. If he switched to ETS now, these two weeks would become a second validation set, with no test left.

The last step plans the next week from all the data up to Sunday 23 December 2012. Then the real week arrives:

ds yhat bikes y bikes needed holiday
2012-12-24 3984.0 1055 920.0 184 0.0
2012-12-25 3824.0 1059 1013.0 203 1.0
2012-12-26 3859.0 1065 441.0 89 0.0
2012-12-27 4135.0 1097 2114.0 423 0.0
2012-12-28 4006.0 1059 3095.0 619 0.0
2012-12-29 3343.0 876 1341.0 269 0.0
2012-12-30 2467.0 550 1796.0 360 0.0
rentals: forecast 25617 | real 10720 | fleet cost of the week: 46140

What this tells us. The plan puts about 1,060 bikes in service on most days; about 200 were needed on 24 and 25 December. The model forecast 25,617 rentals, and only 10,720 came. The fleet costs 46,140 dollars, almost twice a normal week. The crews were planned for 388 crew hours, and 174 were needed.

The data marks only 25 December as a holiday. But the whole week between Christmas and New Year is quiet: many people are away, and offices are closed. That is a calendar fact, as in Calendars, holidays and events, and the model has no input for it.

A planner knows last Christmas. The week of 24 to 30 December 2011 had 54.3% of the mean week of the two weeks before it. The forecast gave this year’s week 81.8%. Daniel scales the plan down to last year’s share:

last year's share: 0.543 | bikes: 4493 | fleet cost of the week: 23460
  • Real rentals per day
  • LightGBM forecast: 25,617 for the week
  • With last year's share (0.543)
Christmas week 2012. The forecast expected an ordinary winter week. Last year's share brings the plan much closer, and halves the cost of the fleet.
Text description of the chart

Seven days, 24 to 30 December 2012. The real rentals: 920, 1,013, 441 (a rainy day), 2,114, 3,095, 1,341 and 1,796. The LightGBM forecast is about 4,000 a day, and 2,467 on the Sunday. With last year’s share, it is about 2,600 a day, and 1,600 on the Sunday: still above most real days.

What this tells us. The calendar rule cuts the bikes from 6,761 bike-days to 4,493, and halves the cost, from 46,140 to 23,460 dollars. It is still too high: the real week had only 34.2% of the weeks before, and it rained on 26 December. This rule comes from one earlier week, so treat it as a planner’s rule, not a tested method.

Daniel writes to Amara in her words:

  • Plan both from LightGBM without weather, at the cost quantiles. Plan the bikes at the 80% quantile of each day, and the crews at the 75% quantile of each hour. Over 13 cut-offs in 2012, this cost 30,090 dollars a week, against 34,983 for “the same weekday or hour last week”.
  • Plan high on purpose. The same LightGBM fleet plan at the forecast itself would cost 24,408 dollars more per 14 days (72,882 against 48,474).
  • Check the fleet plan against ETS in December. In the final test, LightGBM’s fleet was 21.5% too low, and ETS was cheaper.
  • Apply last year’s share to Christmas week. No model here knows it. With the share, the fleet cost of the week falls by half.
  • A weather service is worth at most about 6,300 dollars a week for these two plans: the gain from a perfect forecast. Ask a provider how accurate it is 7 and 14 days ahead before you pay.

And what not to use it for:

  • Not for single stations. The data has only system totals.
  • Not for another city or a later year without a new backtest. This system grew 64% in one year.
  • Not as a promise to the workshop. It is a forecast with a known error (WAPE 18.0% per day), not a contract.
  • The costs are ours. 40 and 10 dollars per bike-day, and 135 and 45 per crew hour, are assumptions. With other costs, the choice can change.
  • Five rentals per bike is a simplification. A real fleet plan also depends on the peak hour and on where the bikes are.
  • One December, fast growth. No cut-off was in December with a year of growth behind it. The final test showed the cost of that gap.

The course’s Watch the forecast shows how a bias alarm would catch a fleet plan that runs 21% too low two weeks in a row.

  • The data is about people’s journeys, added up. Do not join it with trip-level data to follow single riders.
  • Credit the source. CC BY 4.0 allows you to share and change the data, with the attribution text above and a note of your changes.
  • A plan affects workers. The crew roster decides people’s shifts. A plan that is too low leaves drivers overloaded. Share the error and the bias with the people who staff from it.
  • Label the assumptions and the impossible model. The costs, the five rentals per bike and the Christmas rule are choices. Nobody can get the “real weather” result. Say so wherever you show them.

This case study shows an employer that you can take a real series to two plans with their cost. It also shows that you can tell an honest input from a leak, and report a result that goes against you. Make it yours with one change: other costs, a noisy weather input, or a third plan. Then make it public and easy to read in five minutes.

What to put in the repository:

  • README.md, the page that people read first (template below).
  • The notebook, run from top to bottom, with its outputs.
  • backtest_daily.csv and backtest_hourly.csv: every method with MASE, WAPE, bias, coverage and both costs; and final_test.csv, kept apart.
  • assumptions.md: the four costs, five rentals per bike, 100 rentals per crew hour, the crew hours, and the Christmas rule.
  • The data source and licence, with the attribution text above. A link and the checksum of the zip are enough.

A README template:

# Bike-share fleet and crew plans from two forecasts
## Problem
A bike-share system plans its bikes in service 14 days ahead and its rebalancing crews
one week ahead. A short bike-day costs 40 dollars, an extra one 10; a short crew hour
costs 135, an extra one 45 (assumptions). Critical ratios 0.8 and 0.75.
## Data and licence
UCI "Bike Sharing" (Fanaee-T), Capital Bikeshare, Washington DC, 2011-2012, per day
and per hour, CC BY 4.0, https://doi.org/10.24432/C5W894. 165 missing hours set to 0.
## Approach
Backtest: 13 cut-offs Dec 2011 - Nov 2012, 14 days (daily) and 168 hours (hourly).
Seasonal naive, ETS, MSTL, LightGBM without weather; LightGBM with the observed
weather as an impossible upper bound. Plans at the cost quantile from the 80% range.
## Results
LightGBM for both plans: 30,090 dollars a week vs 34,983 for the seasonal naive pair.
A perfect weather forecast would save at most about 6,300 dollars a week. MSTL's range
was too wide (coverage 0.992), so its roster was the most expensive.
Final test (3-16 Dec 2012): crews held; the fleet plan was 21.5% too low. Reported,
not tuned. Christmas week needs a calendar rule.
## Limits
System totals only; rentals not demand; assumed costs; one December in the history.

How to describe it in an interview, in three sentences: “I planned a real bike-share system’s fleet and crews from daily and hourly forecasts, and chose LightGBM by the weekly cost of both plans. A perfect weather forecast would have been worth at most about 6,300 dollars a week. The final test showed the fleet plan 21.5% too low in December, and I reported it with a fix instead of tuning on the test.”

Go further:

  • Add noise to the observed weather to make a “14-day forecast”, and see how much of the 6,300 dollars is left.
  • Give LightGBM the series divided by its 28-day mean, so that it learns shapes instead of levels, and backtest it in December.
  • Compare with the other case studies, Electricity load for many customers and Retail demand with a hierarchy: where does the weather matter, and where is the cost ratio far from one half?

Before you run it, predict: a manager wants the crews at the 0.9 quantile “to be safe”. Will a week cost more or less than at 0.75 (5,853 dollars)? In the notebook, section 7, compute the crew cost of the LightGBM plan at the 0.9 quantile, and put it in crew_90.

Show the answer
crew_90 = round(crew_cost(lgb_h.assign(p=at_quantile(lgb_h, 0.9)), "p"))
print(crew_90)
6767

A week of crews at the 0.9 quantile costs 6,767 dollars: 914 dollars more than at 0.75. The extra crews avoid a few short hours, but they cost 45 dollars in every hour where they are not needed. “Safe” has a price, and the critical ratio tells you where it stops paying.

Check your understanding

Daniel's LightGBM with the observed weather has the best WAPE and the cheapest plans in the backtest. Why does he never plan with it, and what is it still good for?

Show the answer

The observed weather of a future day is not a known input. On Sunday evening, nobody knows the weather of a day in two weeks. So the model uses information from the future, and its backtest looks better than any real plan can be. It is still useful as an upper bound. The gap between it and the honest model, about 6,300 dollars a week, is the most that a perfect weather forecast could save. That answers Amara’s question about a weather service.

Check your understanding

Per hour, MSTL has a better WAPE than the seasonal naive forecast, but its roster at the 0.75 quantile costs the most. Explain why, and what Daniel should check before he plans from any range.

Show the answer

The plan at a quantile is the forecast plus part of the range’s width. MSTL’s 80% range is far too wide. It holds 99.2% of the real hours instead of 80%, and it is 574 rentals an hour wide on average. So its 0.75 quantile is far above the real rentals in most hours, and the roster books many crews that nobody needs. A good point forecast does not make a good range. Daniel should check a range’s coverage in the backtest before he plans from it, and compare methods by the cost of the plan.

Check your understanding

In the final test, LightGBM's fleet plan was 21.5% too low, and ETS was cheaper. Should Daniel switch the fleet to ETS now? What should he do instead?

Show the answer

No. The final test is used once, to report how the choice performs. If he switches because of it, those two weeks become a second validation set, and no honest test is left. He should report the result and its cause: LightGBM learned December from a much smaller year, and a tree cannot forecast above the values it has seen. For the next cycle, he can compare the fleet plan with ETS in December. He can backtest a version that learns shapes instead of levels, and watch the bias with an alarm.

Recap

  • You forecast real rentals per day for 14 days and per hour for one week. You turned them into a fleet plan and a crew roster.
  • The honest methods had almost the same error. The cost of a week chose LightGBM for both plans (30,090 dollars a week, against 34,983 for the baseline).
  • The observed weather is not a known input. As an “impossible” upper bound, it shows that a perfect weather forecast is worth at most about 6,300 dollars a week.
  • MSTL’s range was far too wide (coverage 0.992), so its quantile roster cost the most: score the range before you plan from it.
  • The final test went partly against the choice: the fleet plan was 21.5% too low in December. Christmas week needs a calendar rule.

Next, the second case study: hourly electricity load for 80 customers, with quantile forecasts and a day-ahead purchase.

Next: Electricity load for many customers