Back-testing fantasy basketball drafts and weekly moves

Abstract

Superseded in part: the first two paragraphs report the original studies, which used prior-season statistics only. The update below the abstract, and sections 8 to 12, report later work with an age and games-played model that did find a drafting edge in simulation.

We ran two simulation studies on free, public NBA data. Study 1 simulates 10-team snake drafts for the 2017 to 2026 seasons (end-year labels), 44,000 drafts per run and 4,000 per strategy, and scores the resulting rosters on full-season nine-category roto results. Eleven strategies were compared against drafting by an average-draft-position (ADP) proxy. No strategy beat the baseline on held-out seasons. A guardrail variant that gained +1.74 roto points on the training seasons (2017 to 2021) gained +0.11 on the test seasons (2022 to 2026), with a confidence interval that includes zero.

Study 2 simulates in-season management with weekly granularity from game-level box scores, using only information available at the start of each week. A policy that replaces players who have missed their team's last three games with the best available free agent improved category win rate by 7.2 points over a static roster on the test seasons (95% CI 6.4 to 8.0), and the gain was positive in every test season. More active streaming and waiver policies did not improve on it.

Update (15 October 2026): the combined strategy study

The sections below this update describe the first two studies, which used prior-season statistics only and found no drafting edge. We then added an age curve and a games-played model, tested out of sample (fit on target seasons 2016 to 2021, scored on 2022 to 2026), and re-ran the simulation under the live league rules. Where the older sections and this update disagree, this update governs.

Main results, all on test seasons 2022 to 2026 with 95 percent bootstrap intervals over season by draft-slot cells: the age-and-games-played draft model, with weekly expected-value lineups, replacing injured players, and low-margin streaming, wins 60.8 percent of category contests against all opponents, against 53.6 percent for a consensus-proxy draft that also replaces injured players. The difference is +7.2 points [+6.4, +8.0]. About two thirds of it comes from the draft model and a third from in-season management. The comparator is our own proxy for ADP, not real Yahoo ADP, so the draft part is probably overstated.

1. Introduction

Fantasy basketball advice tends to be asserted rather than tested. Preseason projections and ADP archives are mostly paywalled or overwritten each year, which makes honest back-testing hard: a test is only valid if the strategy sees nothing that was not known before the season began. We asked two questions. First, can a model that uses only prior-season statistics outdraft a consensus-style ADP ranking? Second, once the season starts, does disciplined weekly roster management produce measurable gains over leaving a roster alone?

We report results on held-out seasons and say where the design is generous or weak. We did not have real historical ADP or expert projections, so our market baseline is a proxy built from the same prior-season data. That choice favors our strategies relative to a real market, and the results should be read with that in mind.

2. Data

Study 1 uses season per-game tables from Basketball-Reference for 2015 to 2026 (12 pages, fetched 7 seconds apart to respect the site's rate limit). We kept one row per player, taking the combined row for players traded mid-season. Study 2 uses game-level player box scores from the open sportsdataverse/hoopR-nba-data repository on GitHub (ESPN-derived, regular season only, 2015 to 2026). The data are free and no paywalled or access-restricted source was used.

Study 1 source

basketball-reference.com/leagues/NBA_YYYY_per_game.html

Study 2 source

github.com/sportsdataverse/hoopR-nba-data (player_box parquet files)

Seasons tested

2017 to 2026 (10), using Y-1 and Y-2 data as inputs

Train / test split

2017 to 2021 / 2022 to 2026

We also checked Basketball Monster's free "Preseason" pages as a possible projection source. They contain statistics from a few actual NBA preseason games per player (1 to 5 games), not projections, and optional ADP columns could not be populated without an account. We did not use them. No free archive of true historical preseason ADP or analyst projections was found.

3. Study 1: drafting

4. Study 1 results

Run 2 used projection parameters chosen on the training seasons only: a games-played discount strength of 0.5, an age penalty of 0.03 per year over 29, a prior worth 300 minutes, and a weight of 0.3 on the older season. The choice came from a 54-combination grid over the projection model's parameters, scored on training seasons with 8 repeats. Because that is a search, the training numbers below are optimistic by construction. The test numbers are the honest ones.

Roto points are out of 90, league average 49.5. Difference is the paired mean versus the ADP proxy, with a bootstrap 95% interval over season-by-slot cells.

5. Study 2: weekly management

6. Limitations

• The ADP baseline is a proxy built from the same prior-season statistics the strategies use. A real market also prices in rookies, injuries, trades and expert news, so real ADP is harder to beat and the Study 1 results are, if anything, generous to our strategies.
• Players with no stats in the two prior seasons cannot be drafted in Study 1. Real drafts include rookies.
• Roto scoring counts each team's best 10 players by actual full-season value, a hindsight form of bench management that softens injury losses equally for all teams and probably favors star-heavy rosters.
• The 2019-20 season was interrupted and finished in a bubble, and 2020-21 was shortened to 72 games. We include both and treat the weekly calendar as is.
• Study 2 has no positions, no injured-list slots, no weekly category targeting, a fixed three-add weekly cap, and ignores contention between teams for the same free agent.
• The static baseline in Study 2 never replaces anyone. Part of the replace-injured gain is the cost of leaving a thin roster alone, so the headline difference overstates the gain over a manager who already handles injuries. The better comparison is the active-opponent column, where the policy still scored above 50 percent.
• Study 2 used a different data source than Study 1 and the two were not reconciled.
• The policy parameter grids were searched on training seasons. The test seasons were run once for the five policies shown. Other choices were not re-tuned.
• Intervals treat season-by-slot cells as independent. Seasons share many players, so they are likely narrower than they should be.

7. Conclusion

Using prior-season statistics alone, no drafting strategy reliably beat a consensus-style ranking (studies 1 and 2). Adding an age curve and a games-played model changed that in simulation: a draft model using them, with weekly expected-value lineups, replacing injured players and low-margin streaming, beat a consensus-proxy draft by about 7 points of category win rate on held-out seasons. About 5 points come from the draft model and the rest from management. The draft edge exists only to the extent real drafters under-price age decline and injury history, which we could not test because historical ADP is not available for free. Category-targeting lineups did not help. Conditional age effects were tested: minutes load matters, position does not. No claim here is a forecast for 2026-27.

8. Age, breakout and games-played models (out of sample)

League rules used: head-to-head nine categories (FG%, FT%, 3PM, PTS, REB, AST, STL, BLK, TO), 10 teams, 14 draft rounds, 11 position-eligible starters, 3 bench, 2 IL, at most 50 acquisitions, weekly lineups. Models are ridge regressions fit on earlier target seasons only. Test set: 1,587 player-seasons.

Adding age cuts test squared error by 0.365 [0.237, 0.488]. Adding minutes, usage, late-season form and team change brings test R-squared from 0.682 to 0.720. A gradient-boosted model was worse than ridge.

Breakouts (a rise of 2 or more z-points, 15.5 percent of cases): the classifier reaches AUC 0.720 against 0.655 for a base model, a gain of +0.065 [0.033, 0.099]. Its top decile hits 35.4 percent against 30.4 percent, but that difference's interval includes zero [-0.025, +0.133]. A simple rule, age 24 or under with minutes up more than 2, hit only 18 percent. We treat the breakout signal as modest and did not use it in the draft model.

Games played: predicting next-season games played from two seasons of history, age, minutes, usage and prior low-minute games gives R-squared 0.169, against 0.133 for history alone and below zero for a league mean. Expected games next season by prior games: under 41 gives 42.6, 41 to 57 gives 51.1, 58 to 69 gives 58.0, 70 and over gives 64.1. Age costs about 1 game at 29 to 30, 3 at 33 to 34, 6 at 35 and over, and nothing at 25 to 28. Prediction is weak but real.

9. Does the age effect depend on who the player is?

We interacted the age curve with position, minutes load, prior-year games played, value tier and statistical style, fitting on 2016 to 2021 and scoring on 2022 to 2026. An interaction counts only if its test error gain has an interval above zero.

• Minutes load matters. Players at 30 or more minutes decline far less after 29 (-0.27 at 29-32, -0.40 at 33 and over) than 22-30 minute players (-0.76, -1.38) or bench players (-0.52, -1.21).
• Stars (value above 3, n=263) show no decline through 33 and over in sample (+0.13, +0.06) while role players fall hard (-0.55, -1.27). This did not pass the formal test and overlaps with minutes load, so it is a direction, not a rule.
• Position does not matter. Guards, wings and bigs have similar curves.
• Players who missed many games (under 60 percent) get a weaker youth bump (+0.23 against +0.72) with similar decline.
• Rebound- and block-heavy players decline slightly more at 33 and over (-1.24 against -0.76). The gain is real but tiny.

The per-category version of the conditional model gave mixed, tiny gains across the nine categories, so the draft model keeps the flat age curve. The conditional effects are reported here as findings, not shipped as adjustments.

10. Combined draft and in-season strategy under the live rules

Ten seasons, 1,000 simulated leagues per season, every draft slot, 5,000 simulations per strategy per half. Score is the all-play category win rate (share of the nine categories won against each of the nine opponents each week; 0.50 is average). Strategies were chosen on 2017 to 2021 and scored once on 2022 to 2026.

• Category-targeting lineups (chasing the weekly opponent's weak categories) did not beat plain expected-value lineups at any setting tested: 0.606 against 0.608.
• Replacing injured players is the largest in-season move. Streaming at a low margin adds about 2.4 points and uses about 13 to 15 of the 50 acquisitions. A higher margin did worse on training seasons.
• The draft model with no in-season moves gains little over the proxy (+0.010, interval touches zero). Draft and management gains add up.
• The guardrail: take the highest model value among players whose consensus rank is no worse than the current pick number plus 8; if fewer than 3 qualify, take the best model value left.

11. What-if scenarios

We asked what happens when each assumption fails. 10 seasons, 800 simulations per cell. Edge is the full strategy against ADP proxy with replace-injured.

The strategy never lost to the baseline. The draft edge disappears if the market already prices age and games played as we do, and shrinks to the management edge if projections are noisy. Injuries to our top picks lower both sides but leave the edge intact. In-season management is robust in every scenario.

A separate check on low-games-played former stars (Young, Tatum, Davis class): lighter shrinkage of thin-sample seasons did worse out of sample across 517 player-seasons (squared error 1.15 current against 1.21 to 1.38 lighter), and only 7 former stars were available to test an exception, which is too few to conclude anything.

12. Pick-1 deep dive and season-to-season variance

For the first draft slot we ran 250 leagues per season for 10 seasons (2,500 per strategy), paired against the baseline (ADP-proxy draft, replace injured). All-play category win rate with edge and 95 percent interval from simulation noise within seasons.

On the test seasons alone (2022 to 2026) the model draft edge at slot 1 is +0.072 [+0.068, +0.077]. Across the ten draft slots the edge is about the same, so the first pick neither helps nor hurts it.

These intervals do not cover season-to-season variation. The model draft edge by season: 2017 +0.054, 2018 +0.054, 2019 +0.065, 2020 +0.091, 2021 +0.100, 2022 +0.133, 2023 -0.003, 2024 +0.063, 2025 +0.126, 2026 +0.043. It was positive in nine of ten seasons and about zero in 2023. A single season can land anywhere in that range.

Guardrail width (ADP plus 4, 8, 12, 30, or none) made no difference beyond noise, and the order reversed between train and test seasons, so width is not a source of edge.

Board spot-check vs category consensus

Separate from the simulations, we compared the top 15 of the draft board against external category-league rankings for 2026-27, checked on the morning of 15 October 2026. The league format is head-to-head nine-category with no punting. No board ranks were changed as a result. A player is flagged by one uniform rule: the absolute gap between board rank and the FantasyPros category expert consensus rank (ECR, six experts, dated 13 October) exceeds 3. The expert range is the spread of opinion, not a confidence interval, and the other sources are shown side by side rather than pooled into a consensus.

Negative means the board ranks the player earlier than the FantasyPros consensus. Hashtag is its displayed projection rank.

Appendix: code and run evidence

The code ran in the research workspace under /tmp/bt. These are scratch files with no external hosting. Pickled outputs hold every simulated draft and league result, and results.json holds the tables in this paper.

Draft simulator

data.py, sim.py, run.py (draft logic), an.py (analysis)

Parameter search

grid.py, 54 combinations on 2017-2021

Weekly study

inseason.py (season data), league.py (policies), exp1.py (train), exp2.py (test)

Result files

out.pkl, out2.pkl (draft runs, 44,000 drafts each); train.pkl, test.pkl (weekly)

Table source

dump.py writes results.json from the pickles

Counts

Study 1: 88,000 drafts across two runs. Study 2: 300 train leagues and 600 test leagues, 9,600 policy runs.