Snapcount — Model Health Archive

Per-position prediction accuracy on the 2024 wks 9-22 starter-only cohort

As of • Validation window: 2024 wks 9-22 • Holdout: 2025 (sealed)

Archive — May 2026 snapshot. This page is from the earlier MAE era, when we graded raw point accuracy. The current scoreboard is weekly ranking quality plus calibrated ranges — see the 2025 holdout proof, the 2024 replay, and the methodology page. Numbers below are kept unchanged for the record.

Headline numbers

MAE (mean absolute error) = average miss in yards. Lower is better. sMAPE = error as a % of truth. We grade on the snap-share-filtered cohort (starters and primary backups only) — that's the universe we actually predict for in production.

Yards targets — all 4 positions

Target	n	MAE (XGB)	sMAPE	MAE (shrunk)	Model

TD targets — calibration framework (M6.1)

TDs aren't predictable as point counts (sMAPE blows up because truth is often 0). We reframed TDs as a probability distribution over discrete classes. Lower Brier = sharper + better-calibrated. ECE = expected calibration error (0 = perfect).

Target	n	Classes	Brier	vs uniform	ECE	MAE (E[y])

Year-over-year trend (yards, 2018-2024)

Per-season overall MAE from the walk-forward backtest. Lower bars / lower line = model getting better.

Top 10 players by career production — per-position MAE

Roadmap

What's NOT in here

- The 2025 season was the sealed holdout at the time of this snapshot. It was later opened exactly once — a frozen-model replay published after the season on the 2025 proof page.
- "Backup" players (snap_share below the position floor) are excluded from both training and grading. The unfiltered MAE 51 we shipped in M2-M5 is a misleading number — see the story.
- Live data (closing Vegas line, in-week injury severity, defense-vs-position breakdown) was slated for the serving layer and is not reflected in these archived numbers.