Greenfield from PRD intent. Walk-forward verified. 2025 quarantined. Built by four AI agents and one human in twenty-four hours of working time.
A prediction is not a single number from a single model. It's the output of four cooperating systems running on 484,254 plays of NFL data, gated by anti-leakage tests, audited by an independent LLM, and bound to a freshness contract before it reaches production.
Snapcount fixes all five at the data layer, not the model layer.
XGBoost on a walk-forward harness. Per-player projections for 8 fantasy targets. Holdout 2025 quarantined.
Closed-form formulas for live signals. Injury decay, weather penalties, snap-share trajectories. Adam-owned.
API + correlation matrix. Underwrites parlays, lineups, prop combos. TB12-owned.
Domain LLM. /explain, /reason, /simulate. Trained on SnapScore SHAP traces.
SnapScore learns from history. SnapForm adjusts for the present. SnapPro turns predictions into decisions. SnapCall explains why.
Pull 27 sources for 2016 → current season. Auto-derives season range from today.
Schema drift, coverage invariants, in-season currency. Hard-fails on regression.
Predicate filtering carves validation (2024 wks 9+) and holdout (2025).
Id-keys + lagged player-game features. M3 adds Layers 2/3/4.
Predict (S, W) from (season < S OR week < W). 17 boundary tests.
Same harness, plug-in models. Target-aware loss.
Every stage idempotent. Every stage refuses to start if upstream stage failed.
| Kind | Defining trait | Filter rule |
|---|---|---|
| A · Season + Week | season + week, no game_id | season == X AND (week ≥ min_week OR week IS NULL) |
| B · Season only | season, no week (annual snapshot) | season == X |
| C · Game-keyed | nflverse_game_id is authoritative | nflverse_game_id IN slice.game_ids |
| D · Timestamp-keyed | dt ISO8601 (2025+ depth charts) | dt ∈ [Aug 1 of X, Mar 1 of X+1) |
| Z · All-time | no season key (reference tables) | read whole; not slice-able |
The fix that took 5 codex rounds: the original code built a (season, week) pair set from PBP and inner-joined every Kind A source. PBP 2025 stops at week 22, but NGS has week 0 (preseason) + week 23 (post-SB). Those rows escaped the filter and risked leaking. Predicate-based filtering catches them.
Drives the AutoResearch loop's keep-or-revert decision. 14 walk-forward folds across wks 9-22. Per fold the model trains on everything strictly earlier.
data/validation_2024_w9plus/ · 162 games · 28,198 PBP rows
The sacred ship-gate. Belt + suspenders: run_baseline.py drops 2025 BEFORE the harness; walk_forward_splits filters again; holdout_seasons enforced by both split flavors.
data/holdout_2025/ · 285 games · 48,771 PBP rows · 517,851 depth-chart events
Every row in the train fold satisfies (season < S) OR (season == S AND week < W). Imputer + scaler fit per fold; no global statistics leak across folds. 17 boundary tests pin this contract via synthetic frames.
@realfrankbrank 2025-09-02: "we will backfill each year at the end of the regular season." 2025 lands ~Feb 2026 post-Super-Bowl.
| Position × Target | n | M1 Ridge (lagged) | M3 XGB (+EPA/NGS) | M4 XGB | M1→M4 Δ |
|---|---|---|---|---|---|
| QB × passing_yards | 395 | 67.05 | 51.75 | 51.35 | −23.4% |
| QB × passing_tds | 395 | 0.86 | 0.78 | 0.78 | −9.3% |
| RB × rushing_yards | 906 | 19.79 | 15.03 | 14.99 | −24.3% |
| RB × rushing_tds | 906 | 0.34 | 0.31 | 0.31 | −9.7% |
| WR × receiving_yards | 1,457 | 20.92 | 17.13 | 17.10 | −18.3% |
| WR × receiving_tds | 1,457 | 0.32 | 0.31 | 0.31 | −5.0% |
| TE × receiving_yards | 734 | 15.39 | 12.39 | 12.49 | −18.8% |
| TE × receiving_tds | 734 | 0.27 | 0.26 | 0.25 | −6.7% |
14 walk-forward folds across 2024 wks 9-22. XGB target-aware: reg:squarederror for yards, count:poisson for TDs. M4 = M3 + Layer 5 player attrs.
Both models were squeezing the same juice. With 19 features (lagged versions of six source columns), there was almost no non-linear interaction for trees to exploit. The bottleneck wasn't model class. It was feature surface.— Internal post-M2 review
M3 adds opponent quality (defensive EPA allowed), Vegas implied team totals, NGS weekly aggregates (separation, RYOE, CPOE, avg_time_to_throw), and Adam's live injury feed. Then XGB pulls ahead substantially.
participation.parquet schema.m0-complete · m1-codex-pass · m2-xgboost · m3-codex-pass · m4-features. Reproducible from any tag.Pre-kickoff inputs (current injuries, depth charts, Vegas lines, weather forecast, inactives) are moving targets. Every prediction binds to a single (target_game_id, as_of_ts) tuple.
| Source | Provider | max_staleness |
|---|---|---|
| injuries | Adam's ESPN/PFR scrape | 4h |
| depth_charts | nflverse depth_charts_timestamped | 24h |
| vegas_lines | Sportsbook API | 1h |
| weather_forecast | NWS / Tomorrow.io | 6h |
| inactives | NFL inactives feed (T-90min) | 1.5h |
| pbp_lagged / ngs_lagged | nflverse | 48h |
A snapshot is valid iff every source's actual_staleness ≤ max_staleness. Otherwise POST /api/snapscore/predict returns 503.
CRITICAL: pair-set leak (NGS wk 0/23 + officials wk 23). HIGH: stale manifest, missing schema validator. All closed.
CRITICAL: leakage block-list excluded play_id + nflverse_game_id. HIGH: label-based selection bias. Closed.
Verified: target dispatch, fold isolation, Poisson non-negativity, no silent train-mean. PASS gate.
CRITICAL: NGS wk=0 same-season aggregate leak (caught a 24% MAE inflation). All closed. Layer tests added.
The modeling lane doesn't self-approve. Every milestone gets reviewed by a different LLM (OpenAI codex) with adversarial intent. Every CRITICAL/HIGH finding closes before tag. Process is the moat.
Three milestones. Five codex review rounds. A working harness that recorded the first projection_error on all 8 player-level targets. Independent audits at every tag.
Multi-agent orchestration at full velocity, with no broken windows.
M3 unlocks the feature surface. M_serve formalizes the live-snapshot contract. M_inf adds spatiotemporal once Big Data Bowl lands. Production prediction is the back half of 2026.
snapcount.ai · multi-agent NFL prediction · greenfield 2026 · m4-features · 81/81 tests · 9 codex rounds