Snapcount

AI-native NFL prediction · Progress Report
Progress Report · July 2026

Predictions you can trust — because we prove them, and publish where we're wrong.

Two things shipped this cycle: a projection engine that clears every accuracy bar we set for it, and a language layer that reads this week's news into those projections — the piece a frozen, cutoff-dated chatbot fundamentally cannot do. Below: what we achieved, and exactly how we verified each claim. (This is the exec/status report. For how the prediction engine works, read How we predict; for the exact math, the methodology white paper.)

The foundation · Prediction engine

Every accuracy bar we set — met.

We hold ourselves to nine pre-declared targets across the stats fantasy players actually care about (passing, rushing, receiving yards; receptions; touchdowns). All nine pass — on data the model has never seen, scored the only honest way for this problem.

9 / 9
pre-declared accuracy + boom/bust gates passing
≈ market
at statistical parity with the market leader (RotoWire) at wide receiver
70%
calibrated range coverage — every projection is a floor–ceiling, not false precision
Double-click for technical folks — how the engine is measured
The metric
Position-pure, touchdown-free weekly ranking (Spearman / Top-N / start-sit), scored on a box-score truth that can't be gamed by predicting a floor. Random k-fold is invalid here (it leaks future weeks); we use walk-forward cross-validation — train on season < S or (season == S and week < W), predict week W.
Current ranking (pooled Spearman) vs the market
PositionOur modelRotoWireGap
RB.700.741−.041
WR.680.690−.010 (parity)
TE.623.637−.014
Guardrails that never move
  • The 2025 season is sealed — never trained, tuned, or selected against.
  • An anti-leakage gate runs on every live feature build; post-game-only columns are block-listed and can never reach a live prediction.
  • Ranges are asymmetric-conformal, width-scaled per player by trailing volatility; pooled coverage lands at ~70% (0.699 / 0.712 / 0.708 for RB/WR/TE).
  • Promotions require paired significance tests (sign / Wilcoxon per cell), not eyeballed averages — and every rejected idea is logged with its numbers.
This cycle's advance · The language layer

Reading this week's news — the thing a frozen chatbot can't.

Every general AI model has a training cutoff. Ask one about Sunday's inactives or a coach's Wednesday practice comment and it's guessing from stale memory. Our edge: a language model on our own hardware turns live beat-writer news into structured signals that feed the projection board, which refreshes daily. Same news the market reacts to — read into a calibrated model, not improvised from headlines.

93.8%
extractor accuracy on our acceptance test — with zero fabricated player links
$0
per-query cost — runs locally, no data leaves our machine
daily
board refresh cadence in-season — the moat a cutoff-dated model can't match
Double-click for technical folks — the extractor stack
What it is
A local mixture-of-experts model (Qwen3.5-35B class, 4-bit, served via Apple's MLX on an M2 Max) behind a pluggable OpenAI-compatible interface — any backend (a future in-house model, a partner's Hermes endpoint, or a hosted API) drops into the same slot and is judged by the same test.
The task
Closed-vocabulary event extraction: 6 event types (lead-back named, committee declared, first-team reps, snap-plan hint, role change, availability update), each with a direction and a source-confidence tier — or SKIP. The output is deterministic JSON the projection bridge consumes with bounded, clamped adjustments (a single news item can never swing a projection more than a fixed cap).
The bar
A 50-item golden set (held out, never trained on): ≥90% exact match on all five fields (event · player · tier · direction · team) and zero fabricated player links. Current result: 30/32 positives exact, 18/18 correct skips, 0 fabrications, 0 false positives.
Why you can believe the numbers

We audit ourselves adversarially — and let the honest number win.

The hardest part of a prediction company isn't getting a good number; it's knowing whether to trust it. Our discipline: an independent reviewer tries to break every claim before we ship it. Here's exactly what that looked like this cycle.

⚑ What the self-audit caught — and how we responded
An independent adversarial review found our own extractor test was too lenient: it wasn't scoring the two fields that actually change a projection (which way the news cuts, and which team). We made the test stricter — and our score honestly dropped. Rather than hide that, we rebuilt the model against the tougher bar until it earned the score back for real.
lenient test93.8%
honest test (fixed)65.6%
rebuilt legitimately93.8%

Same headline number at the start and end — but the second one is real. That gap is the whole point: most teams would have shipped the first 93.8% and never known.

Test the field that matters

The gate now scores news direction — a model that flips "more work" to "less work" is marked wrong, because that flips the projection the wrong way.

Training data proven correct

Every synthetic training example is round-tripped back through the model under the real rules and kept only if it reproduces the intended answer — no "correct by assumption."

Publish the misses

We replay a full past season and show, week by week, where the board was right and where it wasn't. Nobody launches a projection product by publishing its errors. We do.

Automated on every change

Continuous integration re-runs the product tests on every code change — and already caught real deploy-blocking bugs before they could reach anyone.

Double-click for technical folks — the audit findings in full
An independent reviewer agent (no stake in the design, instructed to refute) returned an UNSOUND verdict on the original gate, with three critical findings — all confirmed and fixed:
  • C1 — gate blind to direction + team. Exact-match scored only event / player / tier. A snap-up ↔ snap-down flip — which moves the board the wrong way — counted as a pass. Fix: score all five fields. The teacher model fell 93.8% → 65.6% on the honest gate; after a prompt rebuild + repairing 8 exam items whose "correct" team wasn't actually inferable from the text, it recovered to a legitimate 93.8% (30/32, 0 FP, 0 FN).
  • C2 — training corpus scope drift. 53% of generated examples were out-of-scope for the running-back-focused extractor. Fix: regenerate in-scope only; corpus, prompt, and exam now agree.
  • C3 — "correct by construction" over-claimed. The synthesizer sometimes wrote text that didn't match the intended label. Fix: a round-trip validator re-extracts every generated item under the production rules and discards any mismatch; raw vocabulary tokens leaking into "news" text are hard-rejected.
Open items we're honest about (pre-registered before any production model swap): grow the golden set to ≥150 in-scope positives and gate on a Wilson lower bound ≥ 0.90 rather than a point estimate (a 50-item test can't distinguish 90% from 80% with confidence); and add a held-out set of real beat-writer text, since today's exam is still synthetic.
What it means

The moat: fresh, honest, and owned.

For execs & investors

A defensible edge a general-purpose AI can't copy: a calibrated model that refreshes on live news daily, honest calibration as the trust brand, and models we own outright at near-zero marginal cost.

For football analysts

Projections that show their work — a real range, a confidence read, a boom/bust probability, and a published record of hits and misses. No black box, no false precision.

For engineers

Walk-forward CV, a mandatory anti-leakage gate, adversarial self-audit, held-out acceptance gauntlets, and CI on every change. Every claim on this page traces to a re-runnable test.

9/9 accuracy gates 0 fabricated links 70% calibrated coverage 2025 season sealed walk-forward only adversarially audited local · $0 / query