← Models / SOG Model — Performance

SOG Model — Performance

Today's board · Pick history

Reading the Performance Page

Live vs backtest

Live is every pick the model made before puck drop since tracking began. Backtest replays past seasons month by month: the model is re-fitted using only games before each month, then prices that month's Hard Rock closing lines. The page opens on Live once 50 live predictions are graded.

Units, ROI, CLV

Units assume a flat 1-unit bet per pick at Hard Rock's price. ROI = units ÷ settled picks. CLV (live only) is the average move of Hard Rock's de-vigged probability toward the pick between the pick and the close — the most reliable early signal of real edge, because results take thousands of bets to separate skill from luck.

Accuracy

Log loss and Brier score measure how good the probabilities are over every priced line (lower is better), compared with Hard Rock's own de-vigged probability on the same lines. The production probability blends the simulation with the market; the report shows both. Calibration: dots on the diagonal mean that when the model says 60%, the over really hits about 60% of the time.

Backtest report

The model must beat simple baselines (season and last-10 averages), a machine-learning challenger, and two stripped-down versions of itself before it can be trusted, and its edge over the market is shown with a 95% confidence interval. The three pricing questions (star overs, role changes, 1.5 lines) were written down before testing — they are measured, not assumed.

Informational only. Not betting advice.

Live (279 graded) Backtest
All picks
5-9
-2.94u · ROI -21.0%
Best bets
1-1
+0.45u · ROI +22.5%
Leans
4-8
-3.39u · ROI -28.2%
Avg CLV
-1.08
14 with closes
Log loss · model / book
0.6772 / 0.6791
279 lines · lower is better
Cumulative units flat 1u per pick at Hard Rock's price
BestLean
-3.4-2.4-1.5-0.5+0.5Best +0.5uLean -3.4uBest · 2026-10-19 · -1.00u cumulativeBest · 2026-10-20 · -1.00u cumulativeBest · 2026-10-21 · +0.45u cumulativeLean · 2026-10-19 · -1.00u cumulativeLean · 2026-10-20 · -1.39u cumulativeLean · 2026-10-21 · -3.39u cumulative2026-10-192026-10-21
By month
MonthRecordWin %UnitsROICLV
2026-105-935.7%-2.94-21.0%-1.08
By line
LineRecordWin %UnitsROICLV
1.54-544.4%+0.35+3.9%-0.25
2.50-30.0%-3.00-100.0%-1.13
3.51-150.0%-0.29-14.3%-4.72
By side
SideRecordWin %UnitsROICLV
OVER1-0100.0%+1.40+140.0%0.00
UNDER4-930.8%-4.34-33.4%-1.16
By tier
TierRecordWin %UnitsROICLV
best1-150.0%+0.45+22.5%+0.22
lean4-833.3%-3.39-28.2%-1.29
By position
PositionRecordWin %UnitsROICLV
D0-50.0%-5.00-100.0%-0.93
F5-455.6%+2.06+22.9%-1.16
Calibration model P(over) vs how often it hit
00252550507575100100predicted 37.3% · observed 35.9% · n=39predicted 45.6% · observed 41.4% · n=128predicted 54.6% · observed 55.9% · n=102predicted 61.5% · observed 50.0% · n=10
Model PLinesPredictedHitBook
30–40%3937.3%35.9%40.3%
40–50%12845.6%41.4%46.7%
50–60%10254.6%55.9%55.3%
60–70%1061.5%50.0%60.8%
Walk-forward backtest report 2024-10-04 → 2026-10-18 · 41,351 Hard Rock closing lines · run 2026-10-19

Verdict: SHADOW PASS requires beating every baseline, a log-loss edge over Hard Rock whose 95% interval excludes zero, and best-bet ROI whose 95% interval is above zero. Otherwise the model stays in shadow mode.

MethodLog lossBrier
Production probability (sim × market blend)0.681810.24438
Simulation only0.683880.24535
Model without game script0.684030.24542
Model without market inputs0.683860.24534
Season average (Poisson)0.695970.25075
Last 10 (Poisson)0.713750.25855
LightGBM challenger0.684540.24568
Hard Rock (de-vigged)0.682880.24490
Production (blend) − market log loss
-0.00106 95% CI [-0.0015, -0.0006] (negative = better than Hard Rock's close)
Simulation-only − market log loss
+0.00100 95% CI [-0.0001, +0.0022] (negative = model better)
Best-bet ROI
+5.6% n=974, 95% CI [-0.6, +11.5]
Lean ROI
-2.0% n=2096, 95% CI [-6.4, +2.4]
Mean SOG · predicted / actual
2.217 / 2.237
Leftover dispersion
cv² 0.0174 (param 0.0097)
Zone-start coefficient
+0.091 ≈0 means zone starts add nothing

Blending the simulation into the market

Log loss when the probability is a logit blend of the simulation (weight w) and Hard Rock (in-sample over the whole backtest, for reference). Production uses a walk-forward weight fitted only on earlier months (by month: 2024-10 0.3, 2024-11 0.3, 2024-12 0.55, 2025-01 0.55, 2025-02 0.55, 2025-03 0.55, 2025-04 0.45, 2025-05 0.5, 2025-06 0.5, 2025-10 0.5, 2025-11 0.45, 2025-12 0.45, 2026-01 0.45, 2026-02 0.4, 2026-03 0.4, 2026-04 0.4, 2026-05 0.4, 2026-06 0.45, 2026-09 0.45, 2026-10 0.45).

w=0.0w=0.1w=0.2w=0.3w=0.5w=0.7w=1.0
0.682880.682390.682030.681800.681750.682220.68388

By line

LineLinesOver rateBlend LLSim LLBook LLLightGBM LLSeason LL
0.53363.6%0.64480.65020.64630.65380.6818
1.521,32653.7%0.68470.68630.68530.68690.6985
2.516,87345.0%0.67940.68190.68080.68210.6931
3.52,97842.2%0.67580.67900.67870.68210.6941
4.514136.2%0.66600.66310.67240.67480.6975

Pre-registered pricing hypotheses

HypothesisLinesOver hitBook P(over)Gap (95% CI)Blind over ROIBlind under ROI
H1 · 3.5+ lines (star overs)3,11942.0%46.1% -4.1 [-5.7, -2.4] -15.0% [-18.4, -11.6] +0.5% [-2.4, +3.3]
H3 · 1.5 lines21,32653.7%54.3% -0.7 [-1.4, -0.1] -7.7% [-9.0, -6.6] -5.1% [-6.4, -3.7]
H2 · role change (last game's PP/EV role differs from his prior 10)LinesSim − market LL (95% CI)PicksPick ROI (95% CI)
After a role change6,688+0.0030 [-0.0001, +0.0057] 621-1.1% [-9.1, +6.6]
No role change34,663+0.0006 [-0.0006, +0.0017] 2449+0.8% [-3.3, +4.9]