MLB HR PyTorch Experiment
A live notebook-style post separating calibrated HR probability from ranking signals for shortlist discovery.
mlb-hr-v1 / random_forest
8,185 held-out batter games
mlb-hr-torch-v1_heuristic_blend / pytorch_heuristic_blend log loss
Statcast blend AUC
The current random-forest HR model is calibrated and leakage-aware. A validation-chosen PyTorch + heuristic blend improves Brier and log loss, a handedness-enriched blend improves top-10 ranking, and a Statcast-enriched blend has the best AUC/top-25 ranking so far.
The current model is useful because it is leakage-aware and calibrated against a fixed held-out slice. The raw PyTorch model is comparable, but the stronger result is the validation-chosen PyTorch + heuristic blend, which improves probability metrics while preserving the same pregame feature rules. Handedness and Statcast enrichments improve ranking in different parts of the board, so the next production decision should keep probability quality and candidate ranking separate.
Statcast currently improves broad ranking, handedness improves top-10, v1 remains the calibrated production baseline.
| Metric | Random Forest | PyTorch | Blend | Blend Delta |
|---|---|---|---|---|
| Brier | 0.1017 | 0.1019 | 0.1015 | -0.0002 better |
| Log loss | 0.3548 | 0.3550 | 0.3535 | -0.0013 better |
| AUC | 0.6048 | 0.5966 | 0.6057 | +0.0009 better |
| Top 10 hit rate | 18.8% | 16.6% | 17.8% | -0.0094 worse |
| Top 25 hit rate | 18.8% | 17.3% | 18.3% | -0.0050 worse |
Lower Brier/log loss and higher AUC/top-K hit rate are better.
Blend weights: 84.0% PyTorch / 16.0% heuristic. Blend weights are selected on the inner validation split, then scored once on the held-out test split.
| Rows | Brier | Log loss | AUC | Top 10 | Top 25 |
|---|---|---|---|---|---|
| 110 | 0.1642 | 0.5158 | 0.5594 | 10.0% | 20.0% |
Evaluated dates: 2026-06-16. Missing outcome rows: 10. This is live-board feedback for the currently served model, not the held-out training split.
| Source | Brier | Log loss | AUC | Top 10 | Top 25 |
|---|---|---|---|---|---|
Handedness mlb-hr-torch-handed-v1_heuristic_blend | 0.1016 | 0.3538 | 0.5998 | 21.3% | 18.6% |
Statcast mlb-hr-torch-statcast-v1_heuristic_blend | 0.1016 | 0.3536 | 0.6081 | 18.8% | 19.9% |
Handedness enrichment adds batter/pitcher platoon context from MLB Stats API player metadata. Statcast enrichment adds prior pitch-level contact quality and pitch-mix aggregates from Baseball Savant.
Pitch-level pitch type, velocity, launch angle, exit velocity, batted-ball events, and outcomes.
Python access to Baseball Savant, FanGraphs, and Baseball Reference data for reproducible feature backfills.
Current repo source for schedules, probable pitchers, lineups, batter outcomes, and starter history.