Ensemble Methods: Combining Multiple Models

## Why Ensembles Work No single model captures all the signal in sports data. Different models have different strengths: - Poisson regression: excellent for goal distribution - Elo ratings: captures long-run quality dynamics - XGBoost: captures non-linear contextual effects - Logistic regression: simple, well-calibrated for basic features When these models disagree, they are providing complementary information. An ensemble that weights their outputs optimally outperforms any single model. ## Simple Ensemble: Averaging The simplest ensemble: average the probability predictions of all models. P_ensemble = (P_model1 + P_model2 + P_model3) / 3 This is surprisingly effective — the averaging reduces the individual models' biases and variances. ## Weighted Ensemble: Stacking Stacking (or blending) learns the optimal weights for each model's contribution: 1. Generate out-of-sample predictions from each base model (using cross-validation) 2. Fit a meta-model (typically logistic regression or linear regression) on these predictions 3. The meta-model's coefficients are the ensemble weights The meta-model learns that model A is more reliable for high-stakes matches and model B is more reliable for lower leagues — and weights accordingly. ## The Diversity Requirement Ensembles only work if the models are genuinely diverse (make different errors). Combining two near-identical models produces minimal improvement. The ideal ensemble combines models with: - Different architectures (regression vs tree-based vs rating system) - Different feature sets - Different training data periods ## Practical Ensemble for a Betting Operation A practical ensemble for a solo bettor: - Base model: Poisson regression (the workhorse) - Supplementary model: Elo ratings (captures quality dynamics) - Ensemble: 70% Poisson + 30% Elo (weights from backtested log-loss minimisation) Test this ensemble against each model individually. If it improves log-loss: use it. If not: the models are too correlated to benefit from ensembling.