Random Forest and Gradient Boosting in Practice

## Gradient Boosting for Match Prediction Gradient boosting builds an ensemble of decision trees sequentially. Each tree corrects the errors of the previous ones. The result is a powerful, flexible model capable of capturing complex patterns. ## Implementation with XGBoost ```python import xgboost as xgb from sklearn.model_selection import TimeSeriesSplit # Prepare features and target X = match_data[feature_columns] y = match_data['home_goals'] # or outcome label # Time-series cross-validation tscv = TimeSeriesSplit(n_splits=5) log_losses = [] for train_idx, test_idx in tscv.split(X): X_train, X_test = X.iloc[train_idx], X.iloc[test_idx] y_train, y_test = y.iloc[train_idx], y.iloc[test_idx] model = xgb.XGBClassifier( n_estimators=200, max_depth=4, learning_rate=0.05, subsample=0.8, eval_metric='mlogloss' ) model.fit(X_train, y_train) probs = model.predict_proba(X_test) log_losses.append(log_loss(y_test, probs)) print(f"Average log-loss: {np.mean(log_losses):.4f}") ``` ## Hyperparameter Tuning Key parameters for sports prediction: - `max_depth`: 3–5 (deeper trees overfit more easily) - `learning_rate`: 0.01–0.1 (lower = slower learning, less overfitting) - `n_estimators`: 100–500 (more = more complex model) - `subsample`: 0.6–0.9 (random subsample of data per tree, reduces overfitting) Use grid search or Bayesian optimisation with time-series cross-validation to find the best combination. ## Feature Importance XGBoost produces feature importance scores — which features contributed most to the model's predictions. Use this to: - Identify the most valuable features for collection and maintenance - Remove low-importance features that add noise - Generate hypotheses about what drives match outcomes