Log-Loss: The Proper Scoring Rule for Probabilistic Models

## Beyond Accuracy: Why We Need Proper Scoring Rules Accuracy (percentage of correct predictions) is a poor evaluation metric for probabilistic models. A model that always predicts "home team wins" achieves 45% accuracy — not because it is good, but because home teams win approximately 45% of matches. Proper scoring rules evaluate the full probability distribution, rewarding confident correct predictions and penalising confident wrong predictions. ## Log-Loss (Cross-Entropy) Log-loss is the standard proper scoring rule for probabilistic predictions: Log-Loss = −(1/n) × Σ [y_i × log(p_i) + (1−y_i) × log(1−p_i)] Where: - y_i = actual outcome (1 or 0) - p_i = predicted probability **Example:** You predict 70% home win. Home team wins. Contribution = −log(0.70) = 0.357 **Contrast:** You predict 70% home win. Away team wins. Contribution = −log(1−0.70) = −log(0.30) = 1.204 Confident wrong predictions are heavily penalised. This incentivises well-calibrated probabilities. ## Interpreting Log-Loss Lower log-loss = better model. The naive baseline (always predicting base rates, e.g. 45%/25%/30% for home/draw/away) gives a reference log-loss. Your model's log-loss should be significantly below the naive baseline to justify its use. ## Brier Score The Brier score is an alternative proper scoring rule: Brier = (1/n) × Σ (p_i − y_i)² Lower Brier score = better calibration. Also rewards confident correct predictions, but less steeply than log-loss. Both metrics are appropriate for betting model evaluation. Log-loss is more commonly used in the machine learning community; Brier score in the academic sports statistics literature.