Model Iteration: From V1 to a Production Model

## Start Simple, Improve Deliberately Every production model started as a simple prototype. The iteration process — evaluate, identify limitations, hypothesize improvement, implement, re-evaluate — is the mechanism of model improvement. ## Version 1: The Baseline A Version 1 football model might include only: - Home team xG attack rate (5-match rolling) - Away team xG defence rate (5-match rolling) - Home advantage (fixed coefficient) Evaluate V1: calculate log-loss on held-out matches. Compare to the naive baseline (always predict 45% home, 25% draw, 30% away). If V1 beats the naive baseline: it is adding information. Track CLV on all bets placed with V1. ## Version 2: Adding Features Identify where V1 is most wrong. Analyse by: - Match type (cup vs league — does V1 misjudge cups?) - Rest days (does V1 mishandle fixture congestion?) - Season position (early season vs mid-season accuracy?) Add the feature that addresses the most common systematic error. Re-evaluate on held-out data. ## Version N: The Production Model After 5–10 iterations, each adding one tested feature: - Weekly automated data update pipeline - Automated prediction generation for all qualifying matches - Automated CLV comparison against live market prices - Alert system for matches exceeding CLV threshold The production model is not necessarily more complex than V1 — it is better calibrated and proven on out-of-sample data. ## The Danger of Over-Iteration Each added feature risks overfitting: the model learns noise from the training data rather than genuine signal. After each addition, test on held-out data. If held-out performance does not improve: discard the feature regardless of training performance.