Hypothesis Testing for Bettors

## Formal Statistical Testing Hypothesis testing provides a formal framework for deciding whether observed results are likely due to genuine edge or random chance. ## The Null Hypothesis Setup **H₀ (null hypothesis):** My strategy has no edge (ROI = 0%). **H₁ (alternative hypothesis):** My strategy has positive edge (ROI > 0%). We assume H₀ is true and calculate the probability of seeing results at least as extreme as we observed. If this probability (the p-value) is below our significance threshold (typically 5%), we reject H₀. ## The t-Test for Betting For n bets with mean return μ and standard deviation σ: t = (μ − 0) / (σ / √n) = μ√n / σ For n = 500, μ = 0.03 (3% ROI), σ = 1.0: t = 0.03 × √500 / 1.0 = 0.03 × 22.36 = 0.671 With 499 degrees of freedom, t = 0.671 corresponds to p ≈ 0.25. We cannot reject H₀ (p > 0.05). At n = 1,000: t = 0.03 × √1,000 / 1.0 = 0.949. p ≈ 0.17. Still not significant. At n = 5,000: t = 0.03 × √5,000 / 1.0 = 2.12. p ≈ 0.02. Now we can reject H₀ at 5% significance. ## The Implication A genuine 3% edge requires approximately 5,000 bets to reach statistical significance at p < 0.05. This is far more bets than most people place before making strategy decisions. The practical response: use CLV (shorter-sample validation) alongside results (longer-sample validation). Do not claim statistically significant ROI before the sample supports it. ## Type I and Type II Errors Type I error: concluding you have edge when you do not (false positive). Type II error: concluding you do not have edge when you do (false negative). Setting a strict significance threshold (p < 0.01 instead of 0.05) reduces Type I errors but increases Type II errors. Choose the threshold based on the cost of each error type.