Bias-Variance Decomposition of Mean Squared Error

Scenario In quantitative trading and quantitative research, candidates are frequently evaluated on statistics modeling, specifically regarding Statistical Learning / Bias-Variance.

Your Task
Let $y = f(x) + \epsilon$ with $\mathbb{E}[\epsilon] = 0$ and $\text{Var}(\epsilon) = \sigma^2$. Given a training set $\mathcal{D}$, an estimator $\hat{f}(x; \mathcal{D})$ is trained. Prove the Bias-Variance decomposition of the expected test Mean Squared Error: $\mathbb{E}_{\mathcal{D}, \epsilon}[(y - \hat{f}(x))^2] = \text{Bias}(\hat{f}(x))^2 + \text{Var}(\hat{f}(x)) + \sigma^2$

Constraints

• All variables are defined on a standard probability space $(\Omega, \mathcal{F}, \mathbb{P})$.
• Analytical expressions must be in closed form, and empirical simulations should converge within standard statistical tolerances.