Gauss-Markov Theorem and BLUE Property of OLS

Scenario In quantitative trading and quantitative research, candidates are frequently evaluated on statistics modeling, specifically regarding Linear Regression / Gauss-Markov Theorem.

Your Task
Consider the linear regression model $y = X\beta + \epsilon$, with $\mathbb{E}[\epsilon \mid X] = 0$ and $\text{Var}(\epsilon \mid X) = \sigma^2 I_n$. Prove the Gauss-Markov Theorem: the OLS estimator $\hat{\beta} = (X^T X)^{-1} X^T y$ is the Best Linear Unbiased Estimator (BLUE), meaning that for any other linear unbiased estimator $\tilde{\beta}$, $\text{Var}(\tilde{\beta}) - \text{Var}(\hat{\beta})$ is positive semi-definite.

Constraints

• The design matrix $X \in \mathbb{R}^{n \times p}$ has full column rank $p < n$.
• Disturbance terms satisfy standard zero-mean conditional homoskedasticity unless stated otherwise.