Principal Component Analysis and Covariance Eigendecomposition

Scenario In quantitative trading and quantitative research, candidates are frequently evaluated on statistics modeling, specifically regarding Dimensionality Reduction / Linear Algebra.

Your Task
Let $x \in \mathbb{R}^p$ be a centered random vector with covariance matrix $\Sigma = \mathbb{E}[x x^T]$. We seek a unit vector $w_1 \in \mathbb{R}^p$ ($\|w_1\|_2 = 1$) that maximizes the variance of the projected scalar $z_1 = w_1^T x$. Prove using Lagrange multipliers that $w_1$ is the eigenvector of $\Sigma$ corresponding to its largest eigenvalue $\lambda_1$, and that $\text{Var}(z_1) = \lambda_1$.

Constraints

• All variables are defined on a standard probability space $(\Omega, \mathcal{F}, \mathbb{P})$.
• Analytical expressions must be in closed form, and empirical simulations should converge within standard statistical tolerances.