Closed-Form Shrinkage and SVD in Ridge Regression

Scenario In quantitative trading and quantitative research, candidates are frequently evaluated on statistics modeling, specifically regarding Regularization / Linear Algebra.

Your Task
In Ridge regression, the objective is to minimize: $J(\beta) = \|y - X\beta\|_2^2 + \lambda \|\beta\|_2^2$ where $\lambda > 0$.
(1) Derive the closed-form solution $\hat{\beta}_{\text{ridge}} = (X^T X + \lambda I_p)^{-1} X^T y$.
(2) Using the Singular Value Decomposition (SVD) of $X = U \Sigma V^T$, express the fitted values $\hat{y}_{\text{ridge}}$ in terms of singular values $\sigma_j$ and explain how $\lambda$ shrinks the effective degrees of freedom.

Constraints

• The design matrix $X \in \mathbb{R}^{n \times p}$ has full column rank $p < n$.
• Disturbance terms satisfy standard zero-mean conditional homoskedasticity unless stated otherwise.