Logistic Regression MLE via Newton-Raphson Iterative Reweighted Least Squares

Scenario In quantitative trading and quantitative research, candidates are frequently evaluated on statistics modeling, specifically regarding Numerical Optimization / Generalized Linear Models.

Your Task
In binary logistic regression, $y_i \in \{0, 1\}$ with $p_i = \sigma(x_i^T \beta) = \frac{1}{1 + e^{-x_i^T \beta}}$.
(1) Derive the gradient vector $g(\beta)$ and Hessian matrix $H(\beta)$ of the log-likelihood.
(2) Formulate the Newton-Raphson update step and prove that it can be rewritten as Iteratively Reweighted Least Squares (IRLS): $\beta^{(t+1)} = (X^T W_t X)^{-1} X^T W_t z_t$ where $W_t$ is a diagonal weight matrix and $z_t$ is an adjusted response vector.

Constraints

• The design matrix $X \in \mathbb{R}^{n \times p}$ has full column rank $p < n$.
• Disturbance terms satisfy standard zero-mean conditional homoskedasticity unless stated otherwise.