Complexity of the Regularized Newton Method

TL;DR

This paper introduces the damped regularized Newton's method (DRNM), which guarantees global convergence for strictly convex functions and achieves quadratic convergence locally, providing estimates for the steps needed to approximate the minimizer.

Contribution

The paper proposes DRNM, a novel variant of Newton's method that ensures global convergence and quadratic local convergence for strictly convex functions.

Findings

DRNM converges globally for any strictly convex function with a minimizer in R^n.

Locally, DRNM exhibits quadratic convergence rate.

The paper provides estimates on the number of steps needed for ε-approximate solutions.

Abstract

Newton's method for finding an unconstrained minimizer for strictly convex functions, generally speaking, does not converge from any starting point. We introduce and study the damped regularized Newton's method (DRNM). It converges globally for any strictly convex function, which has a minimizer in $R^n$. Locally DRNM converges with a quadratic rate. We characterize the neighborhood of the minimizer, where the quadratic rate occurs. Based on it we estimate the number of DRNM's steps required for finding an $\varepsilon$- approximation for the minimizer.