A Damped Newton Method Achieves Global $O\left(\frac{1}{k^2}\right)$ and Local Quadratic Convergence Rate

TL;DR

This paper introduces a simple, explicit stepsize schedule for the Newton method that guarantees both fast global convergence at an $O(1/k^2)$ rate and local quadratic convergence, under affine-invariance assumptions.

Contribution

The paper presents the first explicit stepsize schedule for Newton's method that achieves optimal global and local convergence rates with a simple formula.

Findings

Achieves $O(1/k^2)$ global convergence rate.

Proves local quadratic convergence.

Demonstrates competitive performance against existing methods.

Abstract

In this paper, we present the first stepsize schedule for Newton method resulting in fast global and local convergence guarantees. In particular, a) we prove an $O\left( \frac 1 {k^2} \right)$ global rate, which matches the state-of-the-art global rate of cubically regularized Newton method of Polyak and Nesterov (2006) and of regularized Newton method of Mishchenko (2021) and Doikov and Nesterov (2021), b) we prove a local quadratic rate, which matches the best-known local rate of second-order methods, and c) our stepsize formula is simple, explicit, and does not require solving any subproblem. Our convergence proofs hold under affine-invariance assumptions closely related to the notion of self-concordance. Finally, our method has competitive performance when compared to existing baselines, which share the same fast global convergence guarantees.