Newton methods beyond Hessian Lipschitz continuity: A nonlinear preconditioning approach

TL;DR

This paper introduces a nonlinear preconditioning approach for Newton methods that relaxes Hessian Lipschitz continuity assumptions, enabling superlinear convergence under broader smoothness conditions.

Contribution

It extends Newton methods with nonlinear preconditioning to second-order problems, providing convergence guarantees under generalized smoothness assumptions.

Findings

Achieves local superlinear and quadratic convergence guarantees.

Develops a globalization strategy for nonregularized methods.

Attains an $O( ext{epsilon}^{-3/2})$ iteration complexity with a regularized variant.

Abstract

Newton-type methods are typically analyzed under Lipschitz continuity of the Hessian, an assumption that can fail for objectives with higher-order or polynomial growth. We introduce a class of nonlinearly preconditioned Newton methods that apply Newton's root-finding scheme to a transformed optimality mapping, thereby extending recent nonlinear preconditioning ideas from first-order methods to the second-order setting. The resulting methods are naturally analyzed under Lipschitz continuity of a preconditioned Hessian, a condition that significantly relaxes the classical Hessian Lipschitz continuity assumption. Under this generalized smoothness model, we establish local superlinear and quadratic convergence guarantees, and develop a globalization strategy for the nonregularized method despite the fact that the preconditioned Newton direction need not be a descent direction. We further propose a regularized variant for isotropic preconditioners, and show that it attains an $O(\varepsilon^{-3/2})$ iteration complexity. An adaptive version removes the need to know the smoothness constant and allows inexact subproblem solutions while preserving the same complexity order.