A Fast Newton Method Under Local Lipschitz Smoothness

TL;DR

This paper introduces a fast second-order Newton method that achieves optimal complexity for finding first-order stationary points under local Lipschitz smoothness, without requiring global Hessian continuity.

Contribution

It proposes a novel Newton-type algorithm that operates under local smoothness assumptions, with inexact Hessian computations and an extended version for second-order critical points.

Findings

Achieves optimal complexity for first-order stationary points.

Demonstrates competitiveness through initial numerical experiments.

Extends to find second-order critical points with near-optimal complexity.

Abstract

A new, fast second-order method is proposed that achieves the optimal $\mathcal{O}\left(|\log(\epsilon)|\epsilon^{-3/2}\right)$ complexity to obtain first-order $\epsilon$-stationary points. Crucially, this is deduced without assuming the standard global Lipschitz Hessian continuity condition, but only using an appropriate local smoothness requirement. The algorithm exploits Hessian information to compute a Newton step and a negative curvature step when needed, in an approach similar to that of the AN2C method.Inexact versions of the Newton step and negative curvature are proposed in order to reduce the cost of evaluating second-order information. Details are given of such an iterative implementation using Krylov subspaces. An extended algorithm for finding second-order critical points is also developed and its complexity is again shown to be within a log factor of the optimal one. Initial numerical experiments are discussed for both factorised and Krylov variants, which demonstrate the competitiveness of the proposed algorithm.