A Newton-CG Algorithm with Complexity Guarantees for Smooth Unconstrained Optimization

TL;DR

This paper introduces a Newton-CG algorithm for smooth nonconvex optimization that guarantees convergence within a certain complexity, leveraging negative curvature directions and improving upon classical methods.

Contribution

It presents a novel Newton-CG algorithm with explicit complexity guarantees for convergence to approximate optimality in nonconvex problems.

Findings

Matches best known complexity results for second-order methods

Incorporates negative curvature detection for improved convergence

Builds on classical Newton-conjugate gradient procedures

Abstract

We consider minimization of a smooth nonconvex objective function using an iterative algorithm based on Newton's method and the linear conjugate gradient algorithm, with explicit detection and use of negative curvature directions for the Hessian of the objective function. The algorithm tracks Newton-conjugate gradient procedures developed in the 1980s closely, but includes enhancements that allow worst-case complexity results to be proved for convergence to points that satisfy approximate first-order and second-order optimality conditions. The complexity results match the best known results in the literature for second-order methods.