Exact linesearch limited-memory quasi-Newton methods for minimizing a quadratic function

TL;DR

This paper introduces a class of limited-memory quasi-Newton methods with exact line search for quadratic functions, achieving finite termination similar to conjugate gradient methods and providing a new framework for constructing these algorithms.

Contribution

It presents a novel class of limited-memory quasi-Newton methods that mimic conjugate gradient directions and offers a reduced-Hessian framework for their construction.

Findings

Methods show finite termination on quadratic problems in exact arithmetic.

Numerical simulations demonstrate competitive performance on linear systems.

Compact Hessian representations facilitate efficient implementation.

Abstract

The main focus in this paper is exact linesearch methods for minimizing a quadratic function whose Hessian is positive definite. We give a class of limited-memory quasi-Newton Hessian approximations which generate search directions parallel to those of the method of preconditioned conjugate gradients, and hence give finite termination on quadratic optimization problems in exact arithmetic. With the framework of reduced-Hessians this class provides a dynamical framework for the construction of limited-memory quasi-Newton methods. We give an indication of the performance of the methods within this framework by showing numerical simulations on sequences of related systems of linear equations, which originate from the CUTEst test collection. In addition, we give a compact representation of the Hessian approximations in the full Broyden class for the general unconstrained optimization problem. This representation consists of explicit matrices and gradients only as vector components.