On exact linesearch quasi-Newton methods for minimizing a quadratic function

TL;DR

This paper analyzes exact linesearch quasi-Newton methods for quadratic minimization, characterizing conditions under which these methods produce search directions parallel to conjugate gradients, including the structure of update matrices.

Contribution

It provides a complete characterization of rank-one update matrices that produce conjugate gradient-like directions while preserving positive definiteness.

Findings

Identifies necessary and sufficient conditions for quasi-Newton methods to mimic conjugate gradients.

Characterizes symmetric rank-one updates that maintain positive definiteness.

Extends analysis to preconditioned conjugate gradient directions.

Abstract

This paper concerns exact linesearch quasi-Newton methods for minimizing a quadratic function whose Hessian is positive definite. We show that by interpreting the method of conjugate gradients as a particular exact linesearch quasi-Newton method, necessary and sufficient conditions can be given for an exact linesearch quasi-Newton method to generate a search direction which is parallel to that of the method of conjugate gradients. We also analyze update matrices and give a complete description of the rank-one update matrices that give search direction parallel to those of the method of conjugate gradients. In particular, we characterize the family of such symmetric rank-one update matrices that preserve positive definiteness of the quasi-Newton matrix. This is in contrast to the classical symmetric-rank-one update where there is no freedom in choosing the matrix, and positive definiteness cannot be preserved. The analysis is extended to search directions that are parallel to those of the preconditioned method of conjugate gradients in a straightforward manner.