A new secant method for unconstrained optimization

TL;DR

This paper introduces a novel gradient-based secant method for unconstrained optimization, leveraging iterated linear basis changes, which performs comparably or more robustly than existing methods like BFGS and DFP.

Contribution

The paper proposes a new secant method derived from linear basis transformations, offering an alternative to traditional quasi-Newton methods with similar or improved robustness.

Findings

Performs comparably to BFGS and DFP

Equivalent to linear conjugate gradient for quadratic functions

More robust in certain practical scenarios

Abstract

We present a gradient-based algorithm for unconstrained minimization derived from iterated linear change of basis. The new method is equivalent to linear conjugate gradient in the case of a quadratic objective function. In the case of exact line search it is a secant method. In practice, it performs comparably to BFGS and DFP and is sometimes more robust.