Improving Quasi-Newton Methods via Image and Projection Operators

TL;DR

This paper introduces a unified framework using image and projection operators to enhance quasi-Newton methods, achieving quadratic termination and improved performance across several popular algorithms without the need for exact line searches.

Contribution

The paper develops a novel operator-based framework that guarantees quadratic termination for multiple quasi-Newton methods and improves their matrix approximation properties.

Findings

Significant performance improvements in DFP, BFGS, PSB, L-BFGS, and BGM methods.

Framework ensures quadratic termination without exact line searches.

Derived operators enhance matrix approximation quality.

Abstract

Designing efficient quasi-Newton methods is an important problem in nonlinear optimization and the solution of systems of nonlinear equations. From the perspective of the matrix approximation process, this paper presents a unified framework for establishing the quadratic termination property that covers the Broyden family, the generalized PSB family, and good Broyden method. Based on this framework, we employ operators to map the correction direction $s_k$ in the quasi-Newton equation to a specific subspace, which ensures quadratic termination for these three classes of methods without relying on exact line searches. We derive the corresponding image and projection operators, analyze their improved properties in matrix approximation, and design practical algorithms accordingly. Preliminary numerical results show that the proposed operator-based methods yield significant improvements in the performance of the Davidon-Fletcher-Powell (DFP), Broyden-Fletcher-Goldfarb-Shanno (BFGS), Powell-Symmetric-Broyden (PSB), limited-memory BFGS (L-BFGS) and Broyden's ``good'' methods (BGM).