Equipping Barzilai-Borwein method with two dimensional quadratic termination property

TL;DR

This paper introduces a new gradient stepsize for the Barzilai-Borwein method that achieves two-dimensional quadratic termination, improving efficiency in quadratic and unconstrained optimization without requiring line searches or Hessian information.

Contribution

A novel stepsize is developed that depends only on previous BB stepsizes, enabling extension to nonlinear, eigenvalue, and constrained optimization problems.

Findings

Outperforms existing gradient methods in numerical tests.

Effectively extends to eigenvalue and constrained optimization.

Achieves quadratic termination property in gradient methods.

Abstract

A novel gradient stepsize is derived at the motivation of equipping the Barzilai-Borwein (BB) method with two dimensional quadratic termination property. A remarkable feature of the novel stepsize is that its computation only depends on the BB stepsizes in previous iterations and does not require any exact line search or the Hessian, and hence it can easily be extended for nonlinear optimization. By adaptively taking long BB steps and some short steps associated with the new stepsize, we develop an efficient gradient method for quadratic optimization and general unconstrained optimization and extend it to solve extreme eigenvalues problems. The proposed method is further extended for box-constrained optimization and singly linearly box-constrained optimization by incorporating gradient projection techniques. Numerical experiments demonstrate that the proposed method outperforms the most successful gradient methods in the literature.