A Parameterized Barzilai-Borwein Method via Interpolated Least Squares

TL;DR

This paper introduces a new parameterized class of Barzilai-Borwein (BB) step sizes using interpolated least squares, enhancing the original method's flexibility while maintaining quasi-Newton properties, and proves its convergence for convex quadratic problems.

Contribution

It proposes a novel class of BB step sizes via interpolated least squares, extending the classical BB method with adaptive parameters and establishing convergence for convex quadratic problems.

Findings

The new method retains quasi-Newton properties.

It achieves R-linear convergence for convex quadratic problems.

Numerical experiments validate the effectiveness of the proposed approach.

Abstract

The Barzilai-Borwein (BB) method is an effective gradient descent algorithm for solving unconstrained optimization problems. Based on the observation of two classical BB step sizes, by constructing an interpolated least squares model, we propose a novel class of BB step sizes, each of which still retains the quasi-Newton property, with the original two BB step sizes being their two extreme cases. We present the mathematical principle underlying the adaptive alternating BB (ABB) method. Based on this principle, we develop a class of effective adaptive interpolation parameters. For strictly convex quadratic optimization problems, we establish the R-linear convergence of this new gradient descent method by investigating the evolution pattern of the ratio of the absolute values of the gradient components. Numerical experiments are conducted to illustrate our findings.