A family of Barzilai-Borwein steplengths from the viewpoint of scaled total least squares

TL;DR

This paper introduces a new family of Barzilai-Borwein steplengths based on scaled total least squares, enhancing gradient methods for unconstrained optimization with improved performance.

Contribution

It proposes a novel family of BB steplengths derived from scaled total least squares, expanding the theoretical framework and practical options for gradient-based optimization.

Findings

High performance achieved with carefully-selected steplengths

Numerical experiments validate the effectiveness of the new family

Enhanced gradient methods for unconstrained optimization

Abstract

The Barzilai-Borwein (BB) steplengths play great roles in practical gradient methods for solving unconstrained optimization problems. Motivated by the observation that the two well-known BB steplengths correspond to the ordinary and the data least squares, respectively, we present a family of BB steplengths from the viewpoint of scaled total least squares. Numerical experiments demonstrate that a high performance can be received by a carefully-selected BB steplength in the new family.