Regularized Barzilai-Borwein method

TL;DR

This paper introduces a regularized Barzilai-Borwein (RBB) stepsize that improves the efficiency and stability of solving challenging optimization problems, especially ill-conditioned ones, by incorporating regularization and adaptive parameter schemes.

Contribution

The paper proposes a novel RBB stepsize that generalizes the original BB stepsize, proves its convergence for convex quadratic problems, and demonstrates its robustness and efficiency through numerical experiments.

Findings

RBB stepsize reduces instability in ill-conditioned problems.

Numerical results show RBB outperforms traditional BB stepsize.

Adaptive parameter scheme enhances RBB's effectiveness.

Abstract

We develop a novel stepsize based on \BB method for solving some challenging optimization problems efficiently, named regularized \BB (RBB) stepsize. We indicate that RBB stepsize is the close solution to a $\ell_{2}^{2}$-regularized least squares problem. When the regularized item vanishes, the RBB stepsize reduces to the original \BB stepsize. RBB stepsize includes a class of valid stepsizes, such as another version of \BB stepsize. The global convergence of the corresponding RBB algorithm is proved in solving convex quadratic optimization problems. One scheme for adaptively generating regularization parameters was proposed, named adaptive two-step parameter. An enhanced RBB stepsize is used for solving quadratic and general optimization problems more efficiently. RBB stepsize could overcome the instability of BB stepsize in many ill-conditioned optimization problems. Moreover, RBB stepsize is more robust than BB stepsize in numerical experiments. Numerical examples show the advantage of using the proposed stepsize to solve some challenging optimization problems vividly.