Modified limited memory BFGS with displacement aggregation and its application to the largest eigenvalue problem

TL;DR

This paper introduces AggMBFGS, a modified limited memory BFGS method with displacement aggregation, which improves convergence and efficiency in large-scale nonconvex optimization and eigenvalue problems.

Contribution

The paper proposes AggMBFGS, a novel BFGS variant with displacement aggregation, offering enhanced convergence properties and computational efficiency for large-scale problems.

Findings

AggMBFGS outperforms M-LBFGS in reducing iterations and function evaluations.

AggMBFGS achieves lower relative errors in eigenvalue computations.

Theoretical proof of superlinear convergence of AggMBFGS.

Abstract

We present a modified limited memory BFGS method with displacement aggregation (AggMBFGS) for solving nonconvex optimization problems. AggMBFGS refines curvature pair updates by removing linearly dependent variable variations, ensuring that the inverse Hessian approximation retains essential curvature properties. As a result, its per iteration complexity and storage requirement is $\mathcal{O}(\tau d)$ where $\tau \leq d$ represents the memory size and $d$ is the problem dimension. We establish the global convergence of both M-LBFGS and AggMBFGS under a backtracking modified Armijo line search (MALS) and prove the local superlinear convergence of AggMBFGS, demonstrating its theoretical advantages over M-LBFGS with the classical Armijo line search~\cite{Shi2016ALM}. Numerical experiments on CUTEst test problems~\cite{gould2015cutest} confirm that AggMBFGS outperforms M-LBFGS in reducing the number of iterations and function evaluations. Additionally, we apply AggMBFGS to compute the largest eigenvalue of high-dimensional real symmetric positive definite matrices, achieving lower relative errors than M-LBFGS~\cite{Shi2016ALM} while maintaining computational efficiency. These results suggest that AggMBFGS is a promising alternative for large-scale nonconvex optimization and eigenvalue computation.