Limited-memory BFGS Systems with Diagonal Updates
Jennifer B. Erway, Roummel F. Marcia

TL;DR
This paper presents an efficient recursive method to solve systems involving limited-memory BFGS matrices with diagonal updates, crucial for large-scale optimization algorithms like trust-region and augmented Lagrangian methods.
Contribution
It introduces a simple condition under which systems with BFGS matrices plus a diagonal can be solved efficiently using a recursion requiring only vector inner products.
Findings
The recursion formula reduces computational complexity to M^2 n.
The method applies to large-scale optimization problems.
It enables efficient solutions in trust-region and augmented Lagrangian methods.
Abstract
In this paper, we investigate a formula to solve systems of the form (B + {\sigma}I)x = y, where B is a limited-memory BFGS quasi-Newton matrix and {\sigma} is a positive constant. These types of systems arise naturally in large-scale optimization such as trust-region methods as well as doubly-augmented Lagrangian methods. We show that provided a simple condition holds on B_0 and \sigma, the system (B + \sigma I)x = y can be solved via a recursion formula that requies only vector inner products. This formula has complexity M^2n, where M is the number of L-BFGS updates and n >> M is the dimension of x.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
