A Bundle-based Augmented Lagrangian Framework: Algorithm, Convergence, and Primal-dual Principles

TL;DR

This paper introduces a novel single-loop bundle-based augmented Lagrangian method for constrained convex optimization, achieving sub-linear and linear convergence under mild and regularity conditions, including for semidefinite programs.

Contribution

It develops a new bundle-based augmented Lagrangian framework with a single-loop process, improving computational efficiency and convergence analysis for convex constrained problems.

Findings

Establishes sub-linear convergence for primal and dual variables.

Achieves linear convergence under quadratic growth conditions.

Applicable to conic optimization problems like semidefinite programming.

Abstract

We propose a new bundle-based augmented Lagrangian framework for solving constrained convex problems. Unlike the classical (inexact) augmented Lagrangian method (ALM) that has a nested double-loop structure, our framework features a $\textit{single-loop}$ process. Motivated by the proximal bundle method (PBM), we use a $\textit{bundle}$ of past iterates to approximate the subproblem in ALM to get a computationally efficient update at each iteration. We establish sub-linear convergences for primal feasibility, primal cost values, and dual iterates under mild assumptions. With further regularity conditions, such as quadratic growth, our algorithm enjoys $\textit{linear}$ convergences. Importantly, this linear convergence can happen for a class of conic optimization problems, including semidefinite programs. Our proof techniques leverage deep connections with inexact ALM and primal-dual principles with PBM.