All roads lead to Rome: Path-following Augmented Lagrangian Methods via Bregman Proximal Regularization

TL;DR

This paper introduces a novel class of Bregman proximal augmented Lagrangian methods for convex optimization, leveraging second-order oracles and non-Euclidean geometries to achieve quadratic convergence and fast rates.

Contribution

It develops a unified framework combining Bregman proximal point algorithms with augmented Lagrangian methods, extending classical approaches with non-Euclidean geometries and operator-theoretic analysis.

Findings

Quadratic convergence under appropriate step-size selection.

Fast complexity bounds derived via metric subregularity.

Special cases include variants of exponential multiplier and interior-point methods.

Abstract

We study Bregman proximal augmented Lagrangian methods with second-order oracles for convex convex-composite optimization problems. The outer loop is an instance of the Bregman proximal point algorithm with relative errors in the sense of Solodov and Svaiter, applied to the KKT operator associated with the problem. Akin to classical Lagrange-Newton methods, including primal-dual interior point methods the Bregman proximal point algorithm repeatedly solves regularized KKT inclusions by minimizing a smooth Bregman augmented Lagrangian function, obtained after marginalizing out the multiplier variables. Thanks to non-Euclidean geometries the marginal function is generalized self-concordant and therefore within the regime of Newton's method which converges quadratically if the step-size in the outer proximal point loop is chosen carefully. The operator-theoretic viewpoint allows us to employ the framework of metric subregularity to derive fast rates for the outer loop, and eventually state a joint complexity bound. Important special cases of our framework are a proximal variant of the exponential multiplier method due to Tseng and Bertsekas and interior-point proximal augmented Lagrangian schemes closely related to those of Pougkakiotis and Gondzio.