Fast Augmented Lagrangian Method in the convex regime with convergence guarantees for the iterates

TL;DR

This paper introduces a fast inertial Augmented Lagrangian method for convex optimization with linear constraints, providing convergence guarantees for both the objective and the iterates without requiring strong convexity.

Contribution

It develops a new inertial algorithm based on discretizing a second-order primal-dual dynamical system, achieving convergence rates and iterates convergence in the convex setting.

Findings

Achieves O(1/k^2) convergence rate for primal-dual gap, feasibility, and objective value.

Proves convergence of primal-dual iterates to a solution without strong convexity assumptions.

Compatible with continuous-time convergence results from prior work.

Abstract

This work aims to minimize a continuously differentiable convex function with Lipschitz continuous gradient under linear equality constraints. The proposed inertial algorithm results from the discretization of the second-order primal-dual dynamical system with asymptotically vanishing damping term addressed by Bot and Nguyen in [Bot, Nguyen, JDE, 2021], and it is formulated in terms of the Augmented Lagrangian associated with the minimization problem. The general setting we consider for the inertial parameters covers the three classical rules by Nesterov, Chambolle-Dossal and Attouch-Cabot used in the literature to formulate fast gradient methods. For these rules, we obtain in the convex regime convergence rates of order ${\cal O}(1/k^{2})$ for the primal-dual gap, the feasibility measure, and the objective function value. In addition, we prove that the generated sequence of primal-dual iterates converges to a primal-dual solution in a general setting that covers the two latter rules. This is the first result which provides the convergence of the sequence of iterates generated by a fast algorithm for linearly constrained convex optimization problems without additional assumptions such as strong convexity. We also emphasize that all convergence results of this paper are compatible with the ones obtained in [Bot, Nguyen, JDE, 2021] in the continuous setting.