Convergence of iterates and improved rates for accelerated augmented Lagrangian methods for linearly constrained convex optimization

TL;DR

This paper introduces accelerated augmented Lagrangian methods inspired by inertial primal-dual dynamics, achieving improved convergence rates for linearly constrained convex optimization.

Contribution

It proposes a novel class of accelerated methods with Nesterov extrapolation, providing the first known little-o convergence rates for feasibility and objective residuals.

Findings

Proved convergence of primal-dual sequences to saddle points.

Achieved improved $o(1/k^2)$ rates for feasibility and objective residuals.

Established convergence under suitable parameter conditions.

Abstract

We consider a linearly constrained convex optimization problem with a differentiable objective function. Motivated by an inertial primal-dual dynamical system with vanishing damping, we propose a class of accelerated augmented Lagrangian methods with Nesterov extrapolation parameters. The framework contains two variants: an implicit-gradient scheme for convex continuously differentiable objectives and a partially explicit scheme for convex smooth objectives. Under suitable parameter conditions, we prove convergence of the primal-dual sequence to a saddle point, together with accelerated estimates for the augmented Lagrangian gap, the feasibility violation, and the objective residual. In the noncritical parameter regime, these estimates are improved from $O(1/k^2)$ to $o(1/k^2)$. To the best of our knowledge, such little-o rates for both feasibility violation and objective residual have not been previously established for accelerated augmented Lagrangian methods in this setting.