Lagrangian-based methods in convex optimization: prediction-correction frameworks with ergodic convergence rates

TL;DR

This paper introduces a generalized prediction-correction framework for convex optimization with equality constraints, achieving $O(1/K^2)$ ergodic convergence rates for various Lagrangian-based methods, including ADMM variants.

Contribution

It develops a unified prediction-correction framework that establishes accelerated convergence rates for multiple Lagrangian-based algorithms under convex and strongly convex assumptions.

Findings

Achieves $O(1/K^2)$ ergodic convergence rates for classical and new Lagrangian methods.

Extends convergence results to ADMM with larger step sizes and multi-block ADMM.

Provides theoretical guarantees for accelerated convex optimization algorithms.

Abstract

We study the convergence rates of the classical Lagrangian-based methods and their variants for solving convex optimization problems with equality constraints. We present a generalized prediction-correction framework to establish $O(1/K^2)$ ergodic convergence rates. Under the strongly convex assumption, based on the presented prediction-correction framework, some Lagrangian-based methods with $O(1/K^2)$ ergodic convergence rates are presented, such as the augmented Lagrangian method with the indefinite proximal term, the alternating direction method of multipliers (ADMM) with a larger step size up to $(1+\sqrt{5})/2$, the linearized ADMM with the indefinite proximal term, and the multi-block ADMM type method (under an alternative assumption that the gradient of one block is Lipschitz continuous).