Improved Analysis of Restarted Accelerated Gradient and Augmented Lagrangian Methods via Inexact Proximal Point Frameworks

TL;DR

This paper introduces new inexact proximal point frameworks to analyze restarted accelerated gradient and augmented Lagrangian methods, achieving optimal complexity bounds for convex composite optimization problems.

Contribution

It develops a unified analysis framework for inexact proximal methods, leading to near-optimal complexity results for both unconstrained and constrained convex optimization.

Findings

Optimal first-order complexity for unconstrained problems.

Near-optimal complexity for linearly constrained problems.

Numerical experiments confirm practical efficiency.

Abstract

This paper studies a class of double-loop (inner-outer) algorithms for convex composite optimization. For unconstrained problems, we develop a restarted accelerated composite gradient method that attains the optimal first-order complexity in both the convex and strongly convex settings. For linearly constrained problems, we introduce inexact augmented Lagrangian methods, including a basic method and an outer-accelerated variant, and establish near-optimal first-order complexity for both methods. The established complexity bounds follow from a unified analysis based on new inexact proximal point frameworks that accommodate relative and absolute inexactness, acceleration, and strongly convex objectives. Numerical experiments on LASSO and linearly constrained quadratic programs demonstrate the practical efficiency of the proposed methods.