Faster Lagrangian-Based Methods in Convex Optimization

TL;DR

This paper introduces a unified framework for Lagrangian-based convex optimization methods, leading to faster convergence rates and broad applicability to existing algorithms through the concept of nice primal algorithmic maps.

Contribution

The paper proposes the notion of nice primal algorithmic maps and a generic scheme for Faster Lagrangian (FLAG) methods, improving convergence analysis and rates for a wide class of algorithms.

Findings

Most well-known Lagrangian schemes admit a nice primal algorithmic map.

FLAG methods achieve new provably sublinear convergence rates.

The approach unifies and simplifies convergence analysis across methods.

Abstract

In this paper, we aim at unifying, simplifying and improving the convergence rate analysis of Lagrangian-based methods for convex optimization problems. We first introduce the notion of nice primal algorithmic map, which plays a central role in the unification and in the simplification of the analysis of most Lagrangian-based methods. Equipped with a nice primal algorithmic map, we then introduce a versatile generic scheme, which allows for the design and analysis of Faster LAGrangian (FLAG) methods with new provably sublinear rate of convergence expressed in terms of function values and feasibility violation of the original (non-ergodic) generated sequence. To demonstrate the power and versatility of our approach and results, we show that most well-known iconic Lagrangian-based schemes admit a nice primal algorithmic map, and hence share the new faster rate of convergence results within their corresponding FLAG.