Convergence of the Augmented Decomposition Algorithm

TL;DR

This paper analyzes the convergence properties of the Augmented Decomposition Algorithm (ADA) for multi-block convex problems, establishing global and local convergence rates, including linear convergence for inexact variants in practical applications.

Contribution

It provides the first convergence rate analysis for the ADA, including inexact versions, under mild assumptions, with applications to l1-regularized problems.

Findings

Global convergence rate of o(1/k) for exact ADA

Local linear convergence of inexact ADA in practical applications

Stability results for associated maximal monotone operators

Abstract

We study the convergence of the Augmented Decomposition Algorithm (ADA) proposed in [32] for solving multi-block separable convex minimization problems subject to linear constraints. We show that the global convergence rate of the exact ADA is o(1/k) under the assumption that there exists a saddle point. We consider the inexact Augmented Decomposition Algorithm (iADA) and establish global and local convergence results under some mild assumptions, by providing a stability result for the maximal monotone operator T associated with the perturbation from both primal and dual perspectives. This result implies the local linear convergence of the inexact ADA for many applications such as the lasso, total variation reconstruction, exchange problem and many other problems from statistics, machine learning and engineering with l1 regularization.