Linear Convergence and Error Bounds for Optimization Without Strong Convexity

TL;DR

This paper establishes that local error-bound conditions are both necessary and sufficient for linear convergence of fixed-point iterations in optimization, especially for piecewise linear and quadratic problems, without requiring strong convexity.

Contribution

It proves the equivalence between local error bounds and linear convergence of averaged operators, providing explicit convergence bounds and extending analysis to common optimization algorithms.

Findings

Piecewise linear operators satisfy local error bounds.

Linear convergence rates depend only on problem condition numbers.

Framework applies to algorithms like ADMM and Douglas-Rachford.

Abstract

Many optimization algorithms$\unicode{x2013}$including gradient descent, proximal methods, and operator splitting techniques$\unicode{x2013}$can be formulated as fixed-point iterations (FPI) of continuous operators. When these operators are averaged, convergence to a fixed point is guaranteed when one exists, but the convergence is generally sublinear. Recent results establish linear convergence of FPI for averaged operators under certain conditions. However, such conditions do not apply to common classes of operators, such as those arising in piecewise linear and quadratic optimization problems. In this work, we prove that a local error-bound condition is both necessary and sufficient for the linear convergence of FPI applied to averaged operators. We provide explicit bounds on the convergence rate and show how these relate to the constants in the error-bound condition. Our main result demonstrates that piecewise linear operators satisfy local error bounds, ensuring linear convergence of the associated optimization algorithms. This leads to a general and practical framework for analyzing convergence behavior in algorithms such as ADMM and Douglas-Rachford in the absence of strong convexity. In particular, we obtain convergence rates that are independent of problem data for linear optimization, and depend only on the condition number of the objective for quadratic optimization.