Optimal Subgradient Methods for Lipschitz Convex Optimization with Error Bounds

TL;DR

This paper analyzes the iteration complexity of Lipschitz convex optimization problems with error bounds, demonstrating that subgradient methods with specific stepsizes achieve optimal convergence guarantees.

Contribution

It introduces a novel lower-bounding technique for establishing minimax optimal convergence guarantees for subgradient methods under error bounds.

Findings

Subgradient descent with Polyak or decaying stepsizes achieves optimal convergence.

A new lower-bounding argument constructs hard functions satisfying error bounds.

The results establish fundamental limits for Lipschitz convex optimization algorithms.

Abstract

We study the iteration complexity of Lipschitz convex optimization problems satisfying a general error bound. We show that for this class of problems, subgradient descent with either Polyak stepsizes or decaying stepsizes achieves minimax optimal convergence guarantees for decreasing distance-to-optimality. The main contribution is a novel lower-bounding argument that produces hard functions simultaneously satisfying zero-chain conditions and global error bounds.