On the oracle complexity of smooth strongly convex minimization

TL;DR

This paper establishes tight lower bounds on the oracle complexity of minimizing smooth strongly convex functions, matching known upper bounds and precisely characterizing the difficulty of the problem.

Contribution

The authors develop a new function construction to derive exact lower bounds on oracle complexity for smooth strongly convex minimization, confirming the optimality of existing algorithms.

Findings

Lower bounds match known upper bounds up to a constant factor.

Exact oracle complexity is established for the case when inaccuracy is measured by distance to the solution set.

The bounds are applicable under various inaccuracy criteria.

Abstract

We construct a family of functions suitable for establishing lower bounds on the oracle complexity of first-order minimization of smooth strongly-convex functions. Based on this construction, we derive new lower bounds on the complexity of strongly-convex minimization under various inaccuracy criteria. The new bounds match the known upper bounds up to a constant factor, and when the inaccuracy of a solution is measured by its distance to the solution set, the new lower bound exactly matches the upper bound obtained by the recent Information-Theoretic Exact Method by the same authors, thereby establishing the exact oracle complexity for this class of problems.