Near-Optimal Lower Bounds For Convex Optimization For All Orders of Smoothness

TL;DR

This paper establishes near-optimal lower bounds for the complexity of convex optimization with smoothness of all orders, matching existing upper bounds and extending the results to randomized and quantum algorithms.

Contribution

It provides the first lower bounds that match the known upper bounds for all orders of smoothness in convex optimization, including randomized and quantum algorithms.

Findings

Lower bounds match existing upper bounds up to log factors.

Bounds apply to deterministic, randomized, and quantum algorithms.

Extends understanding of optimization complexity for highly smooth convex functions.

Abstract

We study the complexity of optimizing highly smooth convex functions. For a positive integer $p$, we want to find an $\epsilon$-approximate minimum of a convex function $f$, given oracle access to the function and its first $p$ derivatives, assuming that the $p$th derivative of $f$ is Lipschitz. Recently, three independent research groups (Jiang et al., PLMR 2019; Gasnikov et al., PLMR 2019; Bubeck et al., PLMR 2019) developed a new algorithm that solves this problem with $\tilde{O}(1/\epsilon^{\frac{2}{3p+1}})$ oracle calls for constant $p$. This is known to be optimal (up to log factors) for deterministic algorithms, but known lower bounds for randomized algorithms do not match this bound. We prove a new lower bound that matches this bound (up to log factors), and holds not only for randomized algorithms, but also for quantum algorithms.