Near-Optimal Methods for Minimizing Star-Convex Functions and Beyond

TL;DR

This paper introduces near-optimal accelerated first-order methods for minimizing smooth quasar-convex functions, a broad class including convex and star-convex functions, with provable efficiency bounds.

Contribution

The authors develop a variant of accelerated gradient descent for smooth quasar-convex functions and establish near-matching lower bounds, extending optimization techniques beyond convex functions.

Findings

Achieves $O(rac{1}{eta} rac{1}{ oot{2}\epsilon} ext{log}(rac{1}{eta \epsilon}))$ complexity for $eta$-quasar-convex functions.

Proves a lower bound of $ ilde{ ext{Omega}}(rac{1}{eta} rac{1}{ oot{2}\epsilon})$ for any deterministic first-order method.

Extends the scope of accelerated methods to a broader class of nonconvex functions.

Abstract

In this paper, we provide near-optimal accelerated first-order methods for minimizing a broad class of smooth nonconvex functions that are strictly unimodal on all lines through a minimizer. This function class, which we call the class of smooth quasar-convex functions, is parameterized by a constant $\gamma \in (0,1]$, where $\gamma = 1$ encompasses the classes of smooth convex and star-convex functions, and smaller values of $\gamma$ indicate that the function can be "more nonconvex." We develop a variant of accelerated gradient descent that computes an $\epsilon$-approximate minimizer of a smooth $\gamma$-quasar-convex function with at most $O(\gamma^{-1} \epsilon^{-1/2} \log(\gamma^{-1} \epsilon^{-1}))$ total function and gradient evaluations. We also derive a lower bound of $\Omega(\gamma^{-1} \epsilon^{-1/2})$ on the worst-case number of gradient evaluations required by any deterministic first-order method, showing that, up to a logarithmic factor, no deterministic first-order method can improve upon ours.