On the complexity of finding stationary points of smooth functions in one dimension

TL;DR

This paper analyzes the query complexity of finding stationary points in one-dimensional smooth functions, revealing the benefits of randomness and zeroth-order information, and establishing the optimality of gradient descent among certain algorithms.

Contribution

It characterizes the query complexity in various settings and proves the optimality of gradient descent among deterministic first-order methods.

Findings

Randomness or zeroth-order info improves algorithm performance.

Gradient descent is optimal among deterministic first-order algorithms.

The study covers all dimensions d ≥ 1.

Abstract

We characterize the query complexity of finding stationary points of one-dimensional non-convex but smooth functions. We consider four settings, based on whether the algorithms under consideration are deterministic or randomized, and whether the oracle outputs $1^{\rm st}$-order or both $0^{\rm th}$- and $1^{\rm st}$-order information. Our results show that algorithms for this task provably benefit by incorporating either randomness or $0^{\rm th}$-order information. Our results also show that, for every dimension $d \geq 1$, gradient descent is optimal among deterministic algorithms using $1^{\rm st}$-order queries only.