Optimal Stochastic Nonconvex Optimization with Bandit Feedback

TL;DR

This paper studies nonconvex optimization in bandit settings, proposing an adaptive bin splitting method that achieves near-optimal regret bounds under certain smoothness assumptions.

Contribution

It introduces an adaptive bin splitting algorithm for nonconvex bandit problems and proves its minimax optimality in expected regret.

Findings

Adaptive bin splitting improves regret performance

The method achieves minimax optimal regret bounds

Theoretical analysis under smoothness assumptions

Abstract

In this paper, we analyze the continuous armed bandit problems for nonconvex cost functions under certain smoothness and sublevel set assumptions. We first derive an upper bound on the expected cumulative regret of a simple bin splitting method. We then propose an adaptive bin splitting method, which can significantly improve the performance. Furthermore, a minimax lower bound is derived, which shows that our new adaptive method achieves locally minimax optimal expected cumulative regret.