Improved Dimension Dependence for Bandit Convex Optimization with Gradient Variations

TL;DR

This paper advances bandit convex optimization by refining gradient variation analysis, leading to improved dimension dependence, new problem-dependent guarantees, and first bounds for one-point feedback and dynamic regret in challenging settings.

Contribution

It introduces a refined analysis of non-consecutive gradient variation, improving dimension dependence and extending bounds to one-point feedback and dynamic regret in bandit convex optimization.

Findings

Improved dimension dependence for convex and strongly convex functions.

First gradient-variation bounds for one-point bandit linear optimization.

Established dynamic and universal regret bounds in challenging bandit tasks.

Abstract

Gradient-variation online learning has drawn increasing attention due to its deep connections to game theory, optimization, etc. It has been studied extensively in the full-information setting, but is underexplored with bandit feedback. In this work, we focus on gradient variation in Bandit Convex Optimization (BCO) with two-point feedback. By proposing a refined analysis on the non-consecutive gradient variation, a fundamental quantity in gradient variation with bandits, we improve the dimension dependence for both convex and strongly convex functions compared with the best known results (Chiang et al., 2013). Our improved analysis for the non-consecutive gradient variation also implies other favorable problem-dependent guarantees, such as gradient-variance and small-loss regrets. Beyond the two-point setup, we demonstrate the versatility of our technique by achieving the first gradient-variation bound for one-point bandit linear optimization over hyper-rectangular domains. Finally, we validate the effectiveness of our results in more challenging tasks such as dynamic/universal regret minimization and bandit games, establishing the first gradient-variation dynamic and universal regret bounds for two-point BCO and fast convergence rates in bandit games.