Fighting Contextual Bandits with Stochastic Smoothing

TL;DR

This paper introduces a stochastic smoothing framework for adversarial contextual bandits, providing new regret bounds for a general class of perturbation-based algorithms, including EXP4.

Contribution

It presents a unified algorithm template using stochastic perturbations with strong regret guarantees, extending and generalizing existing methods like EXP4.

Findings

Achieves an $O(\sqrt{T})$ regret bound for vanilla algorithms.

Provides an $O(L^{*2/3}_{T})$ regret bound for clipped algorithms.

Includes EXP4 as a special case within the new framework.

Abstract

We introduce a new stochastic smoothing perspective to study adversarial contextual bandit problems. We propose a general algorithm template that represents random perturbation based algorithms and identify several perturbation distributions that lead to strong regret bounds. Using the idea of smoothness, we provide an $O(\sqrt{T})$ zero-order bound for the vanilla algorithm and an $O(L^{*2/3}_{T})$ first-order bound for the clipped version. These bounds hold when the algorithms use with a variety of distributions that have a bounded hazard rate. Our algorithm template includes EXP4 as a special case corresponding to the Gumbel perturbation. Our regret bounds match existing results for EXP4 without relying on the specific properties of the algorithm.