Batch Ensemble for Variance Dependent Regret in Stochastic Bandits

TL;DR

This paper introduces a simple batch ensemble method for stochastic bandits that achieves near-optimal regret without relying on distribution-dependent parameters, demonstrated through theoretical analysis and synthetic experiments.

Contribution

It proposes a novel batch ensemble algorithm with a single parameter that provably attains near-optimal regret in stochastic bandits, independent of loss distribution properties.

Findings

Achieves near-optimal regret in stochastic bandits.

Parameter depends only on the number of batches, not on loss distribution.

Effective performance demonstrated on synthetic benchmarks.

Abstract

Efficiently trading off exploration and exploitation is one of the key challenges in online Reinforcement Learning (RL). Most works achieve this by carefully estimating the model uncertainty and following the so-called optimistic model. Inspired by practical ensemble methods, in this work we propose a simple and novel batch ensemble scheme that provably achieves near-optimal regret for stochastic Multi-Armed Bandits (MAB). Crucially, our algorithm has just a single parameter, namely the number of batches, and its value does not depend on distributional properties such as the scale and variance of the losses. We complement our theoretical results by demonstrating the effectiveness of our algorithm on synthetic benchmarks.