Sub-sampling for Efficient Non-Parametric Bandit Exploration

TL;DR

This paper introduces RB-SDA, a new non-parametric bandit algorithm that uses sub-sampling for exploration, achieving optimal regret across various distributions without distribution-specific tuning.

Contribution

The paper presents RB-SDA, the first sub-sampling based bandit algorithm that is distribution-agnostic and achieves asymptotically optimal regret for multiple distribution families.

Findings

RB-SDA achieves asymptotically optimal regret for Bernoulli, Gaussian, and Poisson arms.

RB-SDA does not require distribution-dependent prior tuning unlike Thompson Sampling.

Experimental results show RB-SDA's robustness and flexibility in bandit exploration.

Abstract

In this paper we propose the first multi-armed bandit algorithm based on re-sampling that achieves asymptotically optimal regret simultaneously for different families of arms (namely Bernoulli, Gaussian and Poisson distributions). Unlike Thompson Sampling which requires to specify a different prior to be optimal in each case, our proposal RB-SDA does not need any distribution-dependent tuning. RB-SDA belongs to the family of Sub-sampling Duelling Algorithms (SDA) which combines the sub-sampling idea first used by the BESA [1] and SSMC [2] algorithms with different sub-sampling schemes. In particular, RB-SDA uses Random Block sampling. We perform an experimental study assessing the flexibility and robustness of this promising novel approach for exploration in bandit models.