Improved Regret of Linear Ensemble Sampling

TL;DR

This paper presents an improved theoretical regret bound for linear ensemble sampling in bandit problems, matching state-of-the-art results and revealing a key relationship with LinPHE, thus advancing the understanding of randomized exploration algorithms.

Contribution

It introduces a general regret analysis framework for linear bandit algorithms and establishes a connection between linear ensemble sampling and LinPHE.

Findings

Achieves a regret bound of ^{3/2}b7b7T for linear ensemble sampling.

Shows LinPHE is a special case of linear ensemble sampling with ensemble size T.

Provides theoretical insights aligning ensemble sampling with other exploration algorithms.

Abstract

In this work, we close the fundamental gap of theory and practice by providing an improved regret bound for linear ensemble sampling. We prove that with an ensemble size logarithmic in $T$, linear ensemble sampling can achieve a frequentist regret bound of $\tilde{O}(d^{3/2}\sqrt{T})$, matching state-of-the-art results for randomized linear bandit algorithms, where $d$ and $T$ are the dimension of the parameter and the time horizon respectively. Our approach introduces a general regret analysis framework for linear bandit algorithms. Additionally, we reveal a significant relationship between linear ensemble sampling and Linear Perturbed-History Exploration (LinPHE), showing that LinPHE is a special case of linear ensemble sampling when the ensemble size equals $T$. This insight allows our analysis framework to derive a regret bound of $\tilde{O}(d^{3/2}\sqrt{T})$ for LinPHE, independent of the number of arms. Our techniques advance the theoretical foundation of ensemble sampling, bringing its regret bounds in line with the best known bounds for other randomized exploration algorithms.