An Asymptotically Optimal Policy for Uniform Bandits of Unknown Support

TL;DR

This paper introduces an asymptotically optimal policy for selecting among multiple unknown uniform distributions, minimizing regret and achieving theoretical lower bounds in sequential sampling scenarios.

Contribution

The paper proposes a simple inflated sample mean (ISM) policy that is proven to be asymptotically optimal for uniform bandits with unknown support, with finite horizon regret bounds.

Findings

The ISM policy achieves the asymptotic lower bound of regret.

Finite horizon regret bounds are established.

The policy is simple and effective for uniform bandits with unknown parameters.

Abstract

Consider the problem of a controller sampling sequentially from a finite number of $N \geq 2$ populations, specified by random variables $X^i_k$, $ i = 1,\ldots , N,$ and $k = 1, 2, \ldots$; where $X^i_k$ denotes the outcome from population $i$ the $k^{th}$ time it is sampled. It is assumed that for each fixed $i$, $\{ X^i_k \}_{k \geq 1}$ is a sequence of i.i.d. uniform random variables over some interval $[a_i, b_i]$, with the support (i.e., $a_i, b_i$) unknown to the controller. The objective is to have a policy $\pi$ for deciding, based on available data, from which of the $N$ populations to sample from at any time $n=1,2,\ldots$ so as to maximize the expected sum of outcomes of $n$ samples or equivalently to minimize the regret due to lack on information of the parameters $\{ a_i \}$ and $\{ b_i \}$. In this paper, we present a simple inflated sample mean (ISM) type policy that is asymptotically optimal in the sense of its regret achieving the asymptotic lower bound of Burnetas and Katehakis (1996). Additionally, finite horizon regret bounds are given.