Batched Stochastic Bandit for Nondegenerate Functions

TL;DR

This paper introduces the Geometric Narrowing (GN) algorithm for batched stochastic bandit problems with nondegenerate functions, achieving near-optimal regret with very few batches, and provides matching lower bounds.

Contribution

The paper proposes the GN algorithm that attains near-optimal regret using only logarithmic double log batches, and establishes lower bounds showing the algorithm's near-optimality.

Findings

GN achieves regret of order rac{A_+^d}{\u00f8} \, \, ext{and} \, \, rac{A_-^d}{\u00f8} \, \, ext{for upper and lower bounds}

GN requires only rac{\, \, ext{log log T}}{ ext{batches}} to perform near-optimally

Lower bounds demonstrate the minimal number of batches needed for any policy to achieve low regret

Abstract

This paper studies batched bandit learning problems for nondegenerate functions. We introduce an algorithm that solves the batched bandit problem for nondegenerate functions near-optimally. More specifically, we introduce an algorithm, called Geometric Narrowing (GN), whose regret bound is of order $\widetilde{{\mathcal{O}}} ( A_{+}^d \sqrt{T} )$. In addition, GN only needs $\mathcal{O} (\log \log T)$ batches to achieve this regret. We also provide lower bound analysis for this problem. More specifically, we prove that over some (compact) doubling metric space of doubling dimension $d$: 1. For any policy $\pi$, there exists a problem instance on which $\pi$ admits a regret of order ${\Omega} ( A_-^d \sqrt{T})$; 2. No policy can achieve a regret of order $ A_-^d \sqrt{T} $ over all problem instances, using less than $ \Omega ( \log \log T ) $ rounds of communications. Our lower bound analysis shows that the GN algorithm achieves near optimal regret with minimal number of batches.