Understanding Memory-Regret Trade-Off for Streaming Stochastic Multi-Armed Bandits

TL;DR

This paper characterizes the optimal regret for streaming stochastic multi-armed bandits with memory constraints, providing algorithms and bounds that depend on the number of passes, arms, and memory size.

Contribution

It offers a complete characterization of the optimal regret in the streaming multi-armed bandit problem with memory limitations, including matching upper and lower bounds.

Findings

Designed an algorithm with specific regret bounds.

Proved a matching lower bound for the regret.

Results are tight up to a logarithmic factor.

Abstract

We study the stochastic multi-armed bandit problem in the $P$-pass streaming model. In this problem, the $n$ arms are present in a stream and at most $m<n$ arms and their statistics can be stored in the memory. We give a complete characterization of the optimal regret in terms of $m, n$ and $P$. Specifically, we design an algorithm with $\tilde O\left((n-m)^{1+\frac{2^{P}-2}{2^{P+1}-1}} n^{\frac{2-2^{P+1}}{2^{P+1}-1}} T^{\frac{2^P}{2^{P+1}-1}}\right)$ regret and complement it with an $\tilde \Omega\left((n-m)^{1+\frac{2^{P}-2}{2^{P+1}-1}} n^{\frac{2-2^{P+1}}{2^{P+1}-1}} T^{\frac{2^P}{2^{P+1}-1}}\right)$ lower bound when the number of rounds $T$ is sufficiently large. Our results are tight up to a logarithmic factor in $n$ and $P$.