The Impact of Batch Learning in Stochastic Bandits

TL;DR

This paper analyzes how batch learning affects regret in stochastic bandit problems, providing theoretical bounds and empirical validation, with implications for practical batch size selection in recommender systems.

Contribution

It introduces a batch-centric analysis of bandit problems, deriving regret bounds and exploring the impact of batch size on learning performance.

Findings

Batch learning influences regret bounds in stochastic bandits.

Optimal batch size depends on theoretical and empirical considerations.

Theoretical results are validated through empirical experiments.

Abstract

We consider a special case of bandit problems, namely batched bandits. Motivated by natural restrictions of recommender systems and e-commerce platforms, we assume that a learning agent observes responses batched in groups over a certain time period. Unlike previous work, we consider a more practically relevant batch-centric scenario of batch learning. We provide a policy-agnostic regret analysis and demonstrate upper and lower bounds for the regret of a candidate policy. Our main theoretical results show that the impact of batch learning can be measured in terms of online behavior. Finally, we demonstrate the consistency of theoretical results by conducting empirical experiments and reflect on the optimal batch size choice.