Ensemble Sampling

TL;DR

Ensemble sampling is introduced as a scalable method to approximate Thompson sampling, enabling its application to complex models like neural networks while maintaining computational tractability.

Contribution

The paper develops ensemble sampling, a novel approach that extends Thompson sampling to complex models, supported by theoretical analysis and computational experiments.

Findings

Ensemble sampling effectively approximates Thompson sampling in complex models.

The approach broadens the applicability of Bayesian decision algorithms.

Computational results demonstrate the method's practical viability.

Abstract

Thompson sampling has emerged as an effective heuristic for a broad range of online decision problems. In its basic form, the algorithm requires computing and sampling from a posterior distribution over models, which is tractable only for simple special cases. This paper develops ensemble sampling, which aims to approximate Thompson sampling while maintaining tractability even in the face of complex models such as neural networks. Ensemble sampling dramatically expands on the range of applications for which Thompson sampling is viable. We establish a theoretical basis that supports the approach and present computational results that offer further insight.