A study of Thompson Sampling with Parameter h

TL;DR

This paper investigates a modified Thompson Sampling algorithm with a parameter h that adjusts the importance of the current best arm, demonstrating its robustness in two-armed bandit problems.

Contribution

It introduces a parameterized version of Thompson Sampling and analyzes its robustness, extending understanding of its optimality under perturbations.

Findings

Optimality of Thompson Sampling is robust within a range of h values.

The modified algorithm maintains performance in two-armed bandit scenarios.

Parameter h influences the probability weighting in the sampling strategy.

Abstract

Thompson Sampling algorithm is a well known Bayesian algorithm for solving stochastic multi-armed bandit. At each time step the algorithm chooses each arm with probability proportional to it being the current best arm. We modify the strategy by introducing a paramter h which alters the importance of the probability of an arm being the current best arm. We show that the optimality of Thompson sampling is robust to this perturbation within a range of parameter values for two arm bandits.