Incentivized Exploration for Multi-Armed Bandits under Reward Drift

TL;DR

This paper analyzes incentivized exploration in multi-armed bandits with reward drift, demonstrating that UCB, epsilon-Greedy, and Thompson Sampling achieve logarithmic regret and compensation, thus effectively encouraging exploration despite biased feedback.

Contribution

It introduces a theoretical analysis of incentivized exploration algorithms under reward drift, showing their effectiveness in maintaining low regret and compensation.

Findings

All three algorithms achieve (\,log T) regret and compensation.

Algorithms remain effective despite biased reward feedback.

Numerical examples support theoretical results.

Abstract

We study incentivized exploration for the multi-armed bandit (MAB) problem where the players receive compensation for exploring arms other than the greedy choice and may provide biased feedback on reward. We seek to understand the impact of this drifted reward feedback by analyzing the performance of three instantiations of the incentivized MAB algorithm: UCB, $\varepsilon$-Greedy, and Thompson Sampling. Our results show that they all achieve $\mathcal{O}(\log T)$ regret and compensation under the drifted reward, and are therefore effective in incentivizing exploration. Numerical examples are provided to complement the theoretical analysis.