On Penalization in Stochastic Multi-armed Bandits

TL;DR

This paper introduces a penalization framework for stochastic multi-armed bandits, balancing reward maximization with fairness, and proposes a UCB-like algorithm with strong theoretical guarantees and empirical performance.

Contribution

It formulates a new penalization-based approach for MAB, providing rigorous regret analysis and a novel algorithm that balances reward and fairness effectively.

Findings

Nearly optimal regret bounds established

Algorithm achieves asymptotic fairness

Experimental results show superiority over existing methods

Abstract

We study an important variant of the stochastic multi-armed bandit (MAB) problem, which takes penalization into consideration. Instead of directly maximizing cumulative expected reward, we need to balance between the total reward and fairness level. In this paper, we present some new insights in MAB and formulate the problem in the penalization framework, where rigorous penalized regret can be well defined and more sophisticated regret analysis is possible. Under such a framework, we propose a hard-threshold UCB-like algorithm, which enjoys many merits including asymptotic fairness, nearly optimal regret, better tradeoff between reward and fairness. Both gap-dependent and gap-independent regret bounds have been established. Multiple insightful comments are given to illustrate the soundness of our theoretical analysis. Numerous experimental results corroborate the theory and show the superiority of our method over other existing methods.