Satisficing Regret Minimization in Bandits: Constant Rate and Light-Tailed Distribution

TL;DR

This paper introduces algorithms for satisficing regret minimization in bandit problems, achieving constant or light-tailed regret distributions, and demonstrates their effectiveness through theoretical guarantees and empirical validation.

Contribution

The paper proposes the SELECT and SELECT-LITE algorithms that attain constant and light-tailed satisficing regret, respectively, in bandit optimization, extending applicability to heavy-tailed oracle scenarios.

Findings

SELECT achieves constant expected satisficing regret in realizable cases.

SELECT-LITE attains light-tailed satisficing regret distribution and sub-linear standard regret.

Algorithms perform well on synthetic and real-world datasets, including dynamic pricing.

Abstract

Motivated by the concept of satisficing in decision-making, we consider the problem of satisficing regret minimization in bandit optimization. In this setting, the learner aims at selecting satisficing arms (arms with mean reward exceeding a certain threshold value) as frequently as possible. The performance is measured by satisficing regret, which is the cumulative deficit of the chosen arm's mean reward compared to the threshold. We propose SELECT, a general algorithmic template for Satisficing REgret Minimization via SampLing and LowEr Confidence bound Testing, that attains constant expected satisficing regret for a wide variety of bandit optimization problems in the realizable case (i.e., a satisficing arm exists). As a complement, SELECT also enjoys the same (standard) regret guarantee as the oracle in the non-realizable case. To further ensure stability of the algorithm, we introduce SELECT-LITE that achieves a light-tailed satisficing regret distribution plus a constant expected satisficing regret in the realizable case and a sub-linear expected (standard) regret in the non-realizable case. Notably, SELECT-LITE can operate on learning oracles with heavy-tailed (standard) regret distribution. More importantly, our results reveal the surprising compatibility between constant expected satisficing regret and light-tailed satisficing regret distribution, which is in sharp contrast to the case of (standard) regret. Finally, we conduct numerical experiments to validate the performance of SELECT and SELECT-LITE on both synthetic datasets and a real-world dynamic pricing case study.