Incentive-compatible Bandits: Importance Weighting No More

TL;DR

This paper introduces the first incentive-compatible bandit algorithms with near-optimal regret bounds, improving upon previous work by removing importance weighting and achieving strong guarantees in stochastic and adversarial settings.

Contribution

It presents novel incentive-compatible algorithms with $O( oot{K}{T})$ regret, simplifies existing algorithms via loss-biasing, and achieves best-of-both-worlds guarantees.

Findings

Achieved $O( oot{K}{T})$ regret bounds for incentive-compatible bandit algorithms.

Demonstrated that simple loss-biasing improves regret bounds of existing algorithms.

Developed a bandit algorithm with nearly optimal regret that operates without importance-weighted estimators.

Abstract

We study the problem of incentive-compatible online learning with bandit feedback. In this class of problems, the experts are self-interested agents who might misrepresent their preferences with the goal of being selected most often. The goal is to devise algorithms which are simultaneously incentive-compatible, that is the experts are incentivised to report their true preferences, and have no regret with respect to the preferences of the best fixed expert in hindsight. \citet{freeman2020no} propose an algorithm in the full information setting with optimal $O(\sqrt{T \log(K)})$ regret and $O(T^{2/3}(K\log(K))^{1/3})$ regret in the bandit setting. In this work we propose the first incentive-compatible algorithms that enjoy $O(\sqrt{KT})$ regret bounds. We further demonstrate how simple loss-biasing allows the algorithm proposed in Freeman et al. 2020 to enjoy $\tilde O(\sqrt{KT})$ regret. As a byproduct of our approach we obtain the first bandit algorithm with nearly optimal regret bounds in the adversarial setting which works entirely on the observed loss sequence without the need for importance-weighted estimators. Finally, we provide an incentive-compatible algorithm that enjoys asymptotically optimal best-of-both-worlds regret guarantees, i.e., logarithmic regret in the stochastic regime as well as worst-case $O(\sqrt{KT})$ regret.