Bridging Rested and Restless Bandits with Graph-Triggering: Rising and Rotting

TL;DR

This paper introduces Graph-Triggered Bandits (GTBs), a unified framework that models the evolution of arm rewards through a graph, encompassing rested and restless bandits, and explores optimal policies for rising and rotting reward behaviors.

Contribution

The work proposes GTBs as a generalization of rested and restless bandits, providing algorithms and theoretical analysis for specific monotonic reward cases.

Findings

GTBs unify rested and restless bandits via graph modeling.

Algorithms with theoretical guarantees are developed for rising and rotting cases.

The complexity of learning depends on the underlying graph structure.

Abstract

Rested and Restless Bandits are two well-known bandit settings that are useful to model real-world sequential decision-making problems in which the expected reward of an arm evolves over time due to the actions we perform or due to the nature. In this work, we propose Graph-Triggered Bandits (GTBs), a unifying framework to generalize and extend rested and restless bandits. In this setting, the evolution of the arms' expected rewards is governed by a graph defined over the arms. An edge connecting a pair of arms $(i,j)$ represents the fact that a pull of arm $i$ triggers the evolution of arm $j$, and vice versa. Interestingly, rested and restless bandits are both special cases of our model for some suitable (degenerated) graph. As relevant case studies for this setting, we focus on two specific types of monotonic bandits: rising, where the expected reward of an arm grows as the number of triggers increases, and rotting, where the opposite behavior occurs. For these cases, we study the optimal policies. We provide suitable algorithms for all scenarios and discuss their theoretical guarantees, highlighting the complexity of the learning problem concerning instance-dependent terms that encode specific properties of the underlying graph structure.