Networked Restless Bandits with Positive Externalities

TL;DR

This paper introduces networked restless bandits, modeling community-based resource allocation with positive externalities, and proposes Greta, a graph-aware heuristic that outperforms existing policies in various settings.

Contribution

The paper develops a new networked restless bandit model incorporating externalities and presents Greta, a novel index-based algorithm tailored for this setting.

Findings

Greta outperforms comparison policies across different graph topologies.

The model captures positive externalities in community resource allocation.

Empirical results validate the effectiveness of Greta in various scenarios.

Abstract

Restless multi-armed bandits are often used to model budget-constrained resource allocation tasks where receipt of the resource is associated with an increased probability of a favorable state transition. Prior work assumes that individual arms only benefit if they receive the resource directly. However, many allocation tasks occur within communities and can be characterized by positive externalities that allow arms to derive partial benefit when their neighbor(s) receive the resource. We thus introduce networked restless bandits, a novel multi-armed bandit setting in which arms are both restless and embedded within a directed graph. We then present Greta, a graph-aware, Whittle index-based heuristic algorithm that can be used to efficiently construct a constrained reward-maximizing action vector at each timestep. Our empirical results demonstrate that Greta outperforms comparison policies across a range of hyperparameter values and graph topologies.