Restless Bandits with Constrained Arms: Applications in Social and Information Networks

TL;DR

This paper models the challenge of efficiently gathering high-quality information from social network sources with varying availability and quality using a restless bandit framework, proposing policies to optimize long-term information collection.

Contribution

It formulates a partially observable restless bandit model for social information gathering and analyzes the effectiveness of Whittle's index policy in this context.

Findings

Whittle's index policy outperforms myopic and random policies in simulations.

The model effectively captures the dynamics of information quality and source availability.

Numerical results demonstrate improved long-term reward with the proposed approach.

Abstract

We study a problem of information gathering in a social network with dynamically available sources and time varying quality of information. We formulate this problem as a restless multi-armed bandit (RMAB). In this problem, information quality of a source corresponds to the state of an arm in RMAB. The decision making agent does not know the quality of information from sources a priori. But the agent maintains a belief about the quality of information from each source. This is a problem of RMAB with partially observable states. The objective of the agent is to gather relevant information efficiently from sources by contacting them. We formulate this as a infinite horizon discounted reward problem, where reward depends on quality of information. We study Whittle's index policy which determines the sequence of play of arms that maximizes long term cumulative reward. We illustrate the performance of index policy, myopic policy and compare with uniform random policy through numerical simulation.