Towards Soft Fairness in Restless Multi-Armed Bandits

TL;DR

This paper introduces SoftFair, a novel approach for incorporating soft fairness constraints into restless multi-armed bandits, ensuring equitable resource allocation without significant loss in reward.

Contribution

It proposes a soft fairness constraint for RMABs and develops a softmax-based value iteration algorithm that guarantees asymptotic optimality and fairness.

Findings

SoftFair effectively enforces fairness in simulated benchmarks.

The approach achieves near-optimal rewards while maintaining fairness.

Theoretical guarantees support the method's asymptotic optimality.

Abstract

Restless multi-armed bandits (RMAB) is a framework for allocating limited resources under uncertainty. It is an extremely useful model for monitoring beneficiaries and executing timely interventions to ensure maximum benefit in public health settings (e.g., ensuring patients take medicines in tuberculosis settings, ensuring pregnant mothers listen to automated calls about good pregnancy practices). Due to the limited resources, typically certain communities or regions are starved of interventions that can have follow-on effects. To avoid starvation in the executed interventions across individuals/regions/communities, we first provide a soft fairness constraint and then provide an approach to enforce the soft fairness constraint in RMABs. The soft fairness constraint requires that an algorithm never probabilistically favor one arm over another if the long-term cumulative reward of choosing the latter arm is higher. Our approach incorporates softmax based value iteration method in the RMAB setting to design selection algorithms that manage to satisfy the proposed fairness constraint. Our method, referred to as SoftFair, also provides theoretical performance guarantees and is asymptotically optimal. Finally, we demonstrate the utility of our approaches on simulated benchmarks and show that the soft fairness constraint can be handled without a significant sacrifice on value.