Bayesian Collaborative Bandits with Thompson Sampling for Improved Outreach in Maternal Health Program

TL;DR

This paper introduces a Bayesian Thompson Sampling approach for collaborative multi-armed bandits to optimize outreach timing in maternal health programs, significantly reducing calls and improving beneficiary retention.

Contribution

It presents a novel Bayesian method with Gibbs sampling for low-rank reward matrix inference, outperforming existing heuristics in real-world maternal health mHealth applications.

Findings

16% reduction in calls compared to baselines

47% reduction compared to random policy

7-29% improvement in beneficiary retention

Abstract

Mobile health (mHealth) programs face a critical challenge in optimizing the timing of automated health information calls to beneficiaries. This challenge has been formulated as a collaborative multi-armed bandit problem, requiring online learning of a low-rank reward matrix. Existing solutions often rely on heuristic combinations of offline matrix completion and exploration strategies. In this work, we propose a principled Bayesian approach using Thompson Sampling for this collaborative bandit problem. Our method leverages prior information through efficient Gibbs sampling for posterior inference over the low-rank matrix factors, enabling faster convergence. We demonstrate significant improvements over state-of-the-art baselines on a real-world dataset from the world's largest maternal mHealth program. Our approach achieves a $16\%$ reduction in the number of calls compared to existing methods and a $47$\% reduction compared to the deployed random policy. This efficiency gain translates to a potential increase in program capacity by $0.5-1.4$ million beneficiaries, granting them access to vital ante-natal and post-natal care information. Furthermore, we observe a $7\%$ and $29\%$ improvement in beneficiary retention (an extremely hard metric to impact) compared to state-of-the-art and deployed baselines, respectively. Synthetic simulations further demonstrate the superiority of our approach, particularly in low-data regimes and in effectively utilizing prior information. We also provide a theoretical analysis of our algorithm in a special setting using Eluder dimension.