A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health

TL;DR

This paper introduces a Decision Language Model (DLM) that leverages Large Language Models to dynamically adapt and optimize resource allocation policies in public health using human-language commands, enhancing flexibility over traditional RMAB models.

Contribution

The paper presents a novel DLM framework that enables real-time policy fine-tuning in RMABs through natural language prompts and iterative feedback, bridging LLMs with public health resource management.

Findings

DLM effectively interprets human policy prompts.

DLM can generate and refine reward functions.

Simulation shows improved policy shaping with DLM.

Abstract

Restless multi-armed bandits (RMAB) have demonstrated success in optimizing resource allocation for large beneficiary populations in public health settings. Unfortunately, RMAB models lack flexibility to adapt to evolving public health policy priorities. Concurrently, Large Language Models (LLMs) have emerged as adept automated planners across domains of robotic control and navigation. In this paper, we propose a Decision Language Model (DLM) for RMABs, enabling dynamic fine-tuning of RMAB policies in public health settings using human-language commands. We propose using LLMs as automated planners to (1) interpret human policy preference prompts, (2) propose reward functions as code for a multi-agent RMAB environment, and (3) iterate on the generated reward functions using feedback from grounded RMAB simulations. We illustrate the application of DLM in collaboration with ARMMAN, an India-based non-profit promoting preventative care for pregnant mothers, that currently relies on RMAB policies to optimally allocate health worker calls to low-resource populations. We conduct a technology demonstration in simulation using the Gemini Pro model, showing DLM can dynamically shape policy outcomes using only human prompts as input.