Enhancing Adaptive Behavioral Interventions with LLM Inference from Participant-Described States

TL;DR

This paper introduces a method that uses large language models to interpret participant-described states in adaptive health interventions, expanding the state space and improving policy learning without sacrificing data efficiency.

Contribution

It proposes leveraging LLM inference on natural language participant states to enhance reinforcement learning in adaptive interventions, addressing data scarcity issues.

Findings

Improved policy performance in simulated physical activity interventions

Effective expansion of state space using natural language descriptions

Potential for better personalization in health behavior change programs

Abstract

The use of reinforcement learning (RL) methods to support health behavior change via personalized and just-in-time adaptive interventions is of significant interest to health and behavioral science researchers focused on problems such as smoking cessation support and physical activity promotion. However, RL methods are often applied to these domains using a small collection of context variables to mitigate the significant data scarcity issues that arise from practical limitations on the design of adaptive intervention trials. In this paper, we explore an approach to significantly expanding the state space of an adaptive intervention without impacting data efficiency. The proposed approach enables intervention participants to provide natural language descriptions of aspects of their current state. It then leverages inference with pre-trained large language models (LLMs) to better align the policy of a base RL method with these state descriptions. To evaluate our method, we develop a novel physical activity intervention simulation environment that generates text-based state descriptions conditioned on latent state variables using an auxiliary LLM. We show that this approach has the potential to significantly improve the performance of online policy learning methods.