StepCountJITAI: simulation environment for RL with application to physical activity adaptive intervention

TL;DR

This paper introduces StepCountJITAI, a simulation environment tailored for reinforcement learning in physical activity interventions, addressing data scarcity and realistic dynamics challenges in deploying RL for behavioral health.

Contribution

The paper presents a novel RL simulation environment specifically designed for physical activity JITAIs, facilitating research on effective RL methods in this domain.

Findings

Provides a realistic simulation environment for RL in physical activity interventions

Addresses data scarcity and dynamic relevance issues in RL deployment

Supports development of more effective adaptive intervention policies

Abstract

The use of reinforcement learning (RL) to learn policies for just-in-time adaptive interventions (JITAIs) is of significant interest in many behavioral intervention domains including improving levels of physical activity. In a messaging-based physical activity JITAI, a mobile health app is typically used to send messages to a participant to encourage engagement in physical activity. In this setting, RL methods can be used to learn what intervention options to provide to a participant in different contexts. However, deploying RL methods in real physical activity adaptive interventions comes with challenges: the cost and time constraints of real intervention studies result in limited data to learn adaptive intervention policies. Further, commonly used RL simulation environments have dynamics that are of limited relevance to physical activity adaptive interventions and thus shed little light on what RL methods may be optimal for this challenging application domain. In this paper, we introduce StepCountJITAI, an RL environment designed to foster research on RL methods that address the significant challenges of policy learning for adaptive behavioral interventions.