Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity

TL;DR

This paper presents a reinforcement learning algorithm designed to optimize personalized, just-in-time activity interventions in a mobile health app, improving physical activity promotion through adaptive decision-making.

Contribution

It introduces a novel RL-based approach for real-time personalization of health interventions within a mobile app context, advancing adaptive health technology.

Findings

The RL algorithm effectively learns personalized intervention policies.

Implementation in HeartSteps V2 demonstrates practical utility.

The approach enhances user engagement and activity levels.

Abstract

With the recent evolution of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notification on mobile device and designed to help the user prevent negative health outcomes and promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policy) that takes the user's current context as input and specifies whether and what type of an intervention should be provided at the moment. In this paper, we develop a Reinforcement Learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as the data is being collected from the user. This work is motivated by our collaboration on designing the RL algorithm in HeartSteps V2 based on data from HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this paper is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion.