pH-RL: A personalization architecture to bring reinforcement learning to health practice

TL;DR

This paper introduces pH-RL, a flexible reinforcement learning architecture designed for personalized health applications, demonstrating its effectiveness through integration with a mental health app and a human study.

Contribution

It presents a general RL framework for health personalization, including implementation guidelines and an open-source platform for deployment.

Findings

Policies learn to select appropriate actions within a few days

Learned policies are stable over time

Effective integration with mobile health applications

Abstract

While reinforcement learning (RL) has proven to be the approach of choice for tackling many complex problems, it remains challenging to develop and deploy RL agents in real-life scenarios successfully. This paper presents pH-RL (personalization in e-Health with RL) a general RL architecture for personalization to bring RL to health practice. pH-RL allows for various levels of personalization in health applications and allows for online and batch learning. Furthermore, we provide a general-purpose implementation framework that can be integrated with various healthcare applications. We describe a step-by-step guideline for the successful deployment of RL policies in a mobile application. We implemented our open-source RL architecture and integrated it with the MoodBuster mobile application for mental health to provide messages to increase daily adherence to the online therapeutic modules. We then performed a comprehensive study with human participants over a sustained period. Our experimental results show that the developed policies learn to select appropriate actions consistently using only a few days' worth of data. Furthermore, we empirically demonstrate the stability of the learned policies during the study.