Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials

TL;DR

This paper presents a novel online reinforcement learning agent designed for personalized physical exercise recommendations in N-of-1 trials, demonstrating its feasibility and potential to improve patient outcomes with limited data.

Contribution

It introduces a new contextual bandit recommendation agent and evaluates its effectiveness through simulation studies in a clinical setting.

Findings

Feasibility of implementing the RL agent in personalized interventions

Potential for adaptive interventions to enhance patient benefits

Additional complexity in design and implementation

Abstract

Personalized adaptive interventions offer the opportunity to increase patient benefits, however, there are challenges in their planning and implementation. Once implemented, it is an important question whether personalized adaptive interventions are indeed clinically more effective compared to a fixed gold standard intervention. In this paper, we present an innovative N-of-1 trial study design testing whether implementing a personalized intervention by an online reinforcement learning agent is feasible and effective. Throughout, we use a new study on physical exercise recommendations to reduce pain in endometriosis for illustration. We describe the design of a contextual bandit recommendation agent and evaluate the agent in simulation studies. The results show that, first, implementing a personalized intervention by an online reinforcement learning agent is feasible. Second, such adaptive interventions have the potential to improve patients' benefits even if only few observations are available. As one challenge, they add complexity to the design and implementation process. In order to quantify the expected benefit, data from previous interventional studies is required. We expect our approach to be transferable to other interventions and clinical interventions.