ALPACA: A Reinforcement Learning Environment for Medication Repurposing and Treatment Optimization in Alzheimer's Disease

TL;DR

ALPACA is an open-source reinforcement learning environment that simulates Alzheimer's disease progression, enabling personalized treatment strategy exploration and outperforming baseline policies in memory-related outcomes.

Contribution

This work introduces ALPACA, a novel RL environment for AD treatment optimization, powered by the CAST model trained on longitudinal data, facilitating in silico policy testing.

Findings

CAST generates realistic medication-conditioned trajectories

RL policies outperform no-treatment baselines

Policies rely on meaningful clinical features

Abstract

Evaluating personalized, sequential treatment strategies for Alzheimer's disease (AD) using clinical trials is often impractical due to long disease horizons and substantial inter-patient heterogeneity. To address these constraints, we present the Alzheimer's Learning Platform for Adaptive Care Agents (ALPACA), an open-source, Gym-compatible reinforcement learning (RL) environment for systematically exploring personalized treatment strategies using existing therapies. ALPACA is powered by the Continuous Action-conditioned State Transitions (CAST) model trained on longitudinal trajectories from the Alzheimer's Disease Neuroimaging Initiative (ADNI), enabling medication-conditioned simulation of disease progression under alternative treatment decisions. We show that CAST autoregressively generates realistic medication-conditioned trajectories and that RL policies trained in ALPACA outperform no-treatment and behavior-cloned clinician baselines on memory-related outcomes. Interpretability analyses further indicated that the learned policies relied on clinically meaningful patient features when selecting actions. Overall, ALPACA provides a reusable in silico testbed for studying individualized sequential treatment decision-making for AD.