RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement Pathway Optimization

TL;DR

This paper presents a reinforcement learning approach to optimize clinical pathways for knee replacement surgery, reducing costs and improving decision-making by learning from expert data.

Contribution

It introduces a novel RL-based pipeline that models pathway optimization as a crowdsourcing problem and applies advanced techniques like value iteration and state aggregation.

Findings

Optimized policy reduces overall cost by 7%.

Reduces excessive cost premium by 33%.

Provides accurate pathway forecasts under the learned policy.

Abstract

Joint replacement is the most common inpatient surgical treatment in the US. We investigate the clinical pathway optimization for knee replacement, which is a sequential decision process from onset to recovery. Based on episodic claims from previous cases, we view the pathway optimization as an intelligence crowdsourcing problem and learn the optimal decision policy from data by imitating the best expert at every intermediate state. We develop a reinforcement learning-based pipeline that uses value iteration, state compression and aggregation learning, kernel representation and cross validation to predict the best treatment policy. It also provides forecast of the clinical pathway under the optimized policy. Empirical validation shows that the optimized policy reduces the overall cost by 7 percent and reduces the excessive cost premium by 33 percent.