Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space

TL;DR

This paper introduces a novel offline reinforcement learning approach in continuous space to optimize sepsis treatment strategies, demonstrating improved patient outcomes and providing personalized, interpretable recommendations.

Contribution

It develops a new medical decision model combining offline and deep reinforcement learning for continuous treatment spaces, addressing limitations of traditional methods.

Findings

AI-recommended treatments outperform clinician decisions on average.

Patients matching AI treatment decisions have lower mortality rates.

Model offers personalized, interpretable treatment suggestions.

Abstract

Sepsis is a leading cause of death in the ICU. It is a disease requiring complex interventions in a short period of time, but its optimal treatment strategy remains uncertain. Evidence suggests that the practices of currently used treatment strategies are problematic and may cause harm to patients. To address this decision problem, we propose a new medical decision model based on historical data to help clinicians recommend the best reference option for real-time treatment. Our model combines offline reinforcement learning and deep reinforcement learning to solve the problem of traditional reinforcement learning in the medical field due to the inability to interact with the environment, while enabling our model to make decisions in a continuous state-action space. We demonstrate that, on average, the treatments recommended by the model are more valuable and reliable than those recommended by clinicians. In a large validation dataset, we find out that the patients whose actual doses from clinicians matched the decisions made by AI has the lowest mortality rates. Our model provides personalized and clinically interpretable treatment decisions for sepsis to improve patient care.