Deep Attention Q-Network for Personalized Treatment Recommendation

TL;DR

This paper introduces a Deep Attention Q-Network that uses Transformer architecture to incorporate all past patient data for improved personalized treatment recommendations in critical care, outperforming existing models.

Contribution

The study presents a novel deep reinforcement learning model integrating Transformer-based attention to utilize comprehensive patient history for treatment decisions.

Findings

Outperforms state-of-the-art models on sepsis and hypotension datasets

Demonstrates improved treatment recommendation accuracy

Shows effective integration of historical patient data

Abstract

Tailoring treatment for individual patients is crucial yet challenging in order to achieve optimal healthcare outcomes. Recent advances in reinforcement learning offer promising personalized treatment recommendations; however, they rely solely on current patient observations (vital signs, demographics) as the patient's state, which may not accurately represent the true health status of the patient. This limitation hampers policy learning and evaluation, ultimately limiting treatment effectiveness. In this study, we propose the Deep Attention Q-Network for personalized treatment recommendations, utilizing the Transformer architecture within a deep reinforcement learning framework to efficiently incorporate all past patient observations. We evaluated the model on real-world sepsis and acute hypotension cohorts, demonstrating its superiority to state-of-the-art models. The source code for our model is available at https://github.com/stevenmsm/RL-ICU-DAQN.