Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes

TL;DR

This paper introduces a scalable kernel-based distributed Q-learning algorithm designed for dynamic treatment regimes, effectively handling large EHR datasets with reduced computational complexity while maintaining strong performance.

Contribution

The paper proposes a novel scalable kernel-based distributed Q-learning method that improves computational efficiency for dynamic treatment regimes in large health data.

Findings

Significantly reduces computational complexity compared to deep RL methods

Maintains comparable generalization performance in treatment outcome prediction

Demonstrates effectiveness through theoretical and numerical analysis

Abstract

In recent years, large amounts of electronic health records (EHRs) concerning chronic diseases have been collected to facilitate medical diagnosis. Modeling the dynamic properties of EHRs related to chronic diseases can be efficiently done using dynamic treatment regimes (DTRs). While reinforcement learning (RL) is a widely used method for creating DTRs, there is ongoing research in developing RL algorithms that can effectively handle large amounts of data. In this paper, we present a scalable kernel-based distributed Q-learning algorithm for generating DTRs. We perform both theoretical assessments and numerical analysis for the proposed approach. The results demonstrate that our algorithm significantly reduces the computational complexity associated with the state-of-the-art deep reinforcement learning methods, while maintaining comparable generalization performance in terms of accumulated rewards across stages, such as survival time or cumulative survival probability.