Model-Free Reinforcement Learning for Automated Fluid Administration in Critical Care

TL;DR

This paper introduces a model-free reinforcement learning method for automated fluid administration in critical care, enabling adaptive, robust control without relying on complex physiological models.

Contribution

It presents a novel Q-learning based RL approach for AFAS, eliminating the need for precise patient models and improving robustness in critical care scenarios.

Findings

The RL agent successfully learns optimal fluid infusion strategies.

The method demonstrates robustness against clinical noise.

Simulation results show effective control of blood volume.

Abstract

Fluid administration, also called fluid resuscitation, is a medical treatment to restore the lost blood volume and optimize cardiac functions in critical care scenarios such as burn, hemorrhage, and septic shock. Automated fluid administration systems (AFAS), a potential means to improve the treatment, employ computational control algorithms to automatically adjust optimal fluid infusion dosages by targeting physiological variables (e.g., blood volume or blood pressure). Most of the existing AFAS control algorithms are model-based approaches, and their performance is highly dependent on the model accuracy, making them less desirable in real-world care of critically ill patients due to complexity and variability of modeling patients physiological states. This work presents a novel model-free reinforcement learning (RL) approach for the control of fluid infusion dosages in AFAS systems. The proposed RL agent learns to adjust the blood volume to a desired value by choosing the optimal infusion dosages using a Q-learning algorithm. The RL agent learns the optimal actions by interacting with the environment (without having the knowledge of system dynamics). The proposed methodology (i) overcomes the need for a precise mathematical model in AFAS systems and (ii) provides a robust performance in rejecting clinical noises and reaching desired hemodynamic states, as will be shown by simulation results.