Reinforcement Learning for Target Zone Blood Glucose Control

TL;DR

This paper introduces a novel reinforcement learning framework for blood glucose control in Type 1 Diabetes, combining impulse and switching control modalities to improve safety and effectiveness in treatment management.

Contribution

The work presents a unified RL approach that models complex temporal dynamics and incorporates physiological factors, advancing safe and realistic decision-making in diabetes care.

Findings

Reduced blood glucose violations from 22.4% to 10.8%.

Provides theoretical guarantees of convergence.

Demonstrates empirical improvements in a stylised T1DM control task.

Abstract

Managing physiological variables within clinically safe target zones is a central challenge in healthcare, particularly for chronic conditions such as Type 1 Diabetes Mellitus (T1DM). Reinforcement learning (RL) offers promise for personalising treatment, but struggles with the delayed and heterogeneous effects of interventions. We propose a novel RL framework to study and support decision-making in T1DM technologies, such as automated insulin delivery. Our approach captures the complex temporal dynamics of treatment by unifying two control modalities: \textit{impulse control} for discrete, fast-acting interventions (e.g., insulin boluses), and \textit{switching control} for longer-acting treatments and regime shifts. The core of our method is a constrained Markov decision process augmented with physiological state features, enabling safe policy learning under clinical and resource constraints. The framework incorporates biologically realistic factors, including insulin decay, leading to policies that better reflect real-world therapeutic behaviour. While not intended for clinical deployment, this work establishes a foundation for future safe and temporally-aware RL in healthcare. We provide theoretical guarantees of convergence and demonstrate empirical improvements in a stylised T1DM control task, reducing blood glucose level violations from 22.4\% (state-of-the-art) to as low as 10.8\%.