Cumulative Treatment Effect Testing under Continuous Time Reinforcement Learning

TL;DR

This paper introduces a continuous-time reinforcement learning method for testing cumulative treatment effects over time, improving accuracy and handling irregular observations in medical and scientific studies.

Contribution

It presents a novel continuous-time testing approach for treatment effects, outperforming discrete methods and capturing short-term effects with higher precision.

Findings

Outperforms existing discrete-time tests in irregular observation settings.

Effectively captures short-term and cumulative treatment effects.

Demonstrated on diabetes data to evaluate insulin impact.

Abstract

Understanding the impact of treatment effect over time is a fundamental aspect of many scientific and medical studies. In this paper, we introduce a novel approach under a continuous-time reinforcement learning framework for testing a treatment effect. Specifically, our method provides an effective test on carryover effects of treatment over time utilizing the average treatment effect (ATE). The average treatment effect is defined as difference of value functions over an infinite horizon, which accounts for cumulative treatment effects, both immediate and carryover. The proposed method outperforms existing testing procedures such as discrete time reinforcement learning strategies in multi-resolution observation settings where observation times can be irregular. Another advantage of the proposed method is that it can capture treatment effects of a shorter duration and provide greater accuracy compared to discrete-time approximations, through the use of continuous-time estimation for the value function. We establish the asymptotic normality of the proposed test statistics and apply it to OhioT1DM diabetes data to evaluate the cumulative treatment effects of bolus insulin on patients' glucose levels.