$Q$- and $A$-Learning Methods for Estimating Optimal Dynamic Treatment Regimes

TL;DR

This paper reviews Q- and A-learning methods for estimating optimal dynamic treatment regimes, highlighting their performance and illustrating their application with depression study data.

Contribution

It provides a detailed comparison of Q- and A-learning approaches and demonstrates their practical use in clinical decision-making.

Findings

Q- and A-learning effectively estimate optimal treatment regimes

Performance varies depending on data and context

Application to depression data illustrates real-world utility

Abstract

In clinical practice, physicians make a series of treatment decisions over the course of a patient's disease based on his/her baseline and evolving characteristics. A dynamic treatment regime is a set of sequential decision rules that operationalizes this process. Each rule corresponds to a decision point and dictates the next treatment action based on the accrued information. Using existing data, a key goal is estimating the optimal regime, that, if followed by the patient population, would yield the most favorable outcome on average. Q- and A-learning are two main approaches for this purpose. We provide a detailed account of these methods, study their performance, and illustrate them using data from a depression study.