Optimal treatment strategies for prioritized outcomes

TL;DR

This paper develops a method for estimating optimal treatment strategies when multiple outcomes of different priorities are involved, using inverse reinforcement learning to identify a composite outcome aligned with clinical preferences.

Contribution

It introduces a new definition of optimality for multi-outcome regimes and applies inverse reinforcement learning to find a composite outcome reflecting clinical priorities.

Findings

Proposed a novel optimality criterion for multi-outcome treatment regimes.

Demonstrated the approach's effectiveness through simulations and HIV/STI prevention data.

Showed that the method maximizes mean utility under a broad class of utility functions.

Abstract

Dynamic treatment regimes formalize precision medicine as a sequence of decision rules, one for each stage of clinical intervention, that map current patient information to a recommended intervention. Optimal regimes are typically defined as maximizing some functional of a scalar outcome's distribution, e.g., the distribution's mean or median. However, in many clinical applications, there are multiple outcomes of interest. We consider the problem of estimating an optimal regime when there are multiple outcomes that are ordered by priority but which cannot be readily combined by domain experts into a meaningful single scalar outcome. We propose a definition of optimality in this setting and show that an optimal regime with respect to this definition leads to maximal mean utility under a large class of utility functions. Furthermore, we use inverse reinforcement learning to identify a composite outcome that most closely aligns with our definition within a pre-specified class. Simulation experiments and an application to data from a sequential multiple assignment randomized trial (SMART) on HIV/STI prevention illustrate the usefulness of the proposed approach.