Reliable Off-Policy Learning for Dosage Combinations

TL;DR

This paper introduces a novel, reliable off-policy learning method for personalized dosage combinations in medicine, addressing joint effects of multiple treatments with neural networks and propensity score estimation.

Contribution

It presents the first method for reliable off-policy learning of dosage combinations, incorporating joint effect modeling, overlap detection, and gradient-based optimization.

Findings

Effective estimation of joint dose-response functions.

Reliable identification of treatment regions with sufficient data.

Improved policy learning for personalized medicine.

Abstract

Decision-making in personalized medicine such as cancer therapy or critical care must often make choices for dosage combinations, i.e., multiple continuous treatments. Existing work for this task has modeled the effect of multiple treatments independently, while estimating the joint effect has received little attention but comes with non-trivial challenges. In this paper, we propose a novel method for reliable off-policy learning for dosage combinations. Our method proceeds along three steps: (1) We develop a tailored neural network that estimates the individualized dose-response function while accounting for the joint effect of multiple dependent dosages. (2) We estimate the generalized propensity score using conditional normalizing flows in order to detect regions with limited overlap in the shared covariate-treatment space. (3) We present a gradient-based learning algorithm to find the optimal, individualized dosage combinations. Here, we ensure reliable estimation of the policy value by avoiding regions with limited overlap. We finally perform an extensive evaluation of our method to show its effectiveness. To the best of our knowledge, ours is the first work to provide a method for reliable off-policy learning for optimal dosage combinations.