Optimizing Treatment Allocation to Maximize the Health of a Population

TL;DR

This paper introduces a novel population-level treatment allocation method using Measurized MDPs, optimizing long-term health outcomes while respecting capacity and ethical constraints, demonstrated with CMS data.

Contribution

It develops a scalable, index-based policy for population health management that accounts for long-term effects and ethical considerations, advancing beyond traditional myopic approaches.

Findings

Policy achieves significant health outcome improvements over benchmarks.

Over 1,500 additional home days annually per 1,000 patients at longest horizon.

Method remains flexible and can incorporate machine learning models.

Abstract

Recent shifts in global health priorities have positioned Population Health Management (PHM) as a central area of focus. However, optimizing PHM strategies presents several challenges: managing high-dimensional patient covariates, tracking their evolution and long-term response to interventions, and accounting for the inflow and outflow of individuals within the population. In this paper, we propose a novel approach based on Measurized MDPs that integrates these components. We consider a setting in which a treatment with population-level benefits is available but scarce, and model an MDP that optimizes the long-term distribution of the healthcare population under expected capacity constraints. This formulation allows us to bypass both the dimensionality and practical challenges of handling and tracking individual patient covariates across the population. To ensure ethical compliance, we introduce a non-maleficence constraint that limits the allowable mortality rate. To solve the resulting infinite-dimensional problem, we use ADP and reduce the task to identifying a finite set of high-performing treated and untreated patients. Despite the complexity of the underlying structure, our approach yields a simple, clinically implementable index policy: a patient is selected for treatment if their adjusted impactability exceeds a specified threshold. The adjusted impactability captures the long-term consequences of receiving or not receiving treatment. While straightforward to apply, the policy remains flexible and can incorporate general machine learning models. Using CMS data, we show that our policy yields a statistically significant improvement over a myopic benchmark. This advantage increases with the time horizon, consistent with the forward-looking nature of our policy. At the longest horizon tested, this corresponds to over 1,500 additional home days annually per 1,000 patients.