Recursive Partitioning for Personalization using Observational Data

TL;DR

This paper introduces a recursive partitioning method for personalized treatment selection from observational data, offering a unified, interpretable model that improves over traditional regress-and-compare approaches.

Contribution

It proposes a novel recursive partitioning framework for personalized treatment choice, extending to a globally optimal partitioning method for better interpretability and effectiveness.

Findings

Effective in personalized medicine applications

Improves treatment selection accuracy

Provides new validation tools for observational data

Abstract

We study the problem of learning to choose from m discrete treatment options (e.g., news item or medical drug) the one with best causal effect for a particular instance (e.g., user or patient) where the training data consists of passive observations of covariates, treatment, and the outcome of the treatment. The standard approach to this problem is regress and compare: split the training data by treatment, fit a regression model in each split, and, for a new instance, predict all m outcomes and pick the best. By reformulating the problem as a single learning task rather than m separate ones, we propose a new approach based on recursively partitioning the data into regimes where different treatments are optimal. We extend this approach to an optimal partitioning approach that finds a globally optimal partition, achieving a compact, interpretable, and impactful personalization model. We develop new tools for validating and evaluating personalization models on observational data and use these to demonstrate the power of our novel approaches in a personalized medicine and a job training application.