Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects

TL;DR

This paper introduces the H-learner, a hybrid meta-learner that adaptively combines direct and indirect approaches to estimate heterogeneous treatment effects more effectively across various scenarios.

Contribution

The H-learner is a novel regularization strategy that interpolates between existing meta-learner paradigms, improving CATE estimation by balancing bias and variance.

Findings

H-learner outperforms traditional meta-learners on benchmark datasets.

It operates effectively across diverse data scenarios, balancing bias and variance.

Experiments show consistent Pareto optimality in CATE estimation.

Abstract

Estimating conditional average treatment effects (CATE) from observational data involves modeling decisions that differ from supervised learning, particularly concerning how to regularize model complexity. Previous approaches can be grouped into two primary "meta-learner" paradigms that impose distinct inductive biases. Indirect meta-learners first fit and regularize separate potential outcome (PO) models and then estimate CATE by taking their difference, whereas direct meta-learners construct and directly regularize estimators for the CATE function itself. Neither approach consistently outperforms the other across all scenarios: indirect learners perform well when the PO functions are simple, while direct learners outperform when the CATE is simpler than individual PO functions. In this paper, we introduce the Hybrid Learner (H-learner), a novel regularization strategy that interpolates between the direct and indirect regularizations depending on the dataset at hand. The H-learner achieves this by learning intermediate functions whose difference closely approximates the CATE without necessarily requiring accurate individual approximations of the POs themselves. We demonstrate that intentionally allowing suboptimal fits to the POs improves the bias-variance tradeoff in estimating CATE. Experiments conducted on semi-synthetic and real-world benchmark datasets illustrate that the H-learner consistently operates at the Pareto frontier, effectively combining the strengths of both direct and indirect meta-learners.