Forest Kernel Balancing Weights: Outcome-Guided Features for Causal Inference

TL;DR

This paper introduces forest kernel balancing, a novel method that uses tree-based models to implicitly learn outcome-relevant features for covariate balancing in causal inference, improving over standard kernel methods.

Contribution

It proposes leveraging tree-based models to implicitly estimate kernels that focus on outcome-relevant features, enhancing covariate balancing in observational studies.

Findings

Improved covariate balance in simulations and real data.

Enhanced causal effect estimation accuracy.

Outperforms standard kernel balancing methods.

Abstract

While balancing covariates between groups is central for observational causal inference, selecting which features to balance remains a challenging problem. Kernel balancing is a promising approach that first estimates a kernel that captures similarity across units and then balances a (possibly low-dimensional) summary of that kernel, indirectly learning important features to balance. In this paper, we propose forest kernel balancing, which leverages the underappreciated fact that tree-based machine learning models, namely random forests and Bayesian additive regression trees (BART), implicitly estimate a kernel based on the co-occurrence of observations in the same terminal leaf node. Thus, even though the resulting kernel is solely a function of baseline features, the selected nonlinearities and other interactions are important for predicting the outcome -- and therefore are important for addressing confounding. Through simulations and applied illustrations, we show that forest kernel balancing leads to meaningful computational and statistical improvement relative to standard kernel methods, which do not incorporate outcome information when learning features.