Causal Transfer Random Forest: Combining Logged Data and Randomized Experiments for Robust Prediction

TL;DR

This paper introduces a causal transfer random forest (CTRF) that leverages logged data and randomized experiments to create prediction models resilient to distributional shifts, with theoretical justification and empirical validation.

Contribution

The paper proposes a novel CTRF method that combines data sources for robust prediction under feature shifts, grounded in causal learning theory.

Findings

CTRF outperforms baseline methods under feature shifts

Empirical validation on synthetic and real-world data

Theoretically justified robustness to distributional changes

Abstract

It is often critical for prediction models to be robust to distributional shifts between training and testing data. From a causal perspective, the challenge is to distinguish the stable causal relationships from the unstable spurious correlations across shifts. We describe a causal transfer random forest (CTRF) that combines existing training data with a small amount of data from a randomized experiment to train a model which is robust to the feature shifts and therefore transfers to a new targeting distribution. Theoretically, we justify the robustness of the approach against feature shifts with the knowledge from causal learning. Empirically, we evaluate the CTRF using both synthetic data experiments and real-world experiments in the Bing Ads platform, including a click prediction task and in the context of an end-to-end counterfactual optimization system. The proposed CTRF produces robust predictions and outperforms most baseline methods compared in the presence of feature shifts.