Counterfactual Predictions under Runtime Confounding

TL;DR

This paper addresses counterfactual prediction under runtime confounding, proposing a doubly-robust method that effectively handles situations where some relevant factors cannot be used in the model, supported by theoretical and experimental validation.

Contribution

It introduces a novel doubly-robust approach for counterfactual prediction in runtime confounding settings where some factors are excluded from the model.

Findings

The proposed method often outperforms existing approaches.

Theoretical analysis confirms robustness of the method.

A validation procedure for counterfactual prediction models is presented.

Abstract

Algorithms are commonly used to predict outcomes under a particular decision or intervention, such as predicting whether an offender will succeed on parole if placed under minimal supervision. Generally, to learn such counterfactual prediction models from observational data on historical decisions and corresponding outcomes, one must measure all factors that jointly affect the outcomes and the decision taken. Motivated by decision support applications, we study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data, but it is either undesirable or impermissible to use some such factors in the prediction model. We refer to this setting as runtime confounding. We propose a doubly-robust procedure for learning counterfactual prediction models in this setting. Our theoretical analysis and experimental results suggest that our method often outperforms competing approaches. We also present a validation procedure for evaluating the performance of counterfactual prediction methods.