Statistical Decision Theory with Counterfactual Loss

TL;DR

This paper extends classical statistical decision theory by incorporating counterfactual losses, enabling evaluation of decisions considering all potential outcomes, with implications for treatment policies and decision accuracy.

Contribution

It introduces a counterfactual decision framework with an identification condition based on additive losses, and provides a symbolic method to assess risk identifiability without data.

Findings

Counterfactual risk is identifiable under strong ignorability if the loss is additive.

Additive counterfactual losses can lead to different treatment recommendations than standard losses.

A linear inverse program determines risk identifiability for given counterfactual losses.

Abstract

Many researchers have applied classical statistical decision theory to evaluate treatment choices and learn optimal policies. However, because this framework is based solely on realized outcomes under chosen decisions and ignores counterfactual outcomes, it cannot assess the quality of a decision relative to feasible alternatives. For example, in bail decisions, a judge must consider not only crime prevention but also the avoidance of unnecessary burdens on arrestees. To address this limitation, we generalize standard decision theory by incorporating counterfactual losses, allowing decisions to be evaluated using all potential outcomes. The central challenge in this counterfactual statistical decision framework is identification: since only one potential outcome is observed for each unit, the associated counterfactual risk is generally not identifiable. We prove that, under the assumption of strong ignorability, the counterfactual risk is identifiable if and only if the counterfactual loss function is additive in the potential outcomes. Moreover, we demonstrate that additive counterfactual losses can yield treatment recommendations, which differ from those based on standard loss functions when the decision problem involves more than two treatment options. One interpretation of this result is that additive counterfactual losses can capture the accuracy and difficulty of a decision, whereas standard losses account for accuracy alone. Finally, we formulate a symbolic linear inverse program that, given a counterfactual loss, determines whether its risk is identifiable, without requiring data.