Error Asymmetry in Causal and Anticausal Regression

TL;DR

This paper establishes a theoretical and empirical asymmetry in prediction errors depending on whether cause or effect is predicted, based on causal structure assumptions, with implications for causal inference.

Contribution

It introduces a novel theorem linking causal direction to expected prediction error, highlighting an asymmetry in causal versus anticausal regression.

Findings

Expected error is smaller when predicting effect from cause.

Expected error is greater when predicting cause from effect.

Empirical results support the theoretical asymmetry in real-world data.

Abstract

It is generally difficult to make any statements about the expected prediction error in an univariate setting without further knowledge about how the data were generated. Recent work showed that knowledge about the real underlying causal structure of a data generation process has implications for various machine learning settings. Assuming an additive noise and an independence between data generating mechanism and its input, we draw a novel connection between the intrinsic causal relationship of two variables and the expected prediction error. We formulate the theorem that the expected error of the true data generating function as prediction model is generally smaller when the effect is predicted from its cause and, on the contrary, greater when the cause is predicted from its effect. The theorem implies an asymmetry in the error depending on the prediction direction. This is further corroborated with empirical evaluations in artificial and real-world data sets.