Cross-validating causal discovery via Leave-One-Variable-Out

TL;DR

This paper introduces a novel validation method for causal discovery algorithms called Leave-One-Variable-Out (LOVO), which tests causal models by predicting omitted variables and assessing errors without ground truth.

Contribution

The paper presents the LOVO approach for falsifying causal discovery models, applicable to various algorithms, and demonstrates its effectiveness through simulations.

Findings

LOVO prediction error correlates with causal model accuracy

The method can falsify causal models without ground truth

Applicable to general causal discovery algorithms

Abstract

We propose a new approach to falsify causal discovery algorithms without ground truth, which is based on testing the causal model on a pair of variables that has been dropped when learning the causal model. To this end, we use the "Leave-One-Variable-Out (LOVO)" prediction where $Y$ is inferred from $X$ without any joint observations of $X$ and $Y$, given only training data from $X,Z_1,\dots,Z_k$ and from $Z_1,\dots,Z_k,Y$. We demonstrate that causal models on the two subsets, in the form of Acyclic Directed Mixed Graphs (ADMGs), often entail conclusions on the dependencies between $X$ and $Y$, enabling this type of prediction. The prediction error can then be estimated since the joint distribution $P(X, Y)$ is assumed to be available, and $X$ and $Y$ have only been omitted for the purpose of falsification. After presenting this graphical method, which is applicable to general causal discovery algorithms, we illustrate how to construct a LOVO predictor tailored towards algorithms relying on specific a priori assumptions, such as linear additive noise models. Simulations indicate that the LOVO prediction error is indeed correlated with the accuracy of the causal outputs, affirming the method's effectiveness.