Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery

TL;DR

This paper critically evaluates neural causal discovery methods, revealing their fundamental limitations in accurately identifying causal structures due to issues like faithfulness violations, and suggests that a paradigm shift is necessary for progress.

Contribution

The paper systematically demonstrates the inherent limitations of neural networks in causal discovery, especially regarding faithfulness violations, highlighting the need for new approaches.

Findings

Neural networks struggle to distinguish causal relationships in finite samples.

Faithfulness violations are common and undermine neural causal discovery.

Current methods are fundamentally limited, requiring a paradigm shift.

Abstract

Neural causal discovery methods have recently improved in terms of scalability and computational efficiency. However, our systematic evaluation highlights significant room for improvement in their accuracy when uncovering causal structures. We identify a fundamental limitation: neural networks cannot reliably distinguish between existing and non-existing causal relationships in the finite sample regime. Our experiments reveal that neural networks, as used in contemporary causal discovery approaches, lack the precision needed to recover ground-truth graphs, even for small graphs and relatively large sample sizes. Furthermore, we identify the faithfulness property as a critical bottleneck: (i) it is likely to be violated across any reasonable dataset size range, and (ii) its violation directly undermines the performance of neural discovery methods. These findings lead us to conclude that progress within the current paradigm is fundamentally constrained, necessitating a paradigm shift in this domain.