Interpretable, multi-dimensional Evaluation Framework for Causal Discovery from observational i.i.d. Data

TL;DR

This paper introduces a six-dimensional, interpretable evaluation framework for nonlinear causal discovery methods, assessing their performance under assumption violations and across diverse causal patterns.

Contribution

It presents the first unified, interpretable evaluation metric tailored for causal discovery and evaluates multiple algorithms on complex, non-identifiable nonlinear causal data.

Findings

Amortized causal discovery performs well on non-identifiable patterns.

The DOS metric effectively quantifies structural similarity and causal inference capacity.

Evaluation across seven algorithm families reveals strengths and weaknesses in different scenarios.

Abstract

Nonlinear causal discovery from observational data imposes strict identifiability assumptions on the formulation of structural equations utilized in the data generating process. The evaluation of structure learning methods under assumption violations requires a rigorous and interpretable approach, which quantifies both the structural similarity of the estimation with the ground truth and the capacity of the discovered graphs to be used for causal inference. Motivated by the lack of unified performance assessment framework, we introduce an interpretable, six-dimensional evaluation metric, i.e., distance to optimal solution (DOS), which is specifically tailored to the field of causal discovery. Furthermore, this is the first research to assess the performance of structure learning algorithms from seven different families on increasing percentage of non-identifiable, nonlinear causal patterns, inspired by real-world processes. Our large-scale simulation study, which incorporates seven experimental factors, shows that besides causal order-based methods, amortized causal discovery delivers results with comparatively high proximity to the optimal solution.