Evaluation of Causal Structure Learning Algorithms via Risk Estimation

TL;DR

This paper introduces a decision-theoretic framework for evaluating the practical effectiveness of causal structure learning algorithms using risk estimation, bridging the gap between theoretical and empirical assessments.

Contribution

It formalizes causal risk and proposes data-driven sample measures, enabling practical assessment of causal learning methods under minimal assumptions.

Findings

Theoretical relationship between causal risk and sample estimates established.

Framework applicable across various real-world scenarios.

Simulation studies validate the effectiveness of the proposed assessment method.

Abstract

Recent years have seen many advances in methods for causal structure learning from data. The empirical assessment of such methods, however, is much less developed. Motivated by this gap, we pose the following question: how can one assess, in a given problem setting, the practical efficacy of one or more causal structure learning methods? We formalize the problem in a decision-theoretic framework, via a notion of expected loss or risk for the causal setting. We introduce a theoretical notion of causal risk as well as sample quantities that can be computed from data, and study the relationship between the two, both theoretically and through an extensive simulation study. Our results provide an assumptions-light framework for assessing causal structure learning methods that can be applied in a range of practical use-cases.