Evaluation of Induced Expert Knowledge in Causal Structure Learning by NOTEARS

TL;DR

This paper investigates how incorporating expert knowledge as constraints influences the performance of the NOTEARS causal discovery method, highlighting when and how such knowledge improves causal graph accuracy.

Contribution

It provides a comprehensive analysis of the effects of different types of expert constraints on the NOTEARS causal discovery model, revealing key insights into their benefits and limitations.

Findings

Knowledge correcting model mistakes improves accuracy.

Constraints on active edges have a larger positive impact.

Induced knowledge does not significantly reduce incorrect edges on average.

Abstract

Causal modeling provides us with powerful counterfactual reasoning and interventional mechanism to generate predictions and reason under various what-if scenarios. However, causal discovery using observation data remains a nontrivial task due to unobserved confounding factors, finite sampling, and changes in the data distribution. These can lead to spurious cause-effect relationships. To mitigate these challenges in practice, researchers augment causal learning with known causal relations. The goal of the paper is to study the impact of expert knowledge on causal relations in the form of additional constraints used in the formulation of the nonparametric NOTEARS. We provide a comprehensive set of comparative analyses of biasing the model using different types of knowledge. We found that (i) knowledge that corrects the mistakes of the NOTEARS model can lead to statistically significant improvements, (ii) constraints on active edges have a larger positive impact on causal discovery than inactive edges, and surprisingly, (iii) the induced knowledge does not correct on average more incorrect active and/or inactive edges than expected. We also demonstrate the behavior of the model and the effectiveness of domain knowledge on a real-world dataset.