Data-driven causal path discovery without prior knowledge - a benchmark study

TL;DR

This study evaluates the accuracy of various data-driven causal discovery methods on a cause-effect pairs dataset without prior knowledge, achieving up to 83% accuracy with combined techniques, and provides a benchmark for future research.

Contribution

It introduces a comprehensive benchmark framework for causal path discovery without prior knowledge, comparing multiple methods and combining them for improved accuracy.

Findings

Best accuracy of 80% with combined methods

Bootstrap simulation estimates correct causal paths

Accuracy improved to 83% on selected pairs

Abstract

Causal discovery broadens the inference possibilities, as correlation does not inform about the relationship direction. The common approaches were proposed for cases in which prior knowledge is desired, when the impact of a treatment/intervention variable is discovered or to analyze time-related dependencies. In some practical applications, more universal techniques are needed and have already been presented. Therefore, the aim of the study was to assess the accuracies in determining causal paths in a dataset without considering the ground truth and the contextual information. This benchmark was performed on the database with cause-effect pairs, using a framework consisting of generalized correlations (GC), kernel regression gradients (GR) and absolute residuals criteria (AR), along with causal additive modeling (CAM). The best overall accuracy, 80%, was achieved for the (majority voting) combination of GC, AR, and CAM, however, the most similar sensitivity and specificity values were obtained for AR. Bootstrap simulation established the probability of correct causal path determination (which pairs should remain indeterminate). The mean accuracy was then improved to 83% for the selected subset of pairs. The described approach can be used for preliminary dependence assessment, as an initial step for commonly used causality assessment frameworks or for comparison with prior assumptions.