Obtaining Accurate Probabilistic Causal Inference by Post-Processing Calibration

TL;DR

This paper presents a new framework for deriving well-calibrated probabilities of causal relationships from observational data, enhancing the reliability of causal inferences in scientific research.

Contribution

It introduces a novel calibration framework combining approximate probability estimates, a small calibration set, and a shallow neural network calibration method.

Findings

Improves calibration of causal edge probabilities in simulated data.

Enhances precision and recall of causal predictions.

Demonstrates effectiveness of neural network calibration approach.

Abstract

Discovery of an accurate causal Bayesian network structure from observational data can be useful in many areas of science. Often the discoveries are made under uncertainty, which can be expressed as probabilities. To guide the use of such discoveries, including directing further investigation, it is important that those probabilities be well-calibrated. In this paper, we introduce a novel framework to derive calibrated probabilities of causal relationships from observational data. The framework consists of three components: (1) an approximate method for generating initial probability estimates of the edge types for each pair of variables, (2) the availability of a relatively small number of the causal relationships in the network for which the truth status is known, which we call a calibration training set, and (3) a calibration method for using the approximate probability estimates and the calibration training set to generate calibrated probabilities for the many remaining pairs of variables. We also introduce a new calibration method based on a shallow neural network. Our experiments on simulated data support that the proposed approach improves the calibration of causal edge predictions. The results also support that the approach often improves the precision and recall of predictions.