The relative value of interventional and observational samples in Bayesian Causal Linear Gaussian Models

TL;DR

This paper analyzes how combining observational and interventional data affects the accuracy of Bayesian causal discovery in Gaussian SEMs, revealing a phase transition in convergence rates.

Contribution

It provides a theoretical characterization of the asymptotic behavior of Bayesian causal discovery with mixed data, highlighting the exponential benefits of interventional samples.

Findings

Posterior fails to identify true causal structure with observational data alone.

Incorporating interventional data leads to exponential convergence rates for connected graphs.

Explicit formulas for convergence rates as a function of observational and interventional sample ratio.

Abstract

We investigate the asymptotic properties of Bayesian bivariate causal discovery for Gaussian Linear Structural Equation Models (SEMs) with heteroscedastic noise. We demonstrate that with purely observational data, the posterior distribution over the models fails to consistently identify the true causal structure - a consequence of the fundamental non-identifiability within the Markov Equivalence Class. Specifically, if the true generating mechanism corresponds to a connected graph (A -> B or B -> A), the asymptotic behavior of the posterior is given by the ratio between the prior on the true model and the push-forward prior of the alternative. In contrast, for the independence model, we establish that the posterior concentrates at a stochastic polynomial rate of O_p(n^{-1/2}). To resolve this non-identifiability, we incorporate m interventional samples and characterize the concentration rates as a function of the observational-to-total sample ratio, \eta. We identify a sharp concentration dichotomy: while the independence graph maintains a polynomial O_p(N^{-1/2}) rate (where N = n+m), connected graphs undergo a phase transition to exponentially fast convergence. This highlights an exponential relative importance between the two data types, as altering the amount of one data type directly changes the exponent governing the concentration speed. We derive explicit formulae for the exponential decay rates and provide precise conditions under which mixing observational and interventional data optimizes concentration speed. Finally, our theoretical findings are validated through empirical simulations in Bayesian Gaussian equivalent (BGe)-style prior specifications offering a principled foundation for experimental design in Bayesian causal discovery.