Bayesian ICA for Causal Discovery

TL;DR

This paper introduces a Bayesian ICA framework for causal discovery that effectively handles confounding by quantifying multivariate mutual information, extending classical LiNGAM methods to more realistic scenarios.

Contribution

It proposes a novel information-theoretic approach that estimates causal order under confounding using Bayesian mutual information, unifying and extending existing ICA-based methods.

Findings

Consistent Bayesian mutual information estimator with $O(\log n)$ redundancy.

Recovers classical LiNGAM in the absence of confounding.

Provides a principled ranking of causal orders with confounding.

Abstract

Causal discovery based on Independent Component Analysis (ICA) has achieved remarkable success through the LiNGAM framework, which exploits non-Gaussianity and independence of noise variables to identify causal order. However, classical LiNGAM methods rely on the strong assumption that there exists an ordering under which the noise terms are exactly independent, an assumption that is often violated in the presence of confounding. In this paper, we propose a general information-theoretic framework for causal order estimation that remains applicable under arbitrary confounding. Rather than imposing independence as a hard constraint, we quantify the degree of confounding by the multivariate mutual information among the noise variables. This quantity is decomposed into a sum of mutual information terms along a causal order and is estimated using Bayesian marginal likelihoods. The resulting criterion can be interpreted as Bayesian ICA for causal discovery, where causal order selection is formulated as a model selection problem over permutations. Under standard regularity conditions, we show that the proposed Bayesian mutual information estimator is consistent, with redundancy of order $O(\log n)$. To avoid non-identifiability caused by Gaussian noise, we employ non-Gaussian predictive models, including multivariate $t$ distributions, whose marginal likelihoods can be evaluated via MCMC. The proposed method recovers classical LiNGAM and DirectLiNGAM as limiting cases in the absence of confounding, while providing a principled ranking of causal orders when confounding is present. This establishes a unified, confounding-aware, and information-theoretically grounded extension of ICA-based causal discovery.