Information-theoretic signatures of causality in Bayesian networks and hypergraphs

TL;DR

This paper establishes a theoretical link between higher-order information theory, specifically Partial Information Decomposition, and causal structures in Bayesian networks and hypergraphs, enabling local causal discovery.

Contribution

It introduces the first formal connection between PID components and causal structures, allowing local causal inference without global graph search.

Findings

Unique information characterizes direct causal neighbors.

Synergy identifies collider relationships.

PID signatures differentiate various hypergraph causal roles.

Abstract

Analyzing causality in multivariate systems involves establishing how information is generated, distributed and combined. Traditional causal discovery frameworks are capable of multivariate reasoning but their intrinsic pairwise graph topology restricts them to do so only indirectly by integrating multivariate information across pairwise edges. Higher-order information theory provides direct tools that can explicitly model higher-order interactions. In particular, Partial Information Decomposition (PID) allows the decomposition of the information that a set of sources provides about a target into redundant, unique, and synergistic components. Yet the mathematical connection between such higher-order information-theoretic measures and causal structure remains undeveloped. Here we establish the first theoretical correspondence between PID components and causal structure in both Bayesian networks and hypergraphs. We first show that in Bayesian networks unique information precisely characterizes direct causal neighbors, while synergy identifies collider relationships. This establishes a localist causal discovery paradigm in which the structure surrounding each variable can be recovered from its immediate informational footprint, eliminating the need for global search over graph space. Extending these results to more expressive causal representation, we prove that PID signatures in Bayesian hypergraphs differentiate parents, children, co-heads, and co-tails, revealing a novel collider effect unique to multi-tail hyperedges. Our results position PID as a rigorous, model-agnostic foundation for inferring both pairwise and higher-order causal structure, and introduce a fundamentally local information-theoretic viewpoint on causal discovery.