The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy

TL;DR

This paper quantifies the informational gap between different levels of causal reasoning in Pearl's hierarchy using Kolmogorov complexity, revealing significant separations especially between observational and interventional queries.

Contribution

It formalizes the description length gap across Pearl's causal hierarchy, providing explicit constructions and bounds that quantify the informational complexity differences.

Findings

Binary acyclic SCMs with constant observational complexity but quadratic interventional complexity.

Upper bounds on the observational-interventional gap depending on SCM indegree and size.

The quadratic gap persists even under approximate descriptions with fixed error.

Abstract

Pearl's causal hierarchy shows that observational, interventional, and counterfactual queries are qualitatively distinct. We ask a quantitative version of this question: how many additional bits are needed to specify higher-rung causal answers once lower-rung answers are known? We formalize this via query-class description length, the Kolmogorov complexity of the answer oracle induced by an SCM for a class of queries. Our main construction gives binary acyclic SCMs whose observational distribution has constant description length, while the single-variable interventional answer oracle has description length $\Theta(n^2)$. A degree-sensitive upper bound shows that finite-gate-schema SCMs of indegree $d$ have observational-interventional gap at most $O(nd \log(en/d) + n \log n)$, making the quadratic construction order-optimal in the dense regime and a rooted-tree construction order-optimal for bounded indegree. The quadratic separation persists under $\varepsilon$-accurate total-variation descriptions for every fixed $\varepsilon < 1/4$. At the next rung, the full hard-do interventional oracle can still leave a $\Theta(n)$ counterfactual description gap. A general ambiguity-to-bits theorem and Shannon analogue show that these gaps equal the logarithm of residual higher-rung ambiguity up to lower-order terms.