Inferring latent structures via information inequalities

TL;DR

This paper introduces an information-theoretic method using entropy inequalities to infer latent structures in Bayesian networks, enabling detection of causal relationships and hidden variables.

Contribution

It presents a novel algorithm for deriving entropic tests for latent structures, extending beyond traditional conditional independence tests.

Findings

Effective detection of common ancestors

Quantification of causal influence strength

Inference of causation direction from marginals

Abstract

One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian networks with hidden variables give rise to highly non-trivial constraints on the observed distribution. Here, we propose an information-theoretic approach, based on the insight that conditions on entropies of Bayesian networks take the form of simple linear inequalities. We describe an algorithm for deriving entropic tests for latent structures. The well-known conditional independence tests appear as a special case. While the approach applies for generic Bayesian networks, we presently adopt the causal view, and show the versatility of the framework by treating several relevant problems from that domain: detecting common ancestors, quantifying the strength of causal influence, and inferring the direction of causation from two-variable marginals.