Towards a Learning Theory of Cause-Effect Inference

TL;DR

This paper introduces a novel approach to causal inference by classifying probability distributions using kernel mean embeddings and machine learning, achieving state-of-the-art results and extending to multiple variables.

Contribution

It proposes a new method for cause-effect inference based on distribution classification with theoretical guarantees and practical implementation.

Findings

Achieves state-of-the-art cause-effect inference performance.

Provides generalization bounds and statistical consistency.

Extends methodology to multiple variables.

Abstract

We pose causal inference as the problem of learning to classify probability distributions. In particular, we assume access to a collection $\{(S_i,l_i)\}_{i=1}^n$, where each $S_i$ is a sample drawn from the probability distribution of $X_i \times Y_i$, and $l_i$ is a binary label indicating whether "$X_i \to Y_i$" or "$X_i \leftarrow Y_i$". Given these data, we build a causal inference rule in two steps. First, we featurize each $S_i$ using the kernel mean embedding associated with some characteristic kernel. Second, we train a binary classifier on such embeddings to distinguish between causal directions. We present generalization bounds showing the statistical consistency and learning rates of the proposed approach, and provide a simple implementation that achieves state-of-the-art cause-effect inference. Furthermore, we extend our ideas to infer causal relationships between more than two variables.