Structural Causal Bottleneck Models

TL;DR

This paper introduces structural causal bottleneck models (SCBMs), a new class of causal models that leverage low-dimensional summaries of high-dimensional causes for effective causal effect estimation and dimension reduction.

Contribution

The paper proposes SCBMs, analyzes their identifiability, connects them to information bottlenecks, and demonstrates their practical benefits in low-sample transfer learning.

Findings

SCBMs enable task-specific dimension reduction in causal models.

SCBMs are estimable with simple learning algorithms.

Bottlenecks improve effect estimation in low-sample transfer learning.

Abstract

We introduce structural causal bottleneck models (SCBMs), a novel class of structural causal models. At the core of SCBMs lies the assumption that causal effects between high-dimensional variables only depend on low-dimensional summary statistics, or bottlenecks, of the causes. SCBMs provide a flexible framework for task-specific dimension reduction while being estimable via standard, simple learning algorithms in practice. We analyse identifiability in SCBMs, connect them to information bottlenecks in the sense of Tishby & Zaslavsky (2015), and illustrate how to estimate them experimentally. We also demonstrate the benefit of bottlenecks for effect estimation in low-sample transfer learning settings. We argue that SCBMs provide an alternative to existing causal dimension reduction frameworks like causal representation learning or causal abstraction learning.