When the map is better than the territory

TL;DR

This paper explores how macro-level causal models can sometimes be more informative than detailed micro-level models by leveraging information theory concepts like channel capacity, revealing limits of detailed modeling.

Contribution

It introduces a framework linking causal emergence to Shannon's channel capacity, explaining why macro models can outperform detailed models in capturing causal structure.

Findings

Macroscale models can utilize full causal capacity.

Causal emergence explained via information theory principles.

Model choice affects the utilization of a system's causal capacity.

Abstract

The causal structure of any system can be analyzed at a multitude of spatial and temporal scales. It has long been thought that while higher scale (macro) descriptions of causal structure may be useful to observers, they are at best a compressed description and at worse leave out critical information. However, recent research applying information theory to causal analysis has shown that the causal structure of some systems can actually come into focus (be more informative) at a macroscale (Hoel et al. 2013). That is, a macro model of a system (a map) can be more informative than a fully detailed model of the system (the territory). This has been called causal emergence. While causal emergence may at first glance seem counterintuitive, this paper grounds the phenomenon in a classic concept from information theory: Shannon's discovery of the channel capacity. I argue that systems have a particular causal capacity, and that different causal models of those systems take advantage of that capacity to various degrees. For some systems, only macroscale causal models use the full causal capacity. Such macroscale causal models can either be coarse-grains, or may leave variables and states out of the model (exogenous) in various ways, which can improve the model's efficacy and its informativeness via the same mathematical principles of how error-correcting codes take advantage of an information channel's capacity. As model choice increase, the causal capacity of a system approaches the channel capacity. Ultimately, this provides a general framework for understanding how the causal structure of some systems cannot be fully captured by even the most detailed microscopic model.