Algorithmic causal structure emerging through compression

TL;DR

This paper investigates how causality and symmetry can emerge from data compression across different environments, proposing a new framework where causality arises from minimizing Kolmogorov complexity without relying on traditional assumptions.

Contribution

It introduces the concept of algorithmic causality, generalizing causal inference to settings where causal models are not identifiable, based on data compression principles.

Findings

Causality can emerge from data compression across multiple environments.

Algorithmic causality does not require intervention targets.

Minimizing Kolmogorov complexity can reveal causal and symmetric structures.

Abstract

We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.