Structural Drift: The Population Dynamics of Sequential Learning

TL;DR

This paper develops a theoretical framework for understanding how sequential learners update and pass models downstream, revealing how population dynamics influence learning fidelity, innovation, and information loss.

Contribution

It introduces a generalized drift process model for sequential causal inference, extending genetic drift theory to structured populations with memory.

Findings

Analyzes diffusion and fixation in drift processes

Shows how drift space organization affects learning fidelity

Demonstrates applications to evolution and inference

Abstract

We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream teacher and then pass samples from the model to their downstream student. It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory. We examine the diffusion and fixation properties of several drift processes and propose applications to learning, inference, and evolution. We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.