The Price equation reveals a universal force-metric-bias law of algorithmic learning and natural selection

TL;DR

This paper introduces a universal law derived from the Price equation that unifies various learning algorithms and natural selection processes through a common mathematical framework involving force, metric, bias, and noise.

Contribution

The paper reveals a universal force-metric-bias law of learning and natural selection, unifying diverse algorithms under a single mathematical structure using the Price equation.

Findings

Unifies natural selection, Bayesian updating, and optimization algorithms.

Identifies Fisher information, KL divergence, and d'Alembert's principle as natural in learning.

Provides a framework for understanding and designing algorithms across disciplines.

Abstract

Diverse learning algorithms, optimization methods, and natural selection share a common mathematical structure, despite their apparent differences. Here I show that a simple notational partitioning of change by the Price equation reveals a universal force-metric-bias (FMB) law: $\Delta\mathbf{\theta} = \mathbf{M}\,\mathbf{f} + \mathbf{b} + \mathbf{\xi}$. The force $\mathbf{f}$ drives improvement in parameters, $\Delta\mathbf{\theta}$, in proportion to the slope of performance with respect to the parameters. The metric $\mathbf{M}$ rescales movement by inverse curvature. The bias $\mathbf{b}$ adds momentum or changes in the frame of reference. The noise $\mathbf{\xi}$ enables exploration. This framework unifies natural selection, Bayesian updating, Newton's method, stochastic gradient descent, stochastic Langevin dynamics, Adam optimization, and most other algorithms as special cases of the same underlying process. The Price equation also reveals why Fisher information, Kullback-Leibler divergence, and d'Alembert's principle arise naturally in learning dynamics. By exposing this common structure, the FMB law provides a principled foundation for understanding, comparing, and designing learning algorithms across disciplines.