Information mechanics: conservation and exchange

TL;DR

This paper introduces a foundational framework called information mechanics that reveals how Bayesian inference obeys conservation laws linking uncertainty reduction, information gain, and geometric properties, providing insights into inference and computation.

Contribution

It formalizes an invariant, algorithm-independent framework based on conservation laws in Bayesian updating, connecting information geometry with computational complexity.

Findings

Identifies two conservation relations: Shannon entropy and Fisher information.

Introduces the information potential $\

,

Abstract

Inference and learning are commonly cast in terms of optimisation, yet the fundamental constraints governing uncertainty reduction remain unclear. This work presents a first-principles framework inherent to Bayesian updating, termed information mechanics (infomechanics). Any pointwise reduction in posterior surprisal is exactly balanced by information gained from data, independently of algorithms, dynamics, or implementation. Imposing additivity, symmetry, and robustness collapses the freedom of this identity to only two independent conservation relations. One governs the global redistribution of uncertainty and recovers Shannon entropy. The other captures a complementary local geometric component, formalised as Fisher information. Together, these conserved quantities motivate a non-additive state function, the information potential $\Phi$, which isolates structural degrees of freedom beyond entropy while remaining invariant under reparametrisation. $\Phi$ quantifies local sharpness and ruggedness in posterior beliefs and vanishes uniquely for isotropic Gaussian distributions. In a low-temperature regime, $\Phi$ scales logarithmically with the effective number of local optima, linking information geometry to computational complexity. This formalises an information-computation exchange, whereby information acquisition reshapes the inference landscape and reduces computational demands. By separating invariant informational constraints from inference mechanisms, this framework provides a unified, algorithm-independent foundation for inference, learning, and computation across biological and artificial systems.