Computational Inertia as a Conserved Quantity in Frictionless and Damped Learning Dynamics

TL;DR

This paper introduces the concept of computational inertia as a conserved quantity in idealized continuous-time learning dynamics, providing insights into the behavior of training trajectories and potential tools for analyzing convergence and stability.

Contribution

It formalizes the conservation law of computational inertia and explores its decay under damping and stochastic effects, offering a new perspective on learning dynamics.

Findings

Computational inertia remains invariant in frictionless, idealized training.

Damping and stochastic perturbations cause decay of the conserved quantity.

The invariant helps interpret learning trajectories and analyze convergence.

Abstract

We identify a conserved quantity in continuous-time optimization dynamics, termed computational inertia. Defined as the sum of kinetic energy (parameter velocity) and potential energy (loss), this scalar remains invariant under idealized, frictionless training. We formalize this conservation law, derive its analytic decay under damping and stochastic perturbations, and demonstrate its behavior in a synthetic system. The invariant offers a compact lens for interpreting learning trajectories, and may inform theoretical tools for analyzing convergence, stability, and training geometry.