Thermodynamic Irreversibility of Training Algorithms

TL;DR

This paper develops a comprehensive framework to analyze the irreversibility of training algorithms in AI, linking thermodynamic concepts with learning dynamics.

Contribution

It introduces a unified approach to characterize irreversibility in training algorithms, connecting four different measures and revealing their equivalence.

Findings

Irreversibility characterized by four equivalent measures to leading order in step size

Irreversibility induces a time-reversal-symmetry-breaking force in learning dynamics

Preference for trajectories minimizing entropy production rate

Abstract

The training algorithms for AI systems all introduce far-from-equilibrium dynamical processes, and understanding the irreversibility of these algorithms is a fundamental step towards understanding the learning dynamics of modern AI systems. In this work, we establish a general framework for defining and analyzing the irreversibility of training algorithms. We show that four different ways to characterize the irreversibility of dynamical processes are equivalent to leading order in the step size $\eta$: numerical backward error $\phi_{\rm DE}$, time-renormalized correction $\phi_{\rm TR}$, microscopic time reversal asymmetry $\phi_{\rm TA}$, and the (regularized) stochastic-thermodynamic entropy production $\phi_{\rm ST}$. The irreversibility gives rise to a time-reversal-symmetry-breaking emergent force that generically breaks non-isometric continuous reparametrization symmetries, preserves orthogonal symmetries, and leads to a universal preference for those learning trajectories that minimize the entropy production rate.