Dissipative Learning: A Framework for Viable Adaptive Systems

TL;DR

This paper introduces a dissipative framework for learning, emphasizing the importance of regularization and forgetting as structural elements, and models learning as the evolution of belief states under thermodynamic constraints.

Contribution

It presents the BEDS framework, unifies existing regularization methods, and establishes Fisher-Rao regularization as thermodynamically optimal for adaptive systems.

Findings

Fisher-Rao regularization minimizes dissipation.

Euclidean regularization is suboptimal thermodynamically.

The framework unifies various regularization techniques.

Abstract

We propose a perspective in which learning is an intrinsically dissipative process. Forgetting and regularization are not heuristic add-ons but structural requirements for adaptive systems. Drawing on information theory, thermodynamics, and information geometry, we introduce the BEDS (Bayesian Emergent Dissipative Structures) framework, modeling learning as the evolution of compressed belief states under dissipation constraints. A central contribution is the Conditional Optimality Theorem, showing that Fisher-Rao regularization measuring change via information divergence rather than Euclidean distance is the unique thermodynamically optimal regularization strategy, achieving minimal dissipation. Euclidean regularization is shown to be structurally suboptimal. The framework unifies existing methods (Ridge, SIGReg, EMA, SAC) as special cases of a single governing equation. Within this view, overfitting corresponds to over-crystallization, while catastrophic forgetting reflects insufficient dissipation control. The framework distinguishes BEDS-crystallizable problems, where beliefs converge to stable equilibria, from BEDS-maintainable problems, which require continual adaptation. It extends naturally to continual and multi-agent systems, where viability, stability under adaptation and finite resources replaces asymptotic optimality as the primary criterion. Overall, this work reframes learning as maintaining viable belief states under dissipation constraints, providing a principled lens on forgetting, regularization, and stability.