# Nonequilibrium Thermodynamics of Restricted Boltzmann Machines

## TL;DR

This paper explores the nonequilibrium thermodynamics of Restricted Boltzmann Machines, linking their learning process to thermodynamic principles and fluctuation theorems, and providing a new physical perspective on unsupervised learning.

## Contribution

It introduces a thermodynamic framework for RBMs, connecting their training dynamics to nonequilibrium thermodynamics and fluctuation theorems, and interprets learning as a work protocol.

## Key findings

- Verification of heat exchange fluctuation theorem in RBMs
- Contrastive divergence relates to thermodynamic quantities
- Thermodynamic interpretation of partition function estimation

## Abstract

In this work, we analyze the nonequilibrium thermodynamics of a class of neural networks known as Restricted Boltzmann Machines (RBMs) in the context of unsupervised learning. We show how the network is described as a discrete Markov process and how the detailed balance condition and the Maxwell-Boltzmann equilibrium distribution are sufficient conditions for a complete thermodynamics description, including nonequilibrium fluctuation theorems. Numerical simulations in a fully trained RBM are performed and the heat exchange fluctuation theorem is verified with excellent agreement to the theory. We observe how the contrastive divergence functional, mostly used in unsupervised learning of RBMs, is closely related to nonequilibrium thermodynamic quantities. We also use the framework to interpret the estimation of the partition function of RBMs with the Annealed Importance Sampling method from a thermodynamics standpoint. Finally, we argue that unsupervised learning of RBMs is equivalent to a work protocol in a system driven by the laws of thermodynamics in the absence of labeled data.