Thermodynamics of the Ising model encoded in restricted Boltzmann machines

TL;DR

This paper investigates how restricted Boltzmann machines learn and encode the thermodynamic properties of the 2D and 3D Ising models, revealing indicators of phase transitions and physical quantities from trained models.

Contribution

It provides a detailed analysis of the RBM's internal representations and their relation to the Ising phase transition, offering new insights into the physical interpretability of learned parameters.

Findings

Weight matrix indicators can characterize phase transition

Hidden units tend to have balanced positive and negative states

Energy and loss functions can predict critical points and estimate entropy

Abstract

The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point $T_c$. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the $2d$ and $3d$ Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.