Deep learning and the renormalization group

TL;DR

This paper explores the conceptual parallels between renormalization group methods in physics and deep learning, illustrating how RG-inspired algorithms can be transformed into hierarchical Bayesian models for efficient learning.

Contribution

It demonstrates how the multiscale entanglement renormalization ansatz (MERA) can be converted into a generative hierarchical Bayesian network, bridging physics and machine learning.

Findings

RG-inspired algorithms can be reformulated as hierarchical Bayesian networks

The proposed method avoids sampling by using explicit probability evaluation

The approach leverages local correlations for efficient learning

Abstract

Renormalization group (RG) methods, which model the way in which the effective behavior of a system depends on the scale at which it is observed, are key to modern condensed-matter theory and particle physics. We compare the ideas behind the RG on the one hand and deep machine learning on the other, where depth and scale play a similar role. In order to illustrate this connection, we review a recent numerical method based on the RG---the multiscale entanglement renormalization ansatz (MERA)---and show how it can be converted into a learning algorithm based on a generative hierarchical Bayesian network model. Under the assumption---common in physics---that the distribution to be learned is fully characterized by local correlations, this algorithm involves only explicit evaluation of probabilities, hence doing away with sampling.