Universal Spin Models are Universal Approximators in Machine Learning

TL;DR

This paper proves that universal spin models can approximate any probability distribution, establishing a connection with universal approximation theorems in machine learning and providing a unified framework for various models.

Contribution

It demonstrates that universal spin models are universal approximators of probability distributions and offers a unified method to verify universal approximation conditions.

Findings

Universal spin models can reproduce any probability distribution.

The paper provides a unified recipe for universal approximation theorems.

Validated the approach on Boltzmann machines and deep belief networks.

Abstract

One of the theoretical pillars that sustain certain machine learning models are universal approximation theorems, which prove that they can approximate all functions from a function class to arbitrary precision. Independently, classical spin models are termed universal if they can reproduce the behavior of any other spin model in their low energy sector. Universal spin models have been characterized via sufficient and necessary conditions, showing that simple models such as the 2d Ising with fields are universal. In this work, we prove that universal spin models are universal approximators of probability distributions. This enables us to leverage the characterization of the former to reveal conditions which are sufficient for universal approximation. Deriving universal approximation theorems thus amounts to verifying these conditions, yielding a unified recipe for universal approximation theorems applicable to a wide range of models. We explicitly test this recipe for restricted and deep Boltzmann machines, as well as for deep belief networks. This work illustrates that independently discovered universality statements may be intimately related, enabling the transfer of results.