Restricted Boltzmann Machines: Introduction and Review

TL;DR

This paper provides an introduction and review of restricted Boltzmann machines, highlighting their mathematical structure, recent geometric insights, and their significance in deep learning and graphical models.

Contribution

It offers a comprehensive overview of the mathematical analysis and recent geometric results related to restricted Boltzmann machines, and suggests future research directions.

Findings

Review of the geometric structure of RBMs

Recent results on the probability distribution sets of RBMs

Discussion of future research directions in RBM analysis

Abstract

The restricted Boltzmann machine is a network of stochastic units with undirected interactions between pairs of visible and hidden units. This model was popularized as a building block of deep learning architectures and has continued to play an important role in applied and theoretical machine learning. Restricted Boltzmann machines carry a rich structure, with connections to geometry, applied algebra, probability, statistics, machine learning, and other areas. The analysis of these models is attractive in its own right and also as a platform to combine and generalize mathematical tools for graphical models with hidden variables. This article gives an introduction to the mathematical analysis of restricted Boltzmann machines, reviews recent results on the geometry of the sets of probability distributions representable by these models, and suggests a few directions for further investigation.