Geometry of the restricted Boltzmann machine

TL;DR

This paper explores the algebraic and geometric properties of restricted Boltzmann machines, revealing their structure through tropical geometry and algebraic statistics, and establishing conditions for their identifiability.

Contribution

It introduces a novel algebraic geometric framework for analyzing RBMs, including dimension formulas and identifiability results, connecting them to secant varieties and tropical geometry.

Findings

Zariski closure as Hadamard power of secant variety

Dimension formula for tropicalized RBM

Identifiability in many cases

Abstract

The restricted Boltzmann machine is a graphical model for binary random variables. Based on a complete bipartite graph separating hidden and observed variables, it is the binary analog to the factor analysis model. We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starting with the observation that its Zariski closure is a Hadamard power of the first secant variety of the Segre variety of projective lines. We derive a dimension formula for the tropicalized model, and we use it to show that the restricted Boltzmann machine is identifiable in many cases. Our methods include coding theory and geometry of linear threshold functions.