Dimension of Marginals of Kronecker Product Models

TL;DR

This paper investigates the dimension of Kronecker product models, providing combinatorial conditions for when these models reach their expected dimension and proving that binary restricted Boltzmann machines always do.

Contribution

It introduces a method to estimate the dimension of Kronecker product models using tropical geometry and establishes that binary restricted Boltzmann machines have the expected dimension.

Findings

Derived combinatorial conditions for model dimension

Established the expected dimension for binary restricted Boltzmann machines

Connected model dimension to tropical morphism and slicings

Abstract

A Kronecker product model is the set of visible marginal probability distributions of an exponential family whose sufficient statistics matrix factorizes as a Kronecker product of two matrices, one for the visible variables and one for the hidden variables. We estimate the dimension of these models by the maximum rank of the Jacobian in the limit of large parameters. The limit is described by the tropical morphism; a piecewise linear map with pieces corresponding to slicings of the visible matrix by the normal fan of the hidden matrix. We obtain combinatorial conditions under which the model has the expected dimension, equal to the minimum of the number of natural parameters and the dimension of the ambient probability simplex. Additionally, we prove that the binary restricted Boltzmann machine always has the expected dimension.