Mixtures and products in two graphical models

TL;DR

This paper compares a mixture model and a restricted Boltzmann machine for three binary variables, showing they represent the same distributions and providing algebraic descriptions and maximum likelihood estimates.

Contribution

It demonstrates the equivalence of two different graphical models on the interior of the probability simplex and offers algebraic and closed-form solutions for parameter estimation.

Findings

Models are equivalent up to closure on the interior of the probability simplex

Provides a semi-algebraic description with six binomial inequalities

Derives closed-form maximum likelihood estimates

Abstract

We compare two statistical models of three binary random variables. One is a mixture model and the other is a product of mixtures model called a restricted Boltzmann machine. Although the two models we study look different from their parametrizations, we show that they represent the same set of distributions on the interior of the probability simplex, and are equal up to closure. We give a semi-algebraic description of the model in terms of six binomial inequalities and obtain closed form expressions for the maximum likelihood estimates. We briefly discuss extensions to larger models.