Mixture decompositions of exponential families using a decomposition of their sample spaces

TL;DR

This paper investigates the minimal mixture decompositions of exponential families using geometric and coding theory methods, providing explicit bounds for distributions over binary and q-ary variables.

Contribution

It introduces a novel approach based on face lattice coverings and packings to determine minimal mixture decompositions for exponential families.

Findings

For q-ary variables, the minimal mixture size is q^{N-1}.

For binary variables, the minimal mixture size depends on the interaction order k.

Provides explicit bounds for mixture decompositions of distributions.

Abstract

We study the problem of finding the smallest $m$ such that every element of an exponential family can be written as a mixture of $m$ elements of another exponential family. We propose an approach based on coverings and packings of the face lattice of the corresponding convex support polytopes and results from coding theory. We show that $m=q^{N-1}$ is the smallest number for which any distribution of $N$ $q$-ary variables can be written as mixture of $m$ independent $q$-ary variables. Furthermore, we show that any distribution of $N$ binary variables is a mixture of $m = 2^{N-(k+1)}(1+ 1/(2^k-1))$ elements of the $k$-interaction exponential family.