Estimating Mixture Models via Mixtures of Polynomials

TL;DR

This paper introduces Polymom, a unifying method of moments framework for mixture models with polynomial moments, enabling globally guaranteed estimation via convex optimization and algebraic techniques.

Contribution

Polymom provides a new, generalizable approach to estimate mixture models with polynomial moments, combining moments, convex optimization, and algebraic methods.

Findings

Framework applies to mixture models with polynomial moments.

Estimation reduces to a Generalized Moment Problem solvable by semidefinite programming.

Provides insights and tools for statistical estimation using convex and algebraic methods.

Abstract

Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of a mixture model is non-convex, so EM has no known global convergence guarantees. Recently, method of moments approaches offer global guarantees for some mixture models, but they do not extend easily to the range of mixture models that exist. In this work, we present Polymom, an unifying framework based on method of moments in which estimation procedures are easily derivable, just as in EM. Polymom is applicable when the moments of a single mixture component are polynomials of the parameters. Our key observation is that the moments of the mixture model are a mixture of these polynomials, which allows us to cast estimation as a Generalized Moment Problem. We solve its relaxations using semidefinite optimization, and then extract parameters using ideas from computer algebra. This framework allows us to draw insights and apply tools from convex optimization, computer algebra and the theory of moments to study problems in statistical estimation.