Tight bounds for learning a mixture of two gaussians

TL;DR

This paper establishes optimal sample complexity bounds for learning the parameters of a two-component Gaussian mixture in any dimension, using a moment-based estimator with efficient algorithms and identifying special cases with reduced sample needs.

Contribution

It provides the first tight bounds on the sample complexity for Gaussian mixture learning, extending to multiple dimensions and mixtures of more than two Gaussians, with novel dimensionality reduction techniques.

Findings

Optimal sample complexity of Θ(σ^{12}) in 1D

Extension of bounds to higher dimensions with logarithmic loss

Reduced sample complexity in special cases like well-separated means

Abstract

We consider the problem of identifying the parameters of an unknown mixture of two arbitrary $d$-dimensional gaussians from a sequence of independent random samples. Our main results are upper and lower bounds giving a computationally efficient moment-based estimator with an optimal convergence rate, thus resolving a problem introduced by Pearson (1894). Denoting by $\sigma^2$ the variance of the unknown mixture, we prove that $\Theta(\sigma^{12})$ samples are necessary and sufficient to estimate each parameter up to constant additive error when $d=1.$ Our upper bound extends to arbitrary dimension $d>1$ up to a (provably necessary) logarithmic loss in $d$ using a novel---yet simple---dimensionality reduction technique. We further identify several interesting special cases where the sample complexity is notably smaller than our optimal worst-case bound. For instance, if the means of the two components are separated by $\Omega(\sigma)$ the sample complexity reduces to $O(\sigma^2)$ and this is again optimal. Our results also apply to learning each component of the mixture up to small error in total variation distance, where our algorithm gives strong improvements in sample complexity over previous work. We also extend our lower bound to mixtures of $k$ Gaussians, showing that $\Omega(\sigma^{6k-2})$ samples are necessary to estimate each parameter up to constant additive error.