Minimax Theory for High-dimensional Gaussian Mixtures with Sparse Mean Separation

TL;DR

This paper establishes fundamental limits and precise bounds on the sample complexity and clustering accuracy for high-dimensional Gaussian mixture models with sparse mean separation, advancing theoretical understanding in this area.

Contribution

It provides the first information-theoretic bounds on clustering and sample complexity for high-dimensional Gaussian mixtures with sparse means, linking feature selection to clustering performance.

Findings

Sample complexity depends on relevant dimensions and mean separation

Efficient procedures can achieve optimal bounds

Theoretical foundation for feature selection in clustering

Abstract

While several papers have investigated computationally and statistically efficient methods for learning Gaussian mixtures, precise minimax bounds for their statistical performance as well as fundamental limits in high-dimensional settings are not well-understood. In this paper, we provide precise information theoretic bounds on the clustering accuracy and sample complexity of learning a mixture of two isotropic Gaussians in high dimensions under small mean separation. If there is a sparse subset of relevant dimensions that determine the mean separation, then the sample complexity only depends on the number of relevant dimensions and mean separation, and can be achieved by a simple computationally efficient procedure. Our results provide the first step of a theoretical basis for recent methods that combine feature selection and clustering.