Support Recovery in Mixture Models with Sparse Parameters

TL;DR

This paper introduces efficient algorithms for support recovery in high-dimensional sparse mixture models, applicable to various distributions and regression settings, with logarithmic sample complexity.

Contribution

It provides the first or improved guarantees for support recovery in sparse mixture models across multiple distribution types and regression scenarios.

Findings

Algorithms achieve logarithmic sample complexity.

Applicable to diverse distributions including Gaussian, Poisson, Laplace, Uniform.

First guarantees or improvements over existing methods.

Abstract

Mixture models are widely used to fit complex and multimodal datasets. In this paper we study mixtures with high dimensional sparse latent parameter vectors and consider the problem of support recovery of those vectors. While parameter learning in mixture models is well-studied, the sparsity constraint remains relatively unexplored. Sparsity of parameter vectors is a natural constraint in variety of settings, and support recovery is a major step towards parameter estimation. We provide efficient algorithms for support recovery that have a logarithmic sample complexity dependence on the dimensionality of the latent space. Our algorithms are quite general, namely they are applicable to 1) mixtures of many different canonical distributions including Uniform, Poisson, Laplace, Gaussians, etc. 2) Mixtures of linear regressions and linear classifiers with Gaussian covariates under different assumptions on the unknown parameters. In most of these settings, our results are the first guarantees on the problem while in the rest, our results provide improvements on existing works.