Moment Estimation for Nonparametric Mixture Models Through Implicit Tensor Decomposition

TL;DR

This paper introduces a tensor decomposition-based method for estimating nonparametric mixture models in high dimensions, avoiding high computational costs and providing theoretical guarantees of identifiability and convergence.

Contribution

It develops an efficient tensor-free algorithm for moment-based estimation of mixture models, with theoretical analysis of identifiability and convergence.

Findings

Algorithm demonstrates competitive performance in numerical experiments.

Method applicable to various models and high-dimensional data.

Theoretical guarantees include identifiability and local linear convergence.

Abstract

We present an alternating least squares type numerical optimization scheme to estimate conditionally-independent mixture models in $\mathbb{R}^n$, without parameterizing the distributions. Following the method of moments, we tackle an incomplete tensor decomposition problem to learn the mixing weights and componentwise means. Then we compute the cumulative distribution functions, higher moments and other statistics of the component distributions through linear solves. Crucially for computations in high dimensions, the steep costs associated with high-order tensors are evaded, via the development of efficient tensor-free operations. Numerical experiments demonstrate the competitive performance of the algorithm, and its applicability to many models and applications. Furthermore we provide theoretical analyses, establishing identifiability from low-order moments of the mixture and guaranteeing local linear convergence of the ALS algorithm.