Generalized Grade-of-Membership Estimation for High-dimensional Locally Dependent Data

TL;DR

This paper introduces a scalable spectral method for estimating generalized Grade-of-Membership models in high-dimensional, locally dependent categorical data, overcoming limitations of traditional Bayesian approaches.

Contribution

The authors propose a novel flattening and spectral decomposition approach with theoretical error bounds for high-dimensional, locally dependent data in generalized GoM models.

Findings

Method outperforms existing approaches in simulations.

Applicable to diverse data types like surveys and genetics.

Provides finite-sample error guarantees.

Abstract

This work focuses on the mixed membership models for multivariate categorical data widely used for analyzing survey responses and population genetics data. These grade of membership (GoM) models offer rich modeling power but present significant estimation challenges for high-dimensional polytomous data. Popular existing approaches, such as Bayesian MCMC inference, are not scalable and lack theoretical guarantees in high-dimensional settings. To address this, we first observe that data from this model can be reformulated as a three-way (quasi-)tensor, with many subjects responding to many items with varying numbers of categories. We introduce a novel and simple approach that flattens the three-way quasi-tensor into a "fat" matrix, and then perform a singular value decomposition of it to estimate parameters by exploiting the singular subspace geometry. Our fast spectral method can accommodate a broad range of data distributions with arbitrarily locally dependent noise, which we formalize as the generalized-GoM models. We establish finite-sample entrywise error bounds for the generalized-GoM model parameters. This is supported by a new sharp two-to-infinity singular subspace perturbation theory for locally dependent and flexibly distributed noise, a contribution of independent interest. Simulations and applications to data in political surveys, population genetics, and single-cell sequencing demonstrate our method's superior performance.