A Spectral Method for Identifiable Grade of Membership Analysis with Binary Responses

TL;DR

This paper introduces a spectral SVD-based method for Grade of Membership models with binary data, providing a scalable, efficient, and consistent approach for estimating mixed memberships and parameters, outperforming traditional methods.

Contribution

The paper develops a novel spectral approach for GoM analysis with binary responses, establishing identifiability conditions and proving estimator consistency in large-scale settings.

Findings

Spectral method outperforms Bayesian and likelihood-based methods in simulations.

Method is scalable to high-dimensional, large datasets.

Consistent estimation of mixed memberships is achieved asymptotically.

Abstract

Grade of Membership (GoM) models are popular individual-level mixture models for multivariate categorical data. GoM allows each subject to have mixed memberships in multiple extreme latent profiles. Therefore GoM models have a richer modeling capacity than latent class models that restrict each subject to belong to a single profile. The flexibility of GoM comes at the cost of more challenging identifiability and estimation problems. In this work, we propose a singular value decomposition (SVD) based spectral approach to GoM analysis with multivariate binary responses. Our approach hinges on the observation that the expectation of the data matrix has a low-rank decomposition under a GoM model. For identifiability, we develop sufficient and almost necessary conditions for a notion of expectation identifiability. For estimation, we extract only a few leading singular vectors of the observed data matrix, and exploit the simplex geometry of these vectors to estimate the mixed membership scores and other parameters. We also establish the consistency of our estimator in the double-asymptotic regime where both the number of subjects and the number of items grow to infinity. Our spectral method has a huge computational advantage over Bayesian or likelihood-based methods and is scalable to large-scale and high-dimensional data. Extensive simulation studies demonstrate the superior efficiency and accuracy of our method. We also illustrate our method by applying it to a personality test dataset.