Estimating Higher-Order Mixed Memberships via the $\ell_{2,\infty}$ Tensor Perturbation Bound

TL;DR

This paper introduces a tensor mixed-membership blockmodel for higher-order data, providing an efficient estimation method and establishing theoretical guarantees under noisy conditions, with applications to real and simulated data.

Contribution

It generalizes tensor blockmodels to mixed memberships, develops an $ ext{l}_{2, ext{infinity}}$ perturbation bound for tensor SVD, and proves the consistency of the estimation procedure.

Findings

The proposed method accurately estimates mixed memberships in higher-order tensors.

The $ ext{l}_{2, ext{infinity}}$ perturbation bound is novel and applicable under heteroskedastic noise.

Applications demonstrate the model's ability to reveal effects beyond discrete community structures.

Abstract

Higher-order multiway data is ubiquitous in machine learning and statistics and often exhibits community-like structures, where each component (node) along each different mode has a community membership associated with it. In this paper we propose the tensor mixed-membership blockmodel, a generalization of the tensor blockmodel positing that memberships need not be discrete, but instead are convex combinations of latent communities. We establish the identifiability of our model and propose a computationally efficient estimation procedure based on the higher-order orthogonal iteration algorithm (HOOI) for tensor SVD composed with a simplex corner-finding algorithm. We then demonstrate the consistency of our estimation procedure by providing a per-node error bound, which showcases the effect of higher-order structures on estimation accuracy. To prove our consistency result, we develop the $\ell_{2,\infty}$ tensor perturbation bound for HOOI under independent, heteroskedastic, subgaussian noise that may be of independent interest. Our analysis uses a novel leave-one-out construction for the iterates, and our bounds depend only on spectral properties of the underlying low-rank tensor under nearly optimal signal-to-noise ratio conditions such that tensor SVD is computationally feasible. Finally, we apply our methodology to real and simulated data, demonstrating some effects not identifiable from the model with discrete community memberships.