Optimal Structured Principal Subspace Estimation: Metric Entropy and Minimax Rates

TL;DR

This paper develops a unified theoretical framework for structured principal subspace estimation, deriving minimax bounds and revealing phase transition phenomena across various structural constraints.

Contribution

It introduces a general approach to analyze structured PCA/SVD problems, establishing new minimax rates and phase transition insights that unify and extend prior results.

Findings

Established minimax lower and upper bounds for structured subspace estimation.

Identified phase transition phenomena related to SNR and dimensionality.

Derived optimal convergence rates for non-negative, sparse, and constrained PCA/SVD.

Abstract

Driven by a wide range of applications, many principal subspace estimation problems have been studied individually under different structural constraints. This paper presents a unified framework for the statistical analysis of a general structured principal subspace estimation problem which includes as special cases non-negative PCA/SVD, sparse PCA/SVD, subspace constrained PCA/SVD, and spectral clustering. General minimax lower and upper bounds are established to characterize the interplay between the information-geometric complexity of the structural set for the principal subspaces, the signal-to-noise ratio (SNR), and the dimensionality. The results yield interesting phase transition phenomena concerning the rates of convergence as a function of the SNRs and the fundamental limit for consistent estimation. Applying the general results to the specific settings yields the minimax rates of convergence for those problems, including the previous unknown optimal rates for non-negative PCA/SVD, sparse SVD and subspace constrained PCA/SVD.