Optimal Estimation of Shared Singular Subspaces across Multiple Noisy Matrices

TL;DR

This paper develops optimal methods for estimating shared singular subspaces across multiple noisy matrices, revealing phase transitions and proposing new algorithms for partial sharing scenarios with theoretical guarantees.

Contribution

It establishes minimax optimality of Stack-SVD for fully shared subspaces and introduces novel estimators with proven optimality for partially shared cases.

Findings

Stack-SVD is minimax optimal for fully shared subspaces.

Phase transition phenomena depend on signal-to-noise ratio.

New algorithms effectively identify shared and unshared singular vectors.

Abstract

Estimating singular subspaces from noisy matrices is a fundamental problem with wide-ranging applications across various fields. Driven by the challenges of data integration and multi-view analysis, this study focuses on estimating shared (left) singular subspaces across multiple matrices within a low-rank matrix denoising framework. A common approach for this task is to perform singular value decomposition on the stacked matrix (Stack-SVD), which is formed by concatenating all the individual matrices. We establish that Stack-SVD achieves minimax rate-optimality when the true (left) singular subspaces of the signal matrices are identical. Our analysis reveals some phase transition phenomena in the estimation problem as a function of the underlying signal-to-noise ratio, highlighting how the interplay among multiple matrices collectively determines the fundamental limits of estimation. We then tackle the more complex scenario where the true singular subspaces are only partially shared across matrices. For various cases of partial sharing, we rigorously characterize the conditions under which Stack-SVD remains effective, achieves minimax optimality, or fails to deliver consistent estimates, offering theoretical insights into its practical applicability. To overcome Stack-SVD's limitations in partial sharing scenarios, we propose novel estimators and an efficient algorithm to identify shared and unshared singular vectors, and prove their minimax rate-optimality. Extensive simulation studies and real-world data applications demonstrate the numerous advantages of our proposed approaches.