Mode-wise Principal Subspace Pursuit and Matrix Spiked Covariance Model

TL;DR

This paper presents MOP-UP, a new method for extracting informative subspaces from matrix data by leveraging a novel matrix-variate spiked covariance model, with proven convergence and demonstrated effectiveness on real and simulated data.

Contribution

The paper introduces the MOP-UP framework and a class of matrix-variate spiked covariance models, providing theoretical analysis and practical algorithms for subspace recovery.

Findings

Exact recovery in noiseless settings

Convergence and error bounds established

Effective on both simulated and real datasets

Abstract

This paper introduces a novel framework called Mode-wise Principal Subspace Pursuit (MOP-UP) to extract hidden variations in both the row and column dimensions for matrix data. To enhance the understanding of the framework, we introduce a class of matrix-variate spiked covariance models that serve as inspiration for the development of the MOP-UP algorithm. The MOP-UP algorithm consists of two steps: Average Subspace Capture (ASC) and Alternating Projection (AP). These steps are specifically designed to capture the row-wise and column-wise dimension-reduced subspaces which contain the most informative features of the data. ASC utilizes a novel average projection operator as initialization and achieves exact recovery in the noiseless setting. We analyze the convergence and non-asymptotic error bounds of MOP-UP, introducing a blockwise matrix eigenvalue perturbation bound that proves the desired bound, where classic perturbation bounds fail. The effectiveness and practical merits of the proposed framework are demonstrated through experiments on both simulated and real datasets. Lastly, we discuss generalizations of our approach to higher-order data.