Spiked Singular Values and Vectors under Extreme Aspect Ratios

TL;DR

This paper investigates the behavior of singular values and vectors of noisy low-rank matrices in extreme aspect ratio regimes, revealing phase transitions and asymptotic properties relevant to big data applications.

Contribution

It extends the theoretical understanding of singular value behavior to disproportionate matrices with extreme aspect ratios, a regime not well-understood before.

Findings

Disproportionate aspect ratios still exhibit phase transitions in singular values and vectors.

Singular vectors aligned with the longer dimension become asymptotically uncorrelated with the true signal.

A new measurement scale is introduced to quantify these effects in the disproportionate setting.

Abstract

The behavior of the leading singular values and vectors of noisy low-rank matrices is fundamental to many statistical and scientific problems. Theoretical understanding currently derives from asymptotic analysis under one of two regimes: (1) the classical regime, with a fixed number of rows and large number of columns, or vice versa, and (2) the proportional regime, with large numbers of rows and columns, proportional to one another. This paper is concerned with the disproportional regime, where the matrix is either ``tall and narrow'' or ``short and wide'': we study sequences of matrices of size $n \times m_n$ with aspect ratio $ n/m_n \rightarrow 0$ or $n/m_n \rightarrow \infty$ as $n \rightarrow \infty$. This regime has important ``big data'' applications. Theory derived here shows that the displacement of the empirical singular values and vectors from their noise-free counterparts and the associated phase transitions -- well-known under proportional growth asymptotics -- still occur in the disproportionate setting. They must be quantified, however, on a novel scale of measurement that adjusts with the changing aspect ratio as the matrix size increases. In this setting, the top singular vectors corresponding to the longer of the two matrix dimensions are asymptotically uncorrelated with the noise-free signal.