Discontinuous BBP transitions

TL;DR

This paper reveals that BBP transitions, crucial for detecting low-rank signals in noisy high-dimensional data, can be discontinuous, leading to earlier and more variable signal detection than previously understood.

Contribution

The authors develop a comprehensive theory for discontinuous BBP transitions, expanding the understanding of spectral phase transitions in high-dimensional data analysis.

Findings

Eigenvector overlap jumps discontinuously at the spectral edge.

Finite-size effects significantly influence the transition behavior.

Informative eigenvectors can emerge before the asymptotic threshold.

Abstract

The Baik-Ben Arous-Peche (BBP) transition sets fundamental limits for detecting low-rank structure in noisy high-dimensional data and underlies a wide range of spectral methods in many fields from physics to statistics and data sciences. In standard settings, this transition is continuous, implying that signal recovery emerges gradually above a sharp threshold. We show that BBP transitions can instead be discontinuous in very general settings and provide a full theory of this phenomenon. When the eigenvalue density vanishes faster than linearly at the spectral edge, the overlap between the leading eigenvector and the signal jumps discontinuously at the critical point. We study this mechanism in deformed Gaussian and reweighted Wishart ensembles. We analyze in detail the finite-size effects, which play a central and qualitatively new role in the discontinuous BBP transition. Unlike the continuous BBP transition, we establish the existence of an extended pre-critical region where informative eigenvectors emerge well before the asymptotic threshold. The main consequence-and difference from the continuous BBP transition-is that signal recovery can occur at significantly lower signal-to-noise ratio and it is accompanied by strong sample-to-sample variability. Our results show the relevance and the novelty of the discontinuous BBP transition, and highlight the practical implications for signal detection.