Tracy-Widom distribution for heterogeneous Gram matrices with applications in signal detection

TL;DR

This paper develops a Tracy-Widom distribution-based method for detecting signals in high-dimensional, heterogeneously noisy data, providing a statistically rigorous and effective approach for signal detection and estimation.

Contribution

It introduces a new sequential testing method using edge singular values and establishes Tracy-Widom law for a broad class of Gram matrices with complex variance structures.

Findings

The test accurately detects signals with full power.

The asymptotic distribution of the test statistic follows Tracy-Widom law.

The method is validated both theoretically and numerically.

Abstract

Detection of the number of signals corrupted by high-dimensional noise is a fundamental problem in signal processing and statistics. This paper focuses on a general setting where the high-dimensional noise has an unknown complicated heterogeneous variance structure. We propose a sequential test which utilizes the edge singular values (i.e., the largest few singular values) of the data matrix. It also naturally leads to a consistent sequential testing estimate of the number of signals. We describe the asymptotic distribution of the test statistic in terms of the Tracy-Widom distribution. The test is shown to be accurate and have full power against the alternative, both theoretically and numerically. The theoretical analysis relies on establishing the Tracy-Widom law for a large class of Gram type random matrices with non-zero means and completely arbitrary variance profiles, which can be of independent interest.