# Rapid evaluation of the spectral signal detection threshold and Stieltjes transform

## TL;DR

This paper presents fast algorithms for evaluating spectral signal detection thresholds and the Stieltjes transform in large data matrices with heteroscedastic noise, aiding spectral denoising in statistical applications.

## Contribution

The authors develop novel, provably correct algorithms for computing spectral thresholds and Stieltjes transforms in complex noise models, improving speed and accuracy over existing methods.

## Key findings

- Algorithms are faster and more accurate than previous methods.
- Validated performance through numerical experiments.
- Applicable to models with separable variance profiles.

## Abstract

Accurate detection of signal components is a frequently-encountered challenge in statistical applications with low signal-to-noise ratio. This problem is particularly challenging in settings with heteroscedastic noise. In certain signal-plus-noise models of data, such as the classical spiked covariance model and its variants, there are closed formulas for the spectral signal detection threshold (the largest sample eigenvalue attributable solely to noise) for isotropic noise in the limit of infinitely large data matrices. However, more general noise models currently lack provably fast and accurate methods for numerically evaluating the threshold. In this work, we introduce a rapid algorithm for evaluating the spectral signal detection threshold in the limit of infinitely large data matrices. We consider noise matrices with a separable variance profile (whose variance matrix is rank one), as these arise often in applications. The solution is based on nested applications of Newton's method. We also devise a new algorithm for evaluating the Stieltjes transform of the spectral distribution at real values exceeding the threshold. The Stieltjes transform on this domain is known to be a key quantity in parameter estimation for spectral denoising methods. The correctness of both algorithms is proven from a detailed analysis of the master equations characterizing the Stieltjes transform, and their performance is demonstrated in numerical experiments.