High-Dimensional Covariance Shrinkage for Signal Detection

TL;DR

This paper develops a high-dimensional covariance shrinkage method to improve signal detection performance in scenarios where the number of observations is comparable to data dimension, using asymptotic analysis.

Contribution

It introduces an asymptotically optimal covariance estimator within a shrinkage class for high-dimensional signal detection, addressing limitations of sample covariance matrices.

Findings

Identifies a covariance estimator optimal in large-dimensional asymptotics.

Provides consistent estimates for false-alarm and detection probabilities.

Enhances adaptive matched filter performance in high-dimensional settings.

Abstract

In this paper, we consider the problem of determining the presence of a given signal in a high-dimensional observation with unknown covariance matrix by using an adaptive matched filter. Traditionally such filters are formed from the sample covariance matrix of some given training data, but, as is well-known, the performance of such filters is poor when the number of training data $n$ is not much larger than the data dimension $p$. We thus seek a covariance estimator to replace sample covariance. To account for the fact that $n$ and $p$ may be of comparable size, we adopt the "large-dimensional asymptotic model" in which $n$ and $p$ go to infinity in a fixed ratio. Under this assumption, we identify a covariance estimator that is asymptotically detection-theoretic optimal within a general shrinkage class inspired by C. Stein, and we give consistent estimates for conditional false-alarm and detection rate of the corresponding adaptive matched filter.