Regularized tapered sample covariance matrix

TL;DR

This paper introduces Tabasco, a new high-dimensional covariance matrix estimator that combines tapering and shrinkage, optimized for low data-to-dimension ratios, and demonstrates superior performance in simulations and signal processing applications.

Contribution

The paper proposes a novel regularized tapered covariance estimator called Tabasco, with data-adaptive parameters derived for elliptically symmetric distributions, improving estimation accuracy in high-dimensional settings.

Findings

Tabasco outperforms existing covariance estimators in simulations.

The estimator effectively handles low data-to-dimension ratios.

Application to space-time adaptive processing shows practical benefits.

Abstract

Covariance matrix tapers have a long history in signal processing and related fields. Examples of applications include autoregressive models (promoting a banded structure) or beamforming (widening the spectral null width associated with an interferer). In this paper, the focus is on high-dimensional setting where the dimension $p$ is high, while the data aspect ratio $n/p$ is low. We propose an estimator called Tabasco (TApered or BAnded Shrinkage COvariance matrix) that shrinks the tapered sample covariance matrix towards a scaled identity matrix. We derive optimal and estimated (data adaptive) regularization parameters that are designed to minimize the mean squared error (MSE) between the proposed shrinkage estimator and the true covariance matrix. These parameters are derived under the general assumption that the data is sampled from an unspecified elliptically symmetric distribution with finite 4th order moments (both real- and complex-valued cases are addressed). Simulation studies show that the proposed Tabasco outperforms all competing tapering covariance matrix estimators in diverse setups. A space-time adaptive processing (STAP) application also illustrates the benefit of the proposed estimator in a practical signal processing setup.