Shrinkage Algorithms for MMSE Covariance Estimation

TL;DR

This paper introduces improved shrinkage algorithms for MMSE covariance estimation in high-dimensional settings with limited samples, outperforming existing methods like Ledoit-Wolf and demonstrating practical benefits in adaptive beamforming.

Contribution

It develops the RBLW estimator via Rao-Blackwellization and proposes the OAS iterative estimator, both offering simple, effective solutions with proven dominance and convergence.

Findings

RBLW outperforms Ledoit-Wolf in mean-squared error.

OAS further improves estimation accuracy, especially with very small sample sizes.

Both methods are computationally simple and effective in adaptive beamforming applications.

Abstract

We address covariance estimation in the sense of minimum mean-squared error (MMSE) for Gaussian samples. Specifically, we consider shrinkage methods which are suitable for high dimensional problems with a small number of samples (large p small n). First, we improve on the Ledoit-Wolf (LW) method by conditioning on a sufficient statistic. By the Rao-Blackwell theorem, this yields a new estimator called RBLW, whose mean-squared error dominates that of LW for Gaussian variables. Second, to further reduce the estimation error, we propose an iterative approach which approximates the clairvoyant shrinkage estimator. Convergence of this iterative method is established and a closed form expression for the limit is determined, which is referred to as the oracle approximating shrinkage (OAS) estimator. Both RBLW and OAS estimators have simple expressions and are easily implemented. Although the two methods are developed from different persepctives, their structure is identical up to specified constants. The RBLW estimator provably dominates the LW method. Numerical simulations demonstrate that the OAS approach can perform even better than RBLW, especially when n is much less than p. We also demonstrate the performance of these techniques in the context of adaptive beamforming.