Regularized estimation of large covariance matrices

TL;DR

This paper investigates methods for estimating large covariance matrices using banding and tapering techniques, establishing their consistency and convergence rates under certain conditions, and introduces practical approaches for parameter selection.

Contribution

It provides theoretical guarantees for the consistency of banded and tapered covariance estimators in high-dimensional settings, including explicit rates and extensions to non-Gaussian data.

Findings

Consistency of estimators as (log p)/n → 0

Explicit convergence rates for operator norm

Effective resampling method for parameter tuning

Abstract

This paper considers estimating a covariance matrix of $p$ variables from $n$ observations by either banding or tapering the sample covariance matrix, or estimating a banded version of the inverse of the covariance. We show that these estimates are consistent in the operator norm as long as $(\log p)/n\to0$, and obtain explicit rates. The results are uniform over some fairly natural well-conditioned families of covariance matrices. We also introduce an analogue of the Gaussian white noise model and show that if the population covariance is embeddable in that model and well-conditioned, then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix. The results can be extended to smooth versions of banding and to non-Gaussian distributions with sufficiently short tails. A resampling approach is proposed for choosing the banding parameter in practice. This approach is illustrated numerically on both simulated and real data.