Stable Estimation of a Covariance Matrix Guided by Nuclear Norm Penalties

TL;DR

This paper introduces a new covariance matrix estimator that combines nuclear norm penalties with a shrinkage approach, ensuring stability, invertibility, and efficiency, especially in high-dimensional settings.

Contribution

It proposes a novel prior-based estimator that shrinks towards a stable target, with proven consistency and asymptotic efficiency, improving covariance estimation in high-dimensional data.

Findings

Estimator is consistent and asymptotically efficient.

Performs well in discriminant analysis with high-dimensional data.

Effective in EM clustering when samples are limited.

Abstract

Estimation of covariance matrices or their inverses plays a central role in many statistical methods. For these methods to work reliably, estimated matrices must not only be invertible but also well-conditioned. In this paper we present an intuitive prior that shrinks the classic sample covariance estimator towards a stable target. We prove that our estimator is consistent and asymptotically efficient. Thus, it gracefully transitions towards the sample covariance matrix as the number of samples grows relative to the number of covariates. We also demonstrate the utility of our estimator in two standard situations -- discriminant analysis and EM clustering -- when the number of samples is dominated by or comparable to the number of covariates.