Covariance matrix estimation in the singular case using regularized Cholesky factor

TL;DR

This paper proposes a regularized Cholesky-based method for estimating singular covariance matrices in high-dimensional settings, providing optimal estimators and practical algorithms with demonstrated numerical advantages.

Contribution

It introduces a novel regularized Cholesky factor approach for covariance estimation when samples are fewer than dimensions, including finite sample and oracle estimators.

Findings

The proposed method outperforms existing techniques in numerical experiments.

Performance depends on the covariance matrix's condition number and spectral shape.

The approach effectively handles singular covariance matrices in high-dimensional data.

Abstract

We consider estimating the population covariance matrix when the number of available samples is less than the size of the observations. The sample covariance matrix (SCM) being singular, regularization is mandatory in this case. For this purpose we consider minimizing Stein's loss function and we investigate a method based on augmenting the partial Cholesky decomposition of the SCM. We first derive the finite sample optimum estimator which minimizes the loss for each data realization, then the Oracle estimator which minimizes the risk, i.e., the average value of the loss. Finally a practical scheme is presented where the missing part of the Cholesky decomposition is filled. We conduct a numerical performance study of the proposed method and compare it with available related methods. In particular we investigate the influence of the condition number of the covariance matrix as well as of the shape of its spectrum.