An Orthogonally Equivariant Estimator of the Covariance Matrix in High Dimensions and for Small Sample Sizes

TL;DR

This paper proposes a new orthogonally equivariant covariance matrix estimator for high-dimensional data with small sample sizes, using an adjusted likelihood approach and tuning via bootstrap and cross-validation.

Contribution

It introduces an eigenvector-preserving estimator with a novel likelihood adjustment and a practical tuning method, improving covariance estimation in high dimensions.

Findings

Estimator performs well compared to existing methods in simulations.

Bootstrap and cross-validation effectively select tuning parameters.

Monte Carlo risk estimates validate the estimator's accuracy.

Abstract

We introduce an estimation method of covariance matrices in a high-dimensional setting, i.e., when the dimension of the matrix, , is larger than the sample size . Specifically, we propose an orthogonally equivariant estimator. The eigenvectors of such estimator are the same as those of the sample covariance matrix. The eigenvalue estimates are obtained from an adjusted profile likelihood function derived by approximating the integral of the density function of the sample covariance matrix over its eigenvectors, which is a challenging problem in its own right. Exact solutions to the approximate likelihood equations are obtained and employed to construct estimates that involve a tuning parameter. Bootstrap and cross-validation based algorithms are proposed to choose this tuning parameter under various loss functions. Finally, comparisons with two well-known orthogonally equivariant estimators of the covariance matrix are given, which are based on Monte-Carlo risk estimates for simulated data and misclassification errors in real data analyses. In addition, Monte-Carlo risk estimates are also provided to compare our estimates of eigenvalues to those of a consistent estimator of population eigenvalues.