On estimation of the diagonal elements of a sparse precision matrix

TL;DR

This paper introduces and empirically evaluates four estimators for the diagonal elements of a sparse precision matrix in high-dimensional Gaussian models, highlighting the symmetry-enforced maximum likelihood as most accurate under ideal conditions.

Contribution

It presents four natural estimators for the diagonal entries of the precision matrix and provides a comprehensive empirical comparison, filling a gap in existing statistical methods.

Findings

Symmetry-enforced maximum likelihood performs best when regression vectors are estimated without error.

Residual variance estimator is slightly better in realistic, error-prone estimation scenarios.

All estimators show comparable performance when regression vectors are estimated with sparsity-favoring methods.

Abstract

In this paper, we present several estimators of the diagonal elements of the inverse of the covariance matrix, called precision matrix, of a sample of iid random vectors. The focus is on high dimensional vectors having a sparse precision matrix. It is now well understood that when the underlying distribution is Gaussian, the columns of the precision matrix can be estimated independently form one another by solving linear regression problems under sparsity constraints. This approach leads to a computationally efficient strategy for estimating the precision matrix that starts by estimating the regression vectors, then estimates the diagonal entries of the precision matrix and, in a final step, combines these estimators for getting estimators of the off-diagonal entries. While the step of estimating the regression vector has been intensively studied over the past decade, the problem of deriving statistically accurate estimators of the diagonal entries has received much less attention. The goal of the present paper is to fill this gap by presenting four estimators---that seem the most natural ones---of the diagonal entries of the precision matrix and then performing a comprehensive empirical evaluation of these estimators. The estimators under consideration are the residual variance, the relaxed maximum likelihood, the symmetry-enforced maximum likelihood and the penalized maximum likelihood. We show, both theoretically and empirically, that when the aforementioned regression vectors are estimated without error, the symmetry-enforced maximum likelihood estimator has the smallest estimation error. However, in a more realistic setting when the regression vector is estimated by a sparsity-favoring computationally efficient method, the qualities of the estimators become relatively comparable with a slight advantage for the residual variance estimator.