GraphSPME: Markov Precision Matrix Estimation and Asymptotic Stein-Type Shrinkage

TL;DR

GraphSPME introduces a scalable, open-source method for non-parametric sparse precision matrix estimation that combines Markov properties with Stein-type shrinkage, enabling accurate, stable, and high-dimensional covariance estimation.

Contribution

It presents a novel algorithm for determining the optimal Markov order and integrates Stein-type shrinkage for improved precision matrix estimation in high dimensions.

Findings

Efficient estimation in datasets with p >> n.

Automatic determination of Markov order.

Scalable to dimensions around 10^7.

Abstract

GraphSPME is an open source Python, R and C++ header-only package implement-ing non-parametric sparse precision matrix estimation along with asymptotic Stein-type shrinkage estimation of the covariance matrix. The user defines a potential neighbourhood structure and provides data that potentially are p >> n. This paper introduces a novel approach for finding the optimal order (that data allows to estimate) of a potential Markov property. The algorithm is implemented in the package, alleviating the problem of users making Markov assumptions and implementing corresponding complex higher-order neighbourhood structures. Estimation is made accurate and stable by simultaneously utilising both Markov properties and Stein-type shrinkage. Asymptotic results on Stein-type shrinkage ensure that non-singular well conditioned matrices are obtained in an automatic manner. Final symmetry conversion creates symmetric positive definite estimates. Furthermore, the estimation routine is made efficient and scalable to very high-dimensional problems (~10^7) by utilising the sparse nature of the precision matrix under Markov assumptions. Implementation wise, the sparsity is exploited by employing the sparsity possibilities made available by the Eigen C++ linear-algebra library. The package and examples are available at https://github.com/equinor/GraphSPME