Estimation of High-Dimensional Graphical Models Using Regularized Score Matching

TL;DR

This paper introduces a regularized score matching method for estimating high-dimensional graphical models, offering computational efficiency and improved accuracy, especially in non-Gaussian and sparse settings.

Contribution

It proposes a novel regularized score matching approach for high-dimensional graphical model estimation that is computationally efficient and effective for non-Gaussian data.

Findings

Achieves state-of-the-art performance in Gaussian graphical models.

Provides a computationally efficient alternative for non-Gaussian models.

Ensures consistency in high-dimensional sparse settings.

Abstract

Graphical models are widely used to model stochastic dependences among large collections of variables. We introduce a new method of estimating undirected conditional independence graphs based on the score matching loss, introduced by Hyvarinen (2005), and subsequently extended in Hyvarinen (2007). The regularized score matching method we propose applies to settings with continuous observations and allows for computationally efficient treatment of possibly non-Gaussian exponential family models. In the well-explored Gaussian setting, regularized score matching avoids issues of asymmetry that arise when applying the technique of neighborhood selection, and compared to existing methods that directly yield symmetric estimates, the score matching approach has the advantage that the considered loss is quadratic and gives piecewise linear solution paths under $\ell_1$ regularization. Under suitable irrepresentability conditions, we show that $\ell_1$-regularized score matching is consistent for graph estimation in sparse high-dimensional settings. Through numerical experiments and an application to RNAseq data, we confirm that regularized score matching achieves state-of-the-art performance in the Gaussian case and provides a valuable tool for computationally efficient estimation in non-Gaussian graphical models.