Bayesian Regularization for Graphical Models with Unequal Shrinkage

TL;DR

This paper introduces a Bayesian approach with adaptive Laplace priors for estimating high-dimensional sparse precision matrices, proposing a non-convex penalty and an EM algorithm for efficient computation, with proven error and selection consistency.

Contribution

It develops a novel Bayesian framework with a non-convex penalty for sparse precision matrix estimation, along with an EM algorithm for practical implementation and theoretical guarantees.

Findings

Optimal estimation error rates established.

Selection consistency for sparse structure recovery proven.

Method outperforms existing approaches in simulations and real data.

Abstract

We consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likelihood perspective that gives rise to a new non-convex penalty approximating the $\ell_0$ penalty. Optimal error rates for estimation consistency in terms of various matrix norms along with selection consistency for sparse structure recovery are shown for the unique MAP estimator under mild conditions. For fast and efficient computation, an EM algorithm is proposed to compute the MAP estimator of the precision matrix and (approximate) posterior probabilities on the edges of the underlying sparse structure. Through extensive simulation studies and a real application to a call center data, we have demonstrated the fine performance of our method compared with existing alternatives.