Bayesian Variable Selection and Sparse Estimation for High-Dimensional Graphical Models

TL;DR

This paper proposes a Bayesian method for simultaneous covariate selection and sparse estimation of multiple structures in high-dimensional Gaussian graphical models, improving interpretability and accuracy.

Contribution

It introduces a novel hierarchical Bayesian approach that achieves joint sparse estimation of regression, response dependencies, and covariate effects, unlike existing methods.

Findings

Method performs well with weak signals in simulations.

Efficient EM algorithm for estimation.

Demonstrated predictive accuracy on bike-share data.

Abstract

We introduce a novel Bayesian approach for both covariate selection and sparse precision matrix estimation in the context of high-dimensional Gaussian graphical models involving multiple responses. Our approach provides a sparse estimation of the three distinct sparsity structures: the regression coefficient matrix, the conditional dependency structure among responses, and between responses and covariates. This contrasts with existing methods, which typically focus on any two of these structures but seldom achieve simultaneous sparse estimation for all three. A key aspect of our method is that it leverages the structural sparsity information gained from the presence of irrelevant covariates in the dataset to introduce covariate-level sparsity in the precision and regression coefficient matrices. This is achieved through a Bayesian conditional random field model using a hierarchical spike and slab prior setup. Despite the non-convex nature of the problem, we establish statistical accuracy for points in the high posterior density region, including the maximum-a-posteriori (MAP) estimator. We also present an efficient Expectation-Maximization (EM) algorithm for computing the estimators. Through simulation experiments, we demonstrate the competitive performance of our method, particularly in scenarios with weak signal strength in the precision matrices. Finally, we apply our method to a bike-share dataset, showcasing its predictive performance.