A Bayesian Generalized Bridge Regression Approach to Covariance Estimation in the Presence of Covariates

TL;DR

This paper introduces a Bayesian generalized bridge regression method for joint estimation of regression coefficients and the inverse covariance matrix in multivariate linear models, effectively handling large-scale spectral data.

Contribution

It proposes a novel hierarchical Bayesian framework with a generalized bridge prior, enabling efficient estimation of covariance structures with covariates, suitable for large datasets.

Findings

Competitive performance in precision matrix estimation

Effective in both sparse and dense settings

Demonstrated scalability on large spectral data

Abstract

A hierarchical Bayesian approach that permits simultaneous inference for the regression coefficient matrix and the error precision (inverse covariance) matrix in the multivariate linear model is proposed. Assuming a natural ordering of the elements of the response, the precision matrix is reparameterized so it can be estimated with univariate-response linear regression techniques. A novel generalized bridge regression prior that accommodates both sparse and dense settings and is competitive with alternative methods for univariate-response regression is proposed and used in this framework. Two component-wise Markov chain Monte Carlo algorithms are developed for sampling, including a data augmentation algorithm based on a scale mixture of normals representation. Numerical examples demonstrate that the proposed method is competitive with comparable joint mean-covariance models, particularly in estimation of the precision matrix. The method is also used to estimate the 253 by 253 precision matrix of 90,670 spectra extracted from images taken by the Hubble Space Telescope, demonstrating its computational feasibility for problems with large n and q.