Bayesian Low Rank and Sparse Covariance Matrix Decomposition

TL;DR

This paper introduces a Bayesian approach for estimating high-dimensional covariance matrices that are sums of low rank and sparse matrices, with applications in factor analysis and graphical models.

Contribution

It proposes a novel Bayesian method combining factor models, Bayesian lasso, and binary indicators for rank and sparsity estimation, extending to graphical models.

Findings

Successfully recovers rank and sparsity in simulations.

Accurately estimates the number of latent factors.

Recovers residual graphical models effectively.

Abstract

We consider the problem of estimating high-dimensional covariance matrices of a particular structure, which is a summation of low rank and sparse matrices. This covariance structure has a wide range of applications including factor analysis and random effects models. We propose a Bayesian method of estimating the covariance matrices by representing the covariance model in the form of a factor model with unknown number of latent factors. We introduce binary indicators for factor selection and rank estimation for the low rank component combined with a Bayesian lasso method for the sparse component estimation. Simulation studies show that our method can recover the rank as well as the sparsity of the two components respectively. We further extend our method to a graphical factor model where the graphical model of the residuals as well as selecting the number of factors is of interest. We employ a hyper-inverse Wishart prior for modeling decomposable graphs of the residuals, and a Bayesian graphical lasso selection method for unrestricted graphs. We show through simulations that the extended models can recover both the number of latent factors and the graphical model of the residuals successfully when the sample size is sufficient relative to the dimension.