Fast Variational Inference for Bayesian Factor Analysis in Single and Multi-Study Settings

TL;DR

This paper introduces fast variational inference algorithms for Bayesian factor analysis, significantly reducing computation time and memory usage while maintaining accuracy, especially useful for high-dimensional multi-study data like gene expression in ovarian cancer.

Contribution

The paper presents novel variational inference algorithms for Bayesian factor models with a gamma process prior, improving scalability over traditional MCMC methods.

Findings

Algorithms are faster and more memory-efficient than MCMC.

Maintain comparable accuracy in covariance estimation.

Effective in high-dimensional multi-study gene expression analysis.

Abstract

Factors models are routinely used to analyze high-dimensional data in both single-study and multi-study settings. Bayesian inference for such models relies on Markov Chain Monte Carlo (MCMC) methods which scale poorly as the number of studies, observations, or measured variables increase. To address this issue, we propose variational inference algorithms to approximate the posterior distribution of Bayesian latent factor models using the multiplicative gamma process shrinkage prior. The proposed algorithms provide fast approximate inference at a fraction of the time and memory of MCMC-based implementations while maintaining comparable accuracy in characterizing the data covariance matrix. We conduct extensive simulations to evaluate our proposed algorithms and show their utility in estimating the model for high-dimensional multi-study gene expression data in ovarian cancers. Overall, our proposed approaches enable more efficient and scalable inference for factor models, facilitating their use in high-dimensional settings. An R package VIMSFA implementing our methods is available on GitHub (github.com/blhansen/VI-MSFA).