On Posterior Consistency of Bayesian Factor Models in High Dimensions

TL;DR

This paper investigates the conditions under which Bayesian factor models remain consistent in high-dimensional settings, emphasizing the importance of prior choices and orthogonal assumptions for reliable posterior inference.

Contribution

It demonstrates the validity of orthogonal factor assumptions for consistent Bayesian inference and offers an efficient Gibbs sampler for high-dimensional factor models.

Findings

Orthogonal factor assumption ensures posterior consistency.

Proposed Gibbs sampler facilitates practical inference.

Analysis of noninformative priors in high dimensions.

Abstract

As a principled dimension reduction technique, factor models have been widely adopted in social science, economics, bioinformatics, and many other fields. However, in high-dimensional settings, conducting a 'correct' Bayesianfactor analysis can be subtle since it requires both a careful prescription of the prior distribution and a suitable computational strategy. In particular, we analyze the issues related to the attempt of being "noninformative" for elements of the factor loading matrix, especially for sparse Bayesian factor models in high dimensions, and propose solutions to them. We show here why adopting the orthogonal factor assumption is appropriate and can result in a consistent posterior inference of the loading matrix conditional on the true idiosyncratic variance and the allocation of nonzero elements in the true loading matrix. We also provide an efficient Gibbs sampler to conduct the full posterior inference based on the prior setup from Rockova and George (2016)and a uniform orthogonal factor assumption on the factor matrix.