Decoupling Shrinkage and Selection in Gaussian Linear Factor Analysis

TL;DR

This paper introduces a decision-theoretic method to decouple the determination of the number of factors from the sparsity of loadings in Gaussian factor analysis, enhancing interpretability and model selection.

Contribution

It presents a novel three-step approach linking sparse loadings to factor dimension through posterior summaries, improving factor analysis flexibility.

Findings

Effective in classical factor analysis datasets

Flexible with different priors and factor dimensions

Provides clear visualization of utility degradation

Abstract

Factor Analysis is a popular method for modeling dependence in multivariate data. However, determining the number of factors and obtaining a sparse orientation of the loadings are still major challenges. In this paper, we propose a decision-theoretic approach that brings to light the relation between a sparse representation of the loadings and factor dimension. This relation is done through a summary from information contained in the multivariate posterior. To construct such summary, we introduce a three-step approach. In the first step, the model is fitted with a conservative factor dimension. In the second step, a series of sparse point-estimates, with a decreasing number of factors, is obtained by minimizing an expected predictive loss function. In step three, the degradation in utility in relation to the sparse loadings and factor dimensions is displayed in the posterior summary. The findings are illustrated with applications in classical data from the Factor Analysis literature. We used different prior choices and factor dimensions to demonstrate the flexibility of the proposed method.