Targeted empirical Bayes for more supervised joint factor analysis

TL;DR

This paper introduces a targeted empirical Bayes approach to improve joint factor analysis models, especially when the response isn't aligned with the main predictor components, enhancing predictive accuracy.

Contribution

It proposes a novel empirical Bayes method for residual variance estimation that improves the predictive performance of joint Bayesian factor models.

Findings

Significant simulation improvements with the new method

Enhanced prediction accuracy in environmental health data

Effective in cases where response isn't aligned with principal components

Abstract

Joint Bayesian factor models are popular for characterizing relationships between multivariate correlated predictors and a response variable. Standard models assume that all variables, including both the predictors and the response, are conditionally independent given latent factors. In marginalizing out these factors, one obtains a low rank plus diagonal factorization for the joint covariance, which implies a linear regression for the response given the predictors. Although there are many desirable properties of such models, these methods can struggle to identify the signal when the response is not dependent on the dominant principal components in the predictors. To address this problem, we propose estimating the residual variance in the response model with an empirical Bayes procedure that targets predictive performance of the response given the predictors. We illustrate that this can lead to substantial improvements in simulation performance. We are particularly motivated by studies assessing the health effects of environmental exposures and provide an illustrative application to NHANES data.