The factor paradox: Common factors can be correlated with the variance not accounted for by the common factors!

TL;DR

This paper demonstrates that in factor models, unaccounted covariances can be correlated with common factors, challenging assumptions of independence and impacting model interpretation.

Contribution

It reveals that residual covariances and errors can be correlated with common factors, highlighting a potential flaw in traditional factor analysis assumptions.

Findings

Principal components of unaccounted variance can correlate with common factors

Simulation confirms non-zero correlations between residuals and factors

Implications for interpreting factor models with residual variance

Abstract

The case that the factor model does not account for all the covariances of the observed variables is considered. This is a quite realistic condition because some model error as well as some sampling error should usually occur with empirical data. It is shown that principal components representing covariances not accounted for by the factors of the model can have a non-zero correlation with the common factors of the factor model. Non-zero correlations of components representing variance not accounted for by the factor model with common factors were also found in a simulation study. Based on these results it should be concluded that common factors can be correlated with variance components representing model error as well as sampling error. In consequence, even when researchers decide not to represent some small or trivial variance by means of a common factor, these excluded variances can still be part of the model.