On the Correlation between Random Variables and their Principal Components

TL;DR

This paper derives an algebraic formula linking random variables to their principal components, showing its application in optimizing component and factor selection in PCA and Factor Analysis.

Contribution

It presents a novel algebraic formula for correlation coefficients between random variables and principal components, connecting PCA and Factor Analysis.

Findings

Derived a formula identical to factor loadings in Factor Analysis.

Showed the formula's application in optimizing component and factor numbers.

Connected linear algebra concepts with statistical analysis methods.

Abstract

The article attempts to find an algebraic formula describing the correlation coefficients between random variables and the principal components representing them. As a result of the analysis, starting from selected statistics relating to individual random variables, the equivalents of these statistics relating to a set of random variables were presented in the language of linear algebra, using the concepts of vector and matrix. This made it possible, in subsequent steps, to derive the expected formula. The formula found is identical to the formula used in Factor Analysis to calculate factor loadings. The discussion showed that it is possible to apply this formula to optimize the number of principal components in Principal Component Analysis, as well as to optimize the number of factors in Factor Analysis.