On the Incommensurability Phenomenon

TL;DR

This paper investigates the incommensurability phenomenon in PCA-reduced data sets, quantifying how large fitting errors can occur between noisy measurements of the same process, with implications demonstrated through simulations and real data.

Contribution

It provides a theoretical framework to quantify the incommensurability phenomenon in PCA, linking fitting error to subspace distances and correlation parameters.

Findings

Procrustean fitting-error relates to subspace Hausdorff distance and correlation.

The phenomenon can significantly impact data analysis in practice.

Simulations and real data illustrate the effect's magnitude.

Abstract

Suppose that two large, multi-dimensional data sets are each noisy measurements of the same underlying random process, and principle components analysis is performed separately on the data sets to reduce their dimensionality. In some circumstances it may happen that the two lower-dimensional data sets have an inordinately large Procrustean fitting-error between them. The purpose of this manuscript is to quantify this "incommensurability phenomenon." In particular, under specified conditions, the square Procrustean fitting-error of the two normalized lower-dimensional data sets is (asymptotically) a convex combination (via a correlation parameter) of the Hausdorff distance between the projection subspaces and the maximum possible value of the square Procrustean fitting-error for normalized data. We show how this gives rise to the incommensurability phenomenon, and we employ illustrative simulations as well as a real data experiment to explore how the incommensurability phenomenon may have an appreciable impact.