Optimal whitening and decorrelation

TL;DR

This paper reviews various whitening methods, introduces a theoretical framework to identify optimal transformations, and recommends two approaches—ZCA-cor and PCA-cor—for specific data preprocessing goals.

Contribution

It provides a theoretical analysis of whitening procedures and proposes criteria to select optimal transformations based on cross-covariance and cross-correlation.

Findings

ZCA-cor whitening preserves maximal similarity to original variables

PCA-cor whitening achieves maximal data compression

Theoretical framework breaks rotational invariance in whitening methods

Abstract

Whitening, or sphering, is a common preprocessing step in statistical analysis to transform random variables to orthogonality. However, due to rotational freedom there are infinitely many possible whitening procedures. Consequently, there is a diverse range of sphering methods in use, for example based on principal component analysis (PCA), Cholesky matrix decomposition and zero-phase component analysis (ZCA), among others. Here we provide an overview of the underlying theory and discuss five natural whitening procedures. Subsequently, we demonstrate that investigating the cross-covariance and the cross-correlation matrix between sphered and original variables allows to break the rotational invariance and to identify optimal whitening transformations. As a result we recommend two particular approaches: ZCA-cor whitening to produce sphered variables that are maximally similar to the original variables, and PCA-cor whitening to obtain sphered variables that maximally compress the original variables.