Why (and How) Avoid Orthogonal Procrustes in Regularized Multivariate Analysis

TL;DR

This paper critiques the use of orthogonal Procrustes in regularized multivariate analysis, proposing an eigenvalue-based alternative that better preserves feature uncorrelation and improves interpretability.

Contribution

It introduces a novel eigenvalue-based approach to replace Procrustes in regularized MVA, ensuring uncorrelation and interpretability of features.

Findings

Eigenvalue approach outperforms Procrustes in preserving uncorrelation

The method maintains key properties of original MVA techniques

Experimental results demonstrate improved interpretability

Abstract

Multivariate Analysis (MVA) comprises a family of well-known methods for feature extraction that exploit correlations among input variables of the data representation. One important property that is enjoyed by most such methods is uncorrelation among the extracted features. Recently, regularized versions of MVA methods have appeared in the literature, mainly with the goal to gain interpretability of the solution. In these cases, the solutions can no longer be obtained in a closed manner, and it is frequent to recur to the iteration of two steps, one of them being an orthogonal Procrustes problem. This letter shows that the Procrustes solution is not optimal from the perspective of the overall MVA method, and proposes an alternative approach based on the solution of an eigenvalue problem. Our method ensures the preservation of several properties of the original methods, most notably the uncorrelation of the extracted features, as demonstrated theoretically and through a collection of selected experiments.