High-Dimensional Partial Least Squares: Spectral Analysis and Fundamental Limitations

TL;DR

This paper provides a comprehensive theoretical analysis of high-dimensional Partial Least Squares (PLS), revealing its behavior, limitations, and superiority over PCA in certain regimes through random matrix theory.

Contribution

It offers the first rigorous asymptotic characterization of high-dimensional PLS-SVD, explaining its performance and limitations in data integration tasks.

Findings

PLS-SVD's alignment with true latent directions is characterized asymptotically.

In certain regimes, PLS-SVD outperforms PCA in detecting shared latent structures.

The analysis uncovers regimes where PLS exhibits counter-intuitive behavior.

Abstract

Partial Least Squares (PLS) is a widely used method for data integration, designed to extract latent components shared across paired high-dimensional datasets. Despite decades of practical success, a precise theoretical understanding of its behavior in high-dimensional regimes remains limited. In this paper, we study a data integration model in which two high-dimensional data matrices share a low-rank common latent structure while also containing individual-specific components. We analyze the singular vectors of the associated cross-covariance matrix using tools from random matrix theory and derive asymptotic characterizations of the alignment between estimated and true latent directions. These results provide a quantitative explanation of the reconstruction performance of the PLS variant based on Singular Value Decomposition (PLS-SVD) and identify regimes where the method exhibits counter-intuitive or limiting behavior. Building on this analysis, we compare PLS-SVD with principal component analysis applied separately to each dataset and show its asymptotic superiority in detecting the common latent subspace. Overall, our results offer a comprehensive theoretical understanding of high-dimensional PLS-SVD, clarifying both its advantages and fundamental limitations.