Missing-Data-Induced Phase Transitions in Spectral PLS for Multimodal Learning

TL;DR

This paper analyzes how missing data affects spectral PLS in high-dimensional multimodal learning, revealing a phase transition in the ability to recover shared structure depending on signal strength and missingness.

Contribution

It introduces a theoretical framework predicting a sharp phase transition in spectral PLS performance under missing data, with explicit formulas for the transition threshold.

Findings

Identifies a missing-data-induced phase transition in spectral PLS.

Provides closed-form formulas for the critical signal-to-noise ratio.

Validates predictions with simulations and semi-synthetic experiments.

Abstract

Partial Least Squares (PLS) learns shared structure from paired data via the top singular vectors of the empirical cross-covariance (PLS-SVD), but multimodal datasets often have missing entries in both views. We study PLS-SVD under independent entry-wise missing-completely-at-random masking in a proportional high-dimensional spiked model. After appropriate normalization, the masked cross-covariance behaves like a spiked rectangular random matrix whose effective signal strength is attenuated by $\sqrt{\rho}$, where $\rho$ is the joint entry retention probability. The replica-symmetric analysis predicts a sharp BBP-type phase transition: below a critical signal-to-noise threshold the leading singular vectors are asymptotically uninformative, while above it they achieve nontrivial alignment with the latent shared directions, with closed-form asymptotic overlap formulas. We also state a finite-rank extension as a conjecture, predicting that the same missingness-adjusted threshold applies componentwise when the latent spikes are separated. Simulations and semi-synthetic multimodal experiments agree with the predicted phase diagram and recovery curves across aspect ratios, signal strengths, and missingness levels.