Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model

TL;DR

This paper investigates how correlated multi-modal high-dimensional signals can be efficiently recovered by leveraging their structure, revealing phase transitions, limitations of naive methods, and proposing a sequential curriculum learning approach for optimal recovery.

Contribution

It introduces a new analysis of multi-modal spiked matrix-tensor models, demonstrating the effectiveness of sequential learning strategies over joint optimization in high-dimensional inference.

Findings

Correlation enables efficient Bayesian recovery via AMP.

Naive joint optimization fails due to tensor interference.

Sequential curriculum learning achieves optimal recovery thresholds.

Abstract

We study the recovery of multiple high-dimensional signals from two noisy, correlated modalities: a spiked matrix and a spiked tensor sharing a common low-rank structure. This setting generalizes classical spiked matrix and tensor models, unveiling intricate interactions between inference channels and surprising algorithmic behaviors. Notably, while the spiked tensor model is typically intractable at low signal-to-noise ratios, its correlation with the matrix enables efficient recovery via Bayesian Approximate Message Passing, inducing staircase-like phase transitions reminiscent of neural network phenomena. In contrast, empirical risk minimization for joint learning fails: the tensor component obstructs effective matrix recovery, and joint optimization significantly degrades performance, highlighting the limitations of naive multi-modal learning. We show that a simple Sequential Curriculum Learning strategy-first recovering the matrix, then leveraging it to guide tensor recovery-resolves this bottleneck and achieves optimal weak recovery thresholds. This strategy, implementable with spectral methods, emphasizes the critical role of structural correlation and learning order in multi-modal high-dimensional inference.