Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions

TL;DR

This paper analyzes multi-modal inference in high dimensions, deriving optimal performance thresholds and proposing an AMP algorithm that outperforms traditional methods like PLS and CCA in recovering correlated latent structures.

Contribution

It introduces a Bayesian optimal framework and AMP algorithm for multi-modal high-dimensional data, providing theoretical performance bounds and practical improvements.

Findings

AMP achieves near-optimal recovery thresholds.

Traditional methods like PLS and CCA are sub-optimal.

The analysis applies to various priors and noise models.

Abstract

This work explores multi-modal inference in a high-dimensional simplified model, analytically quantifying the performance gain of multi-modal inference over that of analyzing modalities in isolation. We present the Bayes-optimal performance and recovery thresholds in a model where the objective is to recover the latent structures from two noisy data matrices with correlated spikes. The paper derives the approximate message passing (AMP) algorithm for this model and characterizes its performance in the high-dimensional limit via the associated state evolution. The analysis holds for a broad range of priors and noise channels, which can differ across modalities. The linearization of AMP is compared numerically to the widely used partial least squares (PLS) and canonical correlation analysis (CCA) methods, which are both observed to suffer from a sub-optimal recovery threshold.