Langevin dynamics for high-dimensional optimization: the case of multi-spiked tensor PCA

TL;DR

This paper analyzes the efficiency of Langevin dynamics in high-dimensional multi-spiked tensor PCA, identifying sample complexity thresholds for exact recovery of signals and revealing differences based on tensor order.

Contribution

It provides a detailed characterization of Langevin dynamics for multi-spiked tensor PCA, including sample complexity thresholds and the impact of tensor order on recovery.

Findings

Sample complexity matches known thresholds for single-spike recovery.

Recovery thresholds degrade when estimating multiple spikes.

Detailed analysis of high-dimensional Langevin dynamics trajectories.

Abstract

We study nonconvex optimization in high dimensions through Langevin dynamics, focusing on the multi-spiked tensor PCA problem. This tensor estimation problem involves recovering $r$ hidden signal vectors (spikes) from noisy Gaussian tensor observations using maximum likelihood estimation. We study the number of samples required for Langevin dynamics to efficiently recover the spikes and determine the necessary separation condition on the signal-to-noise ratios (SNRs) for exact recovery, distinguishing the cases $p \ge 3$ and $p=2$, where $p$ denotes the order of the tensor. In particular, we show that the sample complexity required for recovering the spike associated with the largest SNR matches the well-known algorithmic threshold for the single-spike case, while this threshold degrades when recovering all $r$ spikes. As a key step, we provide a detailed characterization of the trajectory and interactions of low-dimensional projections that capture the high-dimensional dynamics.