On Estimating Rank-One Spiked Tensors in the Presence of Heavy Tailed Errors

TL;DR

This paper investigates the estimation of rank-one spiked tensors under heavy tailed noise, revealing how noise moments affect statistical and computational tradeoffs, and proposing practical estimation methods.

Contribution

It characterizes the spectral norm of random tensors with heavy tailed entries and introduces efficient estimation procedures tailored for heavy tailed noise regimes.

Findings

Tradeoff between statistical and computational efficiency depends on noise moments.

Tensor SVD is suboptimal under heavy tailed noise with less than fourth moment.

Spectral norm of random tensors can be precisely characterized by entry moments.

Abstract

In this paper, we study the estimation of a rank-one spiked tensor in the presence of heavy tailed noise. Our results highlight some of the fundamental similarities and differences in the tradeoff between statistical and computational efficiencies under heavy tailed and Gaussian noise. In particular, we show that, for $p$ th order tensors, the tradeoff manifests in an identical fashion as the Gaussian case when the noise has finite $4(p-1)$ th moment. The difference in signal strength requirements, with or without computational constraints, for us to estimate the singular vectors at the optimal rate, interestingly, narrows for noise with heavier tails and vanishes when the noise only has finite fourth moment. Moreover, if the noise has less than fourth moment, tensor SVD, perhaps the most natural approach, is suboptimal even though it is computationally intractable. Our analysis exploits a close connection between estimating the rank-one spikes and the spectral norm of a random tensor with iid entries. In particular, we show that the order of the spectral norm of a random tensor can be precisely characterized by the moment of its entries, generalizing classical results for random matrices. In addition to the theoretical guarantees, we propose estimation procedures for the heavy tailed regime, which are easy to implement and efficient to run. Numerical experiments are presented to demonstrate their practical merits.