Robust Gradient Descent Estimation for Tensor Models under Heavy-Tailed Distributions

TL;DR

This paper introduces a robust truncated gradient descent method for low-rank tensor models that effectively handles heavy-tailed distributions, providing both theoretical guarantees and practical performance improvements.

Contribution

It develops a novel robust estimation procedure for tensor models under heavy-tailed data, with proven convergence and optimal statistical rates.

Findings

Method achieves optimal statistical error rates under heavy-tailed distributions.

Convergence of the proposed algorithm is theoretically established.

Numerical experiments confirm the method's effectiveness.

Abstract

Low-rank tensor models are widely used in statistics. However, most existing methods rely heavily on the assumption that data follows a sub-Gaussian distribution. To address the challenges associated with heavy-tailed distributions encountered in real-world applications, we propose a novel robust estimation procedure based on truncated gradient descent for general low-rank tensor models. We establish the computational convergence of the proposed method and derive optimal statistical rates under heavy-tailed distributional settings of both covariates and noise for various low-rank models. Notably, the statistical error rates are governed by a local moment condition, which captures the distributional properties of tensor variables projected onto certain low-dimensional local regions. Furthermore, we present numerical results to demonstrate the effectiveness of our method.