Robust Tensor Regression with Nonconvexity: Algorithmic and Statistical Theory

TL;DR

This paper introduces a robust tensor regression method that handles outliers and heavy-tailed noise using nonconvex optimization, with proven convergence and statistical guarantees.

Contribution

It proposes a novel nonconvex tensor regression framework with an efficient algorithm and theoretical analysis applicable to various models and loss functions.

Findings

Algorithm converges globally under mild conditions.

Theoretical bounds on prediction error and convergence rates.

Numerical experiments validate the method's robustness and effectiveness.

Abstract

Tensor regression is an important tool for tensor data analysis, but existing works have not considered the impact of outliers, making them potentially sensitive to such data points. This paper proposes a low tubal rank robust regression method for analyzing high-dimensional tensor data with heavy-tailed random noise. The proposed method is based on a nonconvex relaxation of the tensor tubal rank within a general optimization framework, which allows for nonconvexity in both the loss and penalty functions. We develop an implementable estimation algorithm and establish its global convergence under some mild assumptions. Furthermore, we provide general statistical theories regarding stationary point, including the rates of convergence and bounds on the prediction error. These theoretical results cover many important models, such as linear models, generalized linear models, and Huber regression, and even encompass some nonconvex losses like correntropy and minimum distance criterion-induced losses. Supportive numerical evidence is provided through simulations and application studies.