Quantile and pseudo-Huber Tensor Decomposition

TL;DR

This paper introduces a robust tensor decomposition method using quantile and pseudo-Huber losses, achieving optimal statistical rates under heavy-tailed noise and corruptions, with practical applications demonstrated on real datasets.

Contribution

It proposes a novel projected sub-gradient descent algorithm for robust tensor decomposition that handles non-smooth losses and achieves minimax optimal rates under contamination models.

Findings

Algorithm converges linearly with two-phase convergence.

Achieves minimax optimal rates under Huber's contamination.

Effective in handling missing data and real-world datasets.

Abstract

This paper studies the computational and statistical aspects of quantile and pseudo-Huber tensor decomposition. The integrated investigation of computational and statistical issues of robust tensor decomposition poses challenges due to the non-smooth loss functions. We propose a projected sub-gradient descent algorithm for tensor decomposition, equipped with either the pseudo-Huber loss or the quantile loss. In the presence of both heavy-tailed noise and Huber's contamination error, we demonstrate that our algorithm exhibits a so-called phenomenon of two-phase convergence with a carefully chosen step size schedule. The algorithm converges linearly and delivers an estimator that is statistically optimal with respect to both the heavy-tailed noise and arbitrary corruptions. Interestingly, our results achieve the first minimax optimal rates under Huber's contamination model for noisy tensor decomposition. Compared with existing literature, quantile tensor decomposition removes the requirement of specifying a sparsity level in advance, making it more flexible for practical use. We also demonstrate the effectiveness of our algorithms in the presence of missing values. Our methods are subsequently applied to the food balance dataset and the international trade flow dataset, both of which yield intriguing findings.