A Shrinkage Principle for Heavy-Tailed Data: High-Dimensional Robust Low-Rank Matrix Recovery

TL;DR

This paper proposes a shrinkage-based approach for robust high-dimensional matrix recovery that works under only bounded moment conditions, broadening the applicability of existing methods.

Contribution

It introduces a simple shrinkage principle for robust estimation in high-dimensional settings, reducing moment conditions and unifying several problems like matrix completion and regression.

Findings

Achieves optimal statistical error rates under minimal moment assumptions.

Provides a robust covariance matrix estimator with spectral norm concentration.

Demonstrates effectiveness through extensive simulations.

Abstract

This paper introduces a simple principle for robust high-dimensional statistical inference via an appropriate shrinkage on the data. This widens the scope of high-dimensional techniques, reducing the moment conditions from sub-exponential or sub-Gaussian distributions to merely bounded second or fourth moment. As an illustration of this principle, we focus on robust estimation of the low-rank matrix $\Theta^*$ from the trace regression model $Y=Tr (\Theta^{*T}X) +\epsilon$. It encompasses four popular problems: sparse linear models, compressed sensing, matrix completion and multi-task regression. We propose to apply penalized least-squares approach to appropriately truncated or shrunk data. Under only bounded $2+\delta$ moment condition on the response, the proposed robust methodology yields an estimator that possesses the same statistical error rates as previous literature with sub-Gaussian errors. For sparse linear models and multi-tasking regression, we further allow the design to have only bounded fourth moment and obtain the same statistical rates, again, by appropriate shrinkage of the design matrix. As a byproduct, we give a robust covariance matrix estimator and establish its concentration inequality in terms of the spectral norm when the random samples have only bounded fourth moment. Extensive simulations have been carried out to support our theories.